2023-10-26T06:15:13.459 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-26T06:15:13.460 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} --owner scheduled_pdonnell@teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471 -- /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/orig.config.yaml 2023-10-26T06:15:13.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-26T06:15:13.621 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471 branch: wip-batrick-testing-20231024.144545 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '7438471' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20231024.144545 ceph: cephfs: session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 384993681abceb8cb8affea10538d607052db4b4 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 384993681abceb8cb8affea10538d607052db4b4 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231024.144545 sha1: 384993681abceb8cb8affea10538d607052db4b4 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7595 sha1: 384993681abceb8cb8affea10538d607052db4b4 sleep_before_teardown: 0 subset: 66/256 suite: fs suite_branch: wip-batrick-testing-20231024.144545 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 384993681abceb8cb8affea10538d607052db4b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 202b180cb047e798fb131047314a862593f45403 timestamp: 2023-10-26_05:21:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.770726 2023-10-26T06:15:13.622 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa; will attempt to use it 2023-10-26T06:15:13.623 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks 2023-10-26T06:15:13.623 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-26T06:15:13.625 INFO:teuthology.task.internal:Checking packages... 2023-10-26T06:15:13.648 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '384993681abceb8cb8affea10538d607052db4b4' 2023-10-26T06:15:13.649 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-26T06:15:13.649 INFO:teuthology.packaging:ref: None 2023-10-26T06:15:13.650 INFO:teuthology.packaging:tag: None 2023-10-26T06:15:13.650 INFO:teuthology.packaging:branch: wip-batrick-testing-20231024.144545 2023-10-26T06:15:13.650 INFO:teuthology.packaging:sha1: 384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:15:13.651 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20231024.144545 2023-10-26T06:15:13.810 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6863.g38499368 2023-10-26T06:15:13.810 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-26T06:15:13.819 INFO:teuthology.task.internal:no buildpackages task found 2023-10-26T06:15:13.819 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-26T06:15:13.848 INFO:teuthology.task.internal:Saving configuration 2023-10-26T06:15:13.869 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-26T06:15:13.916 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T06:15:13.942 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 06:10:14.354363', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-26T06:15:13.967 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 06:10:14.355533', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-26T06:15:13.992 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 06:10:14.352997', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-26T06:15:13.992 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-26T06:15:14.000 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-26T06:15:14.000 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-26T06:15:14.001 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-26T06:15:14.001 INFO:teuthology.run_tasks:Running task console_log... 2023-10-26T06:15:14.141 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcc3b7b7ca0>, signals=[15]) 2023-10-26T06:15:14.142 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-26T06:15:14.150 INFO:teuthology.task.internal:Opening connections... 2023-10-26T06:15:14.151 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T06:15:14.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:15:14.221 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2023-10-26T06:15:14.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:15:14.293 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2023-10-26T06:15:14.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:15:14.360 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-26T06:15:14.368 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-10-26T06:15:14.388 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-10-26T06:15:14.389 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-10-26T06:15:14.446 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux" 2023-10-26T06:15:14.446 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.6 (Ootpa)" 2023-10-26T06:15:14.446 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel" 2023-10-26T06:15:14.446 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora" 2023-10-26T06:15:14.447 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.6" 2023-10-26T06:15:14.447 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-10-26T06:15:14.447 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-26T06:15:14.447 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-10-26T06:15:14.448 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-26T06:15:14.448 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/" 2023-10-26T06:15:14.448 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-26T06:15:14.448 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-26T06:15:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:15:14.449 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-26T06:15:14.449 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-26T06:15:14.449 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-26T06:15:14.449 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-26T06:15:14.450 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-10-26T06:15:14.577 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2023-10-26T06:15:14.598 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2023-10-26T06:15:14.598 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2023-10-26T06:15:14.661 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Red Hat Enterprise Linux" 2023-10-26T06:15:14.661 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="8.6 (Ootpa)" 2023-10-26T06:15:14.661 INFO:teuthology.orchestra.run.smithi135.stdout:ID="rhel" 2023-10-26T06:15:14.662 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="fedora" 2023-10-26T06:15:14.662 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="8.6" 2023-10-26T06:15:14.662 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el8" 2023-10-26T06:15:14.662 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-26T06:15:14.663 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2023-10-26T06:15:14.663 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-26T06:15:14.663 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.redhat.com/" 2023-10-26T06:15:14.663 INFO:teuthology.orchestra.run.smithi135.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-26T06:15:14.663 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-26T06:15:14.664 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:15:14.664 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-26T06:15:14.664 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-26T06:15:14.664 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-26T06:15:14.664 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-26T06:15:14.665 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2023-10-26T06:15:14.690 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2023-10-26T06:15:14.710 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2023-10-26T06:15:14.711 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2023-10-26T06:15:14.770 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Red Hat Enterprise Linux" 2023-10-26T06:15:14.771 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="8.6 (Ootpa)" 2023-10-26T06:15:14.771 INFO:teuthology.orchestra.run.smithi192.stdout:ID="rhel" 2023-10-26T06:15:14.771 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="fedora" 2023-10-26T06:15:14.772 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="8.6" 2023-10-26T06:15:14.772 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el8" 2023-10-26T06:15:14.772 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-26T06:15:14.772 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2023-10-26T06:15:14.773 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-26T06:15:14.773 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.redhat.com/" 2023-10-26T06:15:14.773 INFO:teuthology.orchestra.run.smithi192.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-26T06:15:14.773 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-26T06:15:14.773 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:15:14.774 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-26T06:15:14.774 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-26T06:15:14.774 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-26T06:15:14.774 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-26T06:15:14.775 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2023-10-26T06:15:14.806 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-26T06:15:14.820 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-26T06:15:14.875 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-26T06:15:14.875 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2023-10-26T06:15:14.878 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2023-10-26T06:15:14.881 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2023-10-26T06:15:14.899 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-26T06:15:14.917 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-26T06:15:14.917 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2023-10-26T06:15:14.936 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2023-10-26T06:15:14.943 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2023-10-26T06:15:14.975 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-26T06:15:15.056 INFO:teuthology.run_tasks:Running task kernel... 2023-10-26T06:15:15.076 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-26T06:15:15.076 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-26T06:15:15.077 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-26T06:15:15.077 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-26T06:15:15.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:15:15.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-26T06:15:15.333 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T06:15:15.333 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-10-26T06:15:15.334 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-10-26T06:15:15.353 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-26T06:15:15.354 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T06:15:15.354 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-26T06:15:15.355 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum upgrade -y linux-firmware 2023-10-26T06:15:16.654 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-26T06:15:16.654 INFO:teuthology.orchestra.run.smithi089.stdout:Unable to read consumer identity 2023-10-26T06:15:18.176 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-10-26T06:15:18.893 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-10-26T06:15:22.452 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 430 kB/s | 24 kB 00:00 2023-10-26T06:15:23.178 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-10-26T06:15:23.179 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2023-10-26T06:15:23.179 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-10-26T06:15:23.315 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-10-26T06:15:23.315 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T06:15:23.455 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ 2023-10-26T06:15:23.455 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-26T06:15:23.458 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi089.front.sepia.ceph.com to complete... 2023-10-26T06:15:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26 06:15:36 URL:https://4.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [383129572/383129572] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-26T06:15:36.561 INFO:teuthology.orchestra.run.smithi089.stderr:FINISHED --2023-10-26 06:15:36-- 2023-10-26T06:15:36.562 INFO:teuthology.orchestra.run.smithi089.stderr:Total wall clock time: 13s 2023-10-26T06:15:36.562 INFO:teuthology.orchestra.run.smithi089.stderr:Downloaded: 1 files, 365M in 13s (28.6 MB/s) 2023-10-26T06:15:36.564 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-10-26T06:15:36.564 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-26T06:15:38.188 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying... ######################################## 2023-10-26T06:15:38.251 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing... ######################################## 2023-10-26T06:15:38.337 INFO:teuthology.orchestra.run.smithi089.stdout:Updating / installing... 2023-10-26T06:16:16.749 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-10-26T06:16:18.273 INFO:teuthology.orchestra.run.smithi089.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-26T06:16:37.900 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-26T06:16:37.901 DEBUG:teuthology.orchestra.run.smithi089:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-26T06:16:37.955 INFO:teuthology.orchestra.run.smithi089.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:37.955 INFO:teuthology.orchestra.run.smithi089.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:37.955 INFO:teuthology.orchestra.run.smithi089.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:37.955 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:37.956 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-10-26T06:16:37.956 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-10-26T06:16:37.956 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-10-26T06:16:37.956 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-10-26T06:16:37.956 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-26T06:16:37.957 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-26T06:16:37.957 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-26T06:16:37.957 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-26T06:16:37.957 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-26T06:16:37.957 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-26T06:16:37.958 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-26T06:16:37.958 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-26T06:16:37.958 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-26T06:16:37.958 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-26T06:16:37.958 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-26T06:16:37.959 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-26T06:16:37.959 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-26T06:16:37.959 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-26T06:16:37.959 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-26T06:16:37.959 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-26T06:16:37.959 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-26T06:16:37.960 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-26T06:16:37.960 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-26T06:16:37.960 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-26T06:16:37.960 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-10-26T06:16:37.960 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-10-26T06:16:37.960 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-10-26T06:16:37.961 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-26T06:16:37.961 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-10-26T06:16:37.961 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-26T06:16:37.961 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-26T06:16:37.961 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-10-26T06:16:37.962 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-10-26T06:16:37.962 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-10-26T06:16:37.962 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-10-26T06:16:37.962 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-10-26T06:16:37.962 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-10-26T06:16:37.962 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-10-26T06:16:37.963 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-10-26T06:16:37.963 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-26T06:16:37.963 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-10-26T06:16:37.963 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-10-26T06:16:37.963 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-10-26T06:16:37.964 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-10-26T06:16:37.964 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-10-26T06:16:37.964 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-10-26T06:16:37.964 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-10-26T06:16:37.964 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-10-26T06:16:37.964 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-10-26T06:16:37.965 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-10-26T06:16:37.965 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-10-26T06:16:37.965 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-10-26T06:16:37.965 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-10-26T06:16:37.965 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-10-26T06:16:37.966 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-10-26T06:16:37.966 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-10-26T06:16:37.966 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-10-26T06:16:37.966 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-10-26T06:16:37.966 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-10-26T06:16:37.967 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-10-26T06:16:37.967 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-10-26T06:16:37.967 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-10-26T06:16:37.967 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-10-26T06:16:37.967 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-10-26T06:16:37.967 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-10-26T06:16:37.968 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-10-26T06:16:37.968 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-10-26T06:16:37.968 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-10-26T06:16:37.968 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-10-26T06:16:37.968 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-10-26T06:16:37.969 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-10-26T06:16:37.969 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-10-26T06:16:37.969 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-10-26T06:16:37.969 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-10-26T06:16:37.969 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-10-26T06:16:37.969 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-10-26T06:16:37.970 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-10-26T06:16:37.970 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-10-26T06:16:37.970 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-10-26T06:16:37.970 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-10-26T06:16:37.970 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-10-26T06:16:37.971 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-10-26T06:16:37.971 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-10-26T06:16:37.971 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-10-26T06:16:37.971 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-26T06:16:37.971 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-10-26T06:16:37.971 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-10-26T06:16:37.972 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-10-26T06:16:37.972 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-10-26T06:16:37.972 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-10-26T06:16:37.972 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-10-26T06:16:37.972 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-10-26T06:16:37.973 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-10-26T06:16:37.973 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-10-26T06:16:37.973 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-10-26T06:16:37.973 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-10-26T06:16:37.973 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-10-26T06:16:37.973 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-10-26T06:16:37.974 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-10-26T06:16:37.974 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-10-26T06:16:37.974 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-10-26T06:16:37.974 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-10-26T06:16:37.974 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-10-26T06:16:37.975 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-10-26T06:16:37.975 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-10-26T06:16:37.975 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-10-26T06:16:37.975 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-10-26T06:16:37.975 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-10-26T06:16:37.975 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-10-26T06:16:37.976 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-26T06:16:37.976 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-26T06:16:37.976 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-10-26T06:16:37.976 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-10-26T06:16:37.976 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-10-26T06:16:37.977 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-10-26T06:16:37.977 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-10-26T06:16:37.977 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-10-26T06:16:37.977 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-10-26T06:16:37.977 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-10-26T06:16:37.977 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-10-26T06:16:37.978 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-10-26T06:16:37.978 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-10-26T06:16:37.978 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-10-26T06:16:37.978 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-10-26T06:16:37.978 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-26T06:16:37.979 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-26T06:16:37.979 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-10-26T06:16:37.979 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-10-26T06:16:37.979 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-10-26T06:16:37.979 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-10-26T06:16:37.980 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-10-26T06:16:37.980 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-10-26T06:16:37.980 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-10-26T06:16:37.980 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-10-26T06:16:37.980 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-10-26T06:16:37.981 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-10-26T06:16:37.981 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-10-26T06:16:37.981 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-10-26T06:16:37.981 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-10-26T06:16:37.981 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-10-26T06:16:37.982 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-10-26T06:16:37.982 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-10-26T06:16:37.982 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-10-26T06:16:37.982 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-10-26T06:16:37.982 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-10-26T06:16:37.983 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-10-26T06:16:37.983 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-10-26T06:16:37.983 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-10-26T06:16:37.983 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-10-26T06:16:37.983 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-10-26T06:16:37.983 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-10-26T06:16:37.984 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-10-26T06:16:37.984 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-10-26T06:16:37.984 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-26T06:16:37.984 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-26T06:16:37.984 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-10-26T06:16:37.985 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-10-26T06:16:37.985 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-10-26T06:16:37.985 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-10-26T06:16:37.985 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-10-26T06:16:37.985 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-10-26T06:16:37.986 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-10-26T06:16:37.986 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-10-26T06:16:37.986 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-10-26T06:16:37.986 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-10-26T06:16:37.986 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-10-26T06:16:37.987 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-26T06:16:37.987 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-10-26T06:16:37.987 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-10-26T06:16:37.987 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-10-26T06:16:37.987 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-10-26T06:16:37.987 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-10-26T06:16:37.988 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-10-26T06:16:37.988 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-10-26T06:16:37.988 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-26T06:16:37.988 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-10-26T06:16:37.988 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-10-26T06:16:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-10-26T06:16:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-10-26T06:16:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-10-26T06:16:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-10-26T06:16:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-26T06:16:37.990 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-26T06:16:37.990 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-26T06:16:37.990 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-10-26T06:16:37.990 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-26T06:16:37.990 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-10-26T06:16:37.990 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-26T06:16:37.991 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-26T06:16:37.991 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-26T06:16:37.991 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-26T06:16:37.991 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-26T06:16:37.991 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-26T06:16:37.992 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-10-26T06:16:37.992 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-10-26T06:16:37.992 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-10-26T06:16:37.992 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-10-26T06:16:37.992 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-10-26T06:16:37.992 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-10-26T06:16:37.993 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-10-26T06:16:37.993 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-26T06:16:37.993 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-26T06:16:37.993 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-26T06:16:37.993 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-26T06:16:37.993 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-26T06:16:37.994 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-10-26T06:16:37.994 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-10-26T06:16:37.994 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-10-26T06:16:37.994 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-26T06:16:37.994 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-26T06:16:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-26T06:16:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-26T06:16:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-26T06:16:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-26T06:16:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-10-26T06:16:37.995 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-10-26T06:16:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-26T06:16:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-10-26T06:16:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-10-26T06:16:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-10-26T06:16:37.996 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-10-26T06:16:37.997 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-26T06:16:37.997 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-26T06:16:37.997 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-26T06:16:37.997 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-10-26T06:16:37.997 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-10-26T06:16:37.997 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-10-26T06:16:37.998 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-26T06:16:37.998 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-10-26T06:16:37.998 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-10-26T06:16:37.998 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-10-26T06:16:37.998 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-26T06:16:37.999 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-10-26T06:16:37.999 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-10-26T06:16:37.999 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-26T06:16:37.999 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-10-26T06:16:37.999 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-26T06:16:38.000 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-10-26T06:16:38.000 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-10-26T06:16:38.000 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-10-26T06:16:38.000 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-10-26T06:16:38.000 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-10-26T06:16:38.000 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-10-26T06:16:38.001 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-10-26T06:16:38.001 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-10-26T06:16:38.001 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-10-26T06:16:38.001 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-10-26T06:16:38.001 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-10-26T06:16:38.002 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-10-26T06:16:38.002 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-10-26T06:16:38.002 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-10-26T06:16:38.002 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-10-26T06:16:38.002 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-10-26T06:16:38.003 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-26T06:16:38.003 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-10-26T06:16:38.003 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-10-26T06:16:38.003 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-26T06:16:38.003 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-10-26T06:16:38.004 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-10-26T06:16:38.004 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-10-26T06:16:38.004 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-10-26T06:16:38.004 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-10-26T06:16:38.004 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-10-26T06:16:38.004 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-10-26T06:16:38.005 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-10-26T06:16:38.005 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-10-26T06:16:38.005 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-10-26T06:16:38.005 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-10-26T06:16:38.005 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-26T06:16:38.006 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-26T06:16:38.006 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-10-26T06:16:38.006 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-10-26T06:16:38.006 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-10-26T06:16:38.006 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-10-26T06:16:38.007 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-10-26T06:16:38.007 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-10-26T06:16:38.007 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-10-26T06:16:38.007 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-10-26T06:16:38.007 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-26T06:16:38.007 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-26T06:16:38.008 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-10-26T06:16:38.008 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-10-26T06:16:38.008 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-10-26T06:16:38.008 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-10-26T06:16:38.008 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-10-26T06:16:38.009 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-10-26T06:16:38.009 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-10-26T06:16:38.009 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-10-26T06:16:38.009 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-10-26T06:16:38.009 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-26T06:16:38.009 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-10-26T06:16:38.010 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-26T06:16:38.010 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-26T06:16:38.010 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-26T06:16:38.010 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-26T06:16:38.010 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-26T06:16:38.011 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-26T06:16:38.011 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-26T06:16:38.011 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-26T06:16:38.011 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-26T06:16:38.011 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-10-26T06:16:38.011 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-10-26T06:16:38.012 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-10-26T06:16:38.012 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-26T06:16:38.012 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-10-26T06:16:38.012 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-10-26T06:16:38.012 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-10-26T06:16:38.013 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-10-26T06:16:38.013 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-10-26T06:16:38.013 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-26T06:16:38.013 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-10-26T06:16:38.013 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-10-26T06:16:38.013 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-10-26T06:16:38.014 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-10-26T06:16:38.014 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-10-26T06:16:38.014 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-10-26T06:16:38.014 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-10-26T06:16:38.014 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-10-26T06:16:38.014 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-10-26T06:16:38.015 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-10-26T06:16:38.015 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-10-26T06:16:38.015 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-10-26T06:16:38.015 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-10-26T06:16:38.015 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-10-26T06:16:38.016 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-10-26T06:16:38.016 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-10-26T06:16:38.016 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-10-26T06:16:38.016 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-10-26T06:16:38.016 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-10-26T06:16:38.016 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-26T06:16:38.017 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-10-26T06:16:38.017 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-10-26T06:16:38.017 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-10-26T06:16:38.017 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-10-26T06:16:38.017 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-10-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-10-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-10-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-10-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-10-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-10-26T06:16:38.018 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-10-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-10-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-10-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-10-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-10-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-10-26T06:16:38.019 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-10-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-10-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-10-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-10-26T06:16:38.020 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-10-26T06:16:38.021 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-26T06:16:38.021 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-10-26T06:16:38.021 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-10-26T06:16:38.021 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-10-26T06:16:38.021 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-10-26T06:16:38.021 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-10-26T06:16:38.022 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-10-26T06:16:38.022 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-10-26T06:16:38.022 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-10-26T06:16:38.022 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-10-26T06:16:38.022 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-10-26T06:16:38.023 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-10-26T06:16:38.023 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-10-26T06:16:38.023 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-10-26T06:16:38.023 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-10-26T06:16:38.023 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-26T06:16:38.024 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-10-26T06:16:38.024 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-10-26T06:16:38.024 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-10-26T06:16:38.024 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-10-26T06:16:38.024 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-10-26T06:16:38.024 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-10-26T06:16:38.025 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-10-26T06:16:38.025 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-10-26T06:16:38.025 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-10-26T06:16:38.025 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-10-26T06:16:38.025 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-10-26T06:16:38.025 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-10-26T06:16:38.026 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-10-26T06:16:38.026 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-10-26T06:16:38.026 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-10-26T06:16:38.026 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-10-26T06:16:38.026 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-10-26T06:16:38.027 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-10-26T06:16:38.027 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-10-26T06:16:38.027 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-10-26T06:16:38.027 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-10-26T06:16:38.027 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-10-26T06:16:38.028 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-10-26T06:16:38.028 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-10-26T06:16:38.028 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-10-26T06:16:38.028 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-10-26T06:16:38.028 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-10-26T06:16:38.029 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-10-26T06:16:38.029 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-10-26T06:16:38.029 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-10-26T06:16:38.029 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-10-26T06:16:38.029 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-10-26T06:16:38.029 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-10-26T06:16:38.030 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-10-26T06:16:38.030 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-10-26T06:16:38.030 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-10-26T06:16:38.030 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-10-26T06:16:38.030 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-10-26T06:16:38.031 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-10-26T06:16:38.031 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-10-26T06:16:38.031 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-10-26T06:16:38.031 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-10-26T06:16:38.031 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-10-26T06:16:38.031 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-10-26T06:16:38.032 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-10-26T06:16:38.032 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-10-26T06:16:38.032 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-10-26T06:16:38.032 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-10-26T06:16:38.032 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-26T06:16:38.033 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-26T06:16:38.033 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-26T06:16:38.033 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-26T06:16:38.033 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-26T06:16:38.033 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-26T06:16:38.034 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-26T06:16:38.034 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-26T06:16:38.034 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-26T06:16:38.034 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-26T06:16:38.034 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-26T06:16:38.035 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-26T06:16:38.035 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-10-26T06:16:38.035 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-10-26T06:16:38.035 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-10-26T06:16:38.035 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-26T06:16:38.035 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-10-26T06:16:38.036 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-10-26T06:16:38.036 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-10-26T06:16:38.036 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-10-26T06:16:38.036 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-10-26T06:16:38.036 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-10-26T06:16:38.037 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-10-26T06:16:38.037 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-10-26T06:16:38.037 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-10-26T06:16:38.037 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-10-26T06:16:38.037 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-10-26T06:16:38.037 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-10-26T06:16:38.038 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-10-26T06:16:38.038 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-10-26T06:16:38.038 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-10-26T06:16:38.038 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-10-26T06:16:38.038 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-10-26T06:16:38.039 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-10-26T06:16:38.039 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-10-26T06:16:38.039 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-10-26T06:16:38.039 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-10-26T06:16:38.039 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-10-26T06:16:38.040 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-10-26T06:16:38.040 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-10-26T06:16:38.040 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-10-26T06:16:38.040 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-10-26T06:16:38.040 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-10-26T06:16:38.040 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-26T06:16:38.041 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-26T06:16:38.041 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-10-26T06:16:38.041 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-26T06:16:38.041 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-26T06:16:38.041 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-26T06:16:38.042 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-26T06:16:38.042 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-26T06:16:38.042 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-26T06:16:38.042 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-26T06:16:38.042 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-26T06:16:38.043 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-26T06:16:38.043 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-26T06:16:38.043 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-26T06:16:38.043 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-26T06:16:38.043 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-26T06:16:38.043 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-26T06:16:38.044 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-26T06:16:38.044 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-26T06:16:38.044 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-26T06:16:38.044 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-26T06:16:38.044 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-26T06:16:38.045 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-26T06:16:38.045 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-26T06:16:38.045 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-26T06:16:38.045 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-26T06:16:38.045 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-10-26T06:16:38.045 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-10-26T06:16:38.046 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-26T06:16:38.046 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-10-26T06:16:38.046 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-10-26T06:16:38.046 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-10-26T06:16:38.046 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-10-26T06:16:38.046 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-26T06:16:38.047 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-10-26T06:16:38.047 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-26T06:16:38.047 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-26T06:16:38.047 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-26T06:16:38.047 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-26T06:16:38.048 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-26T06:16:38.048 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-10-26T06:16:38.048 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-10-26T06:16:38.048 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-26T06:16:38.048 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-10-26T06:16:38.048 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-26T06:16:38.049 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-10-26T06:16:38.049 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-26T06:16:38.049 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-10-26T06:16:38.049 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-26T06:16:38.049 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-10-26T06:16:38.049 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-10-26T06:16:38.050 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-26T06:16:38.050 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-10-26T06:16:38.050 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-26T06:16:38.050 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-10-26T06:16:38.050 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-26T06:16:38.051 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-10-26T06:16:38.051 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-26T06:16:38.051 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-10-26T06:16:38.051 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-26T06:16:38.051 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-10-26T06:16:38.051 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-10-26T06:16:38.052 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-26T06:16:38.052 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-26T06:16:38.052 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-10-26T06:16:38.052 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-10-26T06:16:38.052 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-10-26T06:16:38.052 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-26T06:16:38.053 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-10-26T06:16:38.053 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-10-26T06:16:38.053 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-10-26T06:16:38.053 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-10-26T06:16:38.053 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-26T06:16:38.054 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-10-26T06:16:38.054 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-26T06:16:38.054 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-26T06:16:38.054 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-26T06:16:38.054 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-10-26T06:16:38.054 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-26T06:16:38.055 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-10-26T06:16:38.055 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-10-26T06:16:38.055 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-10-26T06:16:38.055 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-26T06:16:38.055 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-10-26T06:16:38.056 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-10-26T06:16:38.056 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-10-26T06:16:38.056 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-10-26T06:16:38.056 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-26T06:16:38.056 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-10-26T06:16:38.056 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-10-26T06:16:38.057 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-10-26T06:16:38.057 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-10-26T06:16:38.057 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-10-26T06:16:38.057 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-10-26T06:16:38.057 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-26T06:16:38.058 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-26T06:16:38.058 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-26T06:16:38.058 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-26T06:16:38.058 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-26T06:16:38.058 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-26T06:16:38.058 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-26T06:16:38.059 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-26T06:16:38.059 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-10-26T06:16:38.059 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-10-26T06:16:38.059 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-10-26T06:16:38.059 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-10-26T06:16:38.060 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-10-26T06:16:38.060 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-10-26T06:16:38.060 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-10-26T06:16:38.060 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-10-26T06:16:38.060 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-10-26T06:16:38.060 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-10-26T06:16:38.061 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-10-26T06:16:38.061 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-10-26T06:16:38.061 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-10-26T06:16:38.061 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-10-26T06:16:38.061 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-10-26T06:16:38.062 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-10-26T06:16:38.062 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-10-26T06:16:38.062 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-10-26T06:16:38.062 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-10-26T06:16:38.062 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-10-26T06:16:38.062 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-10-26T06:16:38.063 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-10-26T06:16:38.063 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-26T06:16:38.063 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-10-26T06:16:38.063 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-10-26T06:16:38.063 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-10-26T06:16:38.064 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-10-26T06:16:38.064 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-10-26T06:16:38.064 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-10-26T06:16:38.064 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-10-26T06:16:38.064 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-10-26T06:16:38.065 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-10-26T06:16:38.065 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-10-26T06:16:38.065 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-10-26T06:16:38.065 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-10-26T06:16:38.065 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-10-26T06:16:38.065 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-10-26T06:16:38.066 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-10-26T06:16:38.066 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-10-26T06:16:38.066 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-10-26T06:16:38.066 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-10-26T06:16:38.066 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-10-26T06:16:38.067 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-10-26T06:16:38.067 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-10-26T06:16:38.067 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-10-26T06:16:38.067 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-10-26T06:16:38.067 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-26T06:16:38.067 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-10-26T06:16:38.068 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-10-26T06:16:38.068 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-10-26T06:16:38.068 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-10-26T06:16:38.068 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-10-26T06:16:38.068 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-10-26T06:16:38.068 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-10-26T06:16:38.069 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-26T06:16:38.069 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-26T06:16:38.069 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-10-26T06:16:38.069 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-10-26T06:16:38.069 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-10-26T06:16:38.070 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-10-26T06:16:38.070 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-10-26T06:16:38.070 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-10-26T06:16:38.070 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-26T06:16:38.070 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-26T06:16:38.070 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-10-26T06:16:38.071 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-10-26T06:16:38.071 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-10-26T06:16:38.071 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-10-26T06:16:38.071 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-10-26T06:16:38.071 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-26T06:16:38.072 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-10-26T06:16:38.072 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-26T06:16:38.072 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-26T06:16:38.072 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-26T06:16:38.072 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-10-26T06:16:38.073 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-10-26T06:16:38.073 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-10-26T06:16:38.073 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-10-26T06:16:38.073 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-10-26T06:16:38.073 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-26T06:16:38.073 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-10-26T06:16:38.074 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-10-26T06:16:38.074 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-10-26T06:16:38.074 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-10-26T06:16:38.074 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-10-26T06:16:38.074 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-26T06:16:38.075 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-26T06:16:38.075 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-26T06:16:38.075 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-10-26T06:16:38.075 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-26T06:16:38.075 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-10-26T06:16:38.075 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-10-26T06:16:38.076 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-10-26T06:16:38.076 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-10-26T06:16:38.076 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-26T06:16:38.076 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-26T06:16:38.076 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-26T06:16:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-26T06:16:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-10-26T06:16:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-26T06:16:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-26T06:16:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-26T06:16:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-26T06:16:38.078 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-10-26T06:16:38.078 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-26T06:16:38.078 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-10-26T06:16:38.078 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-26T06:16:38.078 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-10-26T06:16:38.079 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-26T06:16:38.079 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-10-26T06:16:38.079 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-26T06:16:38.079 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-10-26T06:16:38.079 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-26T06:16:38.080 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-26T06:16:38.080 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-26T06:16:38.081 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-10-26T06:16:38.081 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-10-26T06:16:38.081 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-26T06:16:38.081 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-26T06:16:38.082 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-26T06:16:38.082 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-26T06:16:38.082 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-26T06:16:38.082 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-10-26T06:16:38.082 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-26T06:16:38.082 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-26T06:16:38.083 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-10-26T06:16:38.083 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-26T06:16:38.083 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-26T06:16:38.083 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-26T06:16:38.083 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-26T06:16:38.084 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-26T06:16:38.084 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-10-26T06:16:38.084 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-26T06:16:38.084 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-26T06:16:38.084 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-10-26T06:16:38.085 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-10-26T06:16:38.085 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-26T06:16:38.085 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-10-26T06:16:38.085 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-26T06:16:38.085 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-26T06:16:38.085 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-26T06:16:38.086 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-26T06:16:38.086 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-26T06:16:38.086 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-26T06:16:38.086 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-26T06:16:38.086 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-26T06:16:38.086 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-26T06:16:38.087 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-26T06:16:38.087 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-10-26T06:16:38.087 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-10-26T06:16:38.087 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-26T06:16:38.087 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-10-26T06:16:38.088 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-10-26T06:16:38.088 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-26T06:16:38.088 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-26T06:16:38.088 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-26T06:16:38.088 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-26T06:16:38.088 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-26T06:16:38.089 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-26T06:16:38.089 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-10-26T06:16:38.089 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-26T06:16:38.089 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-26T06:16:38.089 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-10-26T06:16:38.089 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-10-26T06:16:38.090 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-10-26T06:16:38.090 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-26T06:16:38.090 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-10-26T06:16:38.090 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-10-26T06:16:38.090 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-10-26T06:16:38.091 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-26T06:16:38.091 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-10-26T06:16:38.091 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-26T06:16:38.091 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-10-26T06:16:38.091 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-26T06:16:38.091 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-10-26T06:16:38.092 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-26T06:16:38.092 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-10-26T06:16:38.092 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-26T06:16:38.092 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-10-26T06:16:38.092 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-26T06:16:38.093 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-10-26T06:16:38.093 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-26T06:16:38.093 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-10-26T06:16:38.093 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-26T06:16:38.093 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-26T06:16:38.093 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-26T06:16:38.094 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-26T06:16:38.094 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-26T06:16:38.094 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-10-26T06:16:38.094 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-10-26T06:16:38.094 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-26T06:16:38.094 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-26T06:16:38.095 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-26T06:16:38.095 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-10-26T06:16:38.095 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-26T06:16:38.095 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-26T06:16:38.095 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-26T06:16:38.096 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-26T06:16:38.096 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-26T06:16:38.096 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-26T06:16:38.096 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-10-26T06:16:38.096 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-26T06:16:38.097 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-10-26T06:16:38.097 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-26T06:16:38.097 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-26T06:16:38.097 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-10-26T06:16:38.097 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-26T06:16:38.098 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-26T06:16:38.098 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-10-26T06:16:38.098 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-26T06:16:38.098 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-10-26T06:16:38.098 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-26T06:16:38.098 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-10-26T06:16:38.099 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-26T06:16:38.099 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-26T06:16:38.099 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-10-26T06:16:38.099 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-26T06:16:38.099 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-26T06:16:38.099 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-26T06:16:38.100 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-26T06:16:38.100 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-26T06:16:38.100 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-10-26T06:16:38.100 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-26T06:16:38.100 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-10-26T06:16:38.101 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-26T06:16:38.101 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-26T06:16:38.101 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-26T06:16:38.101 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-26T06:16:38.101 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-10-26T06:16:38.101 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-26T06:16:38.102 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-10-26T06:16:38.102 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-26T06:16:38.102 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-10-26T06:16:38.102 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-26T06:16:38.102 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-10-26T06:16:38.102 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-26T06:16:38.103 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-26T06:16:38.103 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-10-26T06:16:38.103 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-26T06:16:38.103 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-26T06:16:38.103 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-26T06:16:38.104 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-26T06:16:38.104 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-10-26T06:16:38.104 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-26T06:16:38.104 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-26T06:16:38.104 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-26T06:16:38.104 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-10-26T06:16:38.105 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-26T06:16:38.105 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-26T06:16:38.105 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-26T06:16:38.105 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-26T06:16:38.105 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-10-26T06:16:38.105 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-26T06:16:38.106 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-10-26T06:16:38.106 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-26T06:16:38.106 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-10-26T06:16:38.106 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-26T06:16:38.106 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-26T06:16:38.106 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-10-26T06:16:38.107 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-26T06:16:38.107 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-26T06:16:38.107 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-10-26T06:16:38.107 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-26T06:16:38.107 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-10-26T06:16:38.108 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-10-26T06:16:38.108 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-26T06:16:38.108 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-10-26T06:16:38.108 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-26T06:16:38.108 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-10-26T06:16:38.108 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-10-26T06:16:38.109 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-26T06:16:38.109 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-10-26T06:16:38.109 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-10-26T06:16:38.109 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-10-26T06:16:38.109 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-10-26T06:16:38.109 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-26T06:16:38.110 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-26T06:16:38.110 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-26T06:16:38.110 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-10-26T06:16:38.110 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-10-26T06:16:38.110 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-10-26T06:16:38.110 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-10-26T06:16:38.111 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-10-26T06:16:38.111 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-26T06:16:38.111 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-26T06:16:38.111 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-10-26T06:16:38.111 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-10-26T06:16:38.112 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-26T06:16:38.112 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-26T06:16:38.112 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-26T06:16:38.112 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-10-26T06:16:38.112 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-10-26T06:16:38.112 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-10-26T06:16:38.113 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-10-26T06:16:38.113 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-10-26T06:16:38.113 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-10-26T06:16:38.113 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-10-26T06:16:38.113 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-10-26T06:16:38.113 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-10-26T06:16:38.114 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-10-26T06:16:38.114 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-10-26T06:16:38.114 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-10-26T06:16:38.114 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-10-26T06:16:38.114 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-10-26T06:16:38.115 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-10-26T06:16:38.115 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-10-26T06:16:38.115 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-10-26T06:16:38.115 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-10-26T06:16:38.115 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-26T06:16:38.116 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-10-26T06:16:38.116 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-26T06:16:38.116 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-26T06:16:38.116 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-26T06:16:38.116 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-26T06:16:38.116 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-10-26T06:16:38.117 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-10-26T06:16:38.117 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-10-26T06:16:38.117 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-10-26T06:16:38.117 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-10-26T06:16:38.117 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-10-26T06:16:38.117 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-10-26T06:16:38.118 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-10-26T06:16:38.118 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-10-26T06:16:38.118 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-10-26T06:16:38.118 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-26T06:16:38.118 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-26T06:16:38.119 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-26T06:16:38.119 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-10-26T06:16:38.119 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-26T06:16:38.119 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-10-26T06:16:38.119 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-10-26T06:16:38.120 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-10-26T06:16:38.120 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-10-26T06:16:38.120 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-26T06:16:38.120 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-10-26T06:16:38.120 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-10-26T06:16:38.120 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-10-26T06:16:38.121 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-26T06:16:38.121 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-10-26T06:16:38.121 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-10-26T06:16:38.121 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-10-26T06:16:38.121 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-10-26T06:16:38.121 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-10-26T06:16:38.122 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-10-26T06:16:38.122 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-10-26T06:16:38.122 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-10-26T06:16:38.122 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-10-26T06:16:38.122 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-10-26T06:16:38.123 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-10-26T06:16:38.123 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-10-26T06:16:38.123 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-10-26T06:16:38.123 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-10-26T06:16:38.123 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-26T06:16:38.123 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-10-26T06:16:38.124 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-26T06:16:38.124 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-10-26T06:16:38.124 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-10-26T06:16:38.124 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-10-26T06:16:38.124 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-10-26T06:16:38.124 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-26T06:16:38.125 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-10-26T06:16:38.125 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-10-26T06:16:38.125 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-26T06:16:38.125 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-10-26T06:16:38.125 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-10-26T06:16:38.125 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-26T06:16:38.126 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-10-26T06:16:38.126 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-26T06:16:38.126 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-26T06:16:38.126 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-26T06:16:38.126 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-26T06:16:38.126 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-26T06:16:38.127 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-26T06:16:38.127 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-26T06:16:38.127 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-26T06:16:38.127 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-26T06:16:38.127 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-10-26T06:16:38.127 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-10-26T06:16:38.128 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-26T06:16:38.128 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-10-26T06:16:38.128 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-26T06:16:38.128 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-26T06:16:38.128 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-26T06:16:38.129 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-26T06:16:38.129 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-26T06:16:38.129 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-26T06:16:38.129 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-26T06:16:38.129 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-26T06:16:38.129 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-26T06:16:38.130 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-10-26T06:16:38.130 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-10-26T06:16:38.130 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-26T06:16:38.130 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-26T06:16:38.130 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-10-26T06:16:38.130 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-10-26T06:16:38.131 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-26T06:16:38.131 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-26T06:16:38.131 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-10-26T06:16:38.131 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-10-26T06:16:38.131 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-10-26T06:16:38.131 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-10-26T06:16:38.132 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-10-26T06:16:38.132 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-10-26T06:16:38.132 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-26T06:16:38.132 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-26T06:16:38.132 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-26T06:16:38.132 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-26T06:16:38.133 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-26T06:16:38.133 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-26T06:16:38.133 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-26T06:16:38.133 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-26T06:16:38.133 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-26T06:16:38.133 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-26T06:16:38.134 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-10-26T06:16:38.134 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-26T06:16:38.134 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-26T06:16:38.134 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-26T06:16:38.134 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-26T06:16:38.135 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-26T06:16:38.135 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-26T06:16:38.135 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-10-26T06:16:38.135 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-26T06:16:38.135 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-26T06:16:38.135 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-26T06:16:38.136 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-26T06:16:38.136 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-26T06:16:38.136 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-26T06:16:38.136 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-26T06:16:38.136 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-26T06:16:38.136 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-26T06:16:38.137 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-26T06:16:38.137 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-26T06:16:38.137 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-26T06:16:38.137 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-10-26T06:16:38.137 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-10-26T06:16:38.137 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-10-26T06:16:38.138 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-10-26T06:16:38.138 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-10-26T06:16:38.138 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-10-26T06:16:38.138 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-10-26T06:16:38.138 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-10-26T06:16:38.138 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-26T06:16:38.139 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-10-26T06:16:38.139 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-10-26T06:16:38.139 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-26T06:16:38.139 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-26T06:16:38.139 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-10-26T06:16:38.140 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-10-26T06:16:38.140 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-26T06:16:38.140 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-10-26T06:16:38.140 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-10-26T06:16:38.140 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-26T06:16:38.140 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-10-26T06:16:38.141 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-10-26T06:16:38.141 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-10-26T06:16:38.141 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-26T06:16:38.141 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-10-26T06:16:38.141 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-26T06:16:38.141 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-10-26T06:16:38.142 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-10-26T06:16:38.142 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-26T06:16:38.142 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-10-26T06:16:38.142 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-26T06:16:38.142 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-26T06:16:38.143 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-26T06:16:38.143 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-26T06:16:38.143 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-26T06:16:38.143 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-10-26T06:16:38.143 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-10-26T06:16:38.143 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-26T06:16:38.144 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-26T06:16:38.144 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-26T06:16:38.144 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-26T06:16:38.144 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-10-26T06:16:38.144 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-10-26T06:16:38.144 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-26T06:16:38.145 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-10-26T06:16:38.145 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-10-26T06:16:38.145 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-26T06:16:38.145 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-26T06:16:38.145 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-10-26T06:16:38.145 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-10-26T06:16:38.146 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-10-26T06:16:38.146 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-10-26T06:16:38.146 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-10-26T06:16:38.146 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-10-26T06:16:38.146 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-10-26T06:16:38.147 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-10-26T06:16:38.147 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-10-26T06:16:38.147 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-10-26T06:16:38.147 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-26T06:16:38.147 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-26T06:16:38.147 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-10-26T06:16:38.148 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-10-26T06:16:38.148 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-10-26T06:16:38.148 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-26T06:16:38.148 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-10-26T06:16:38.148 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-10-26T06:16:38.148 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-10-26T06:16:38.149 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-26T06:16:38.149 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-10-26T06:16:38.149 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-10-26T06:16:38.149 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-26T06:16:38.149 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-26T06:16:38.150 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-26T06:16:38.150 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-10-26T06:16:38.150 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-26T06:16:38.150 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-10-26T06:16:38.150 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-26T06:16:38.150 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-10-26T06:16:38.151 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-10-26T06:16:38.151 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-26T06:16:38.151 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-10-26T06:16:38.151 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-10-26T06:16:38.151 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-10-26T06:16:38.151 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-26T06:16:38.152 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-26T06:16:38.152 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-10-26T06:16:38.152 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-26T06:16:38.152 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-10-26T06:16:38.152 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-10-26T06:16:38.152 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-26T06:16:38.153 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-26T06:16:38.153 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-26T06:16:38.153 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-26T06:16:38.153 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-26T06:16:38.153 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-10-26T06:16:38.153 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-10-26T06:16:38.154 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-10-26T06:16:38.154 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-10-26T06:16:38.154 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-26T06:16:38.154 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-10-26T06:16:38.154 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-26T06:16:38.154 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-10-26T06:16:38.155 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-10-26T06:16:38.155 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-10-26T06:16:38.155 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-10-26T06:16:38.155 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-10-26T06:16:38.155 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-26T06:16:38.155 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-10-26T06:16:38.156 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-26T06:16:38.156 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-10-26T06:16:38.156 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-10-26T06:16:38.156 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-10-26T06:16:38.156 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-10-26T06:16:38.156 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-10-26T06:16:38.157 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-10-26T06:16:38.157 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-26T06:16:38.157 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-10-26T06:16:38.157 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-10-26T06:16:38.157 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-10-26T06:16:38.157 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-26T06:16:38.158 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-26T06:16:38.158 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-26T06:16:38.158 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-26T06:16:38.158 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-26T06:16:38.158 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-26T06:16:38.158 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-26T06:16:38.159 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-26T06:16:38.159 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-26T06:16:38.159 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-26T06:16:38.159 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-26T06:16:38.159 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-26T06:16:38.159 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-26T06:16:38.160 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-26T06:16:38.160 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-26T06:16:38.160 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-10-26T06:16:38.160 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-10-26T06:16:38.160 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-26T06:16:38.161 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-10-26T06:16:38.161 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-10-26T06:16:38.161 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-10-26T06:16:38.161 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-10-26T06:16:38.161 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-10-26T06:16:38.161 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-26T06:16:38.162 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-10-26T06:16:38.162 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-10-26T06:16:38.162 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-26T06:16:38.162 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-10-26T06:16:38.162 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-10-26T06:16:38.162 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-26T06:16:38.163 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-10-26T06:16:38.163 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-26T06:16:38.163 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-10-26T06:16:38.163 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-10-26T06:16:38.163 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-10-26T06:16:38.163 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-10-26T06:16:38.164 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-10-26T06:16:38.164 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-10-26T06:16:38.164 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-10-26T06:16:38.164 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-10-26T06:16:38.164 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-10-26T06:16:38.164 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-26T06:16:38.165 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-10-26T06:16:38.165 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-10-26T06:16:38.165 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-10-26T06:16:38.165 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-26T06:16:38.165 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-10-26T06:16:38.165 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-10-26T06:16:38.166 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-10-26T06:16:38.166 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-10-26T06:16:38.166 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-10-26T06:16:38.166 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-10-26T06:16:38.166 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-10-26T06:16:38.167 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-10-26T06:16:38.167 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-10-26T06:16:38.167 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-10-26T06:16:38.167 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-10-26T06:16:38.167 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-26T06:16:38.167 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-10-26T06:16:38.167 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-10-26T06:16:38.168 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-10-26T06:16:38.168 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-26T06:16:38.168 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-26T06:16:38.168 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-10-26T06:16:38.168 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-10-26T06:16:38.169 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-26T06:16:38.169 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-10-26T06:16:38.169 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-10-26T06:16:38.169 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-10-26T06:16:38.169 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-10-26T06:16:38.169 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-10-26T06:16:38.170 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-10-26T06:16:38.170 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-10-26T06:16:38.170 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-26T06:16:38.170 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-10-26T06:16:38.170 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-10-26T06:16:38.170 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-10-26T06:16:38.171 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-26T06:16:38.171 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-26T06:16:38.171 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-10-26T06:16:38.171 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-26T06:16:38.171 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-26T06:16:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-26T06:16:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-10-26T06:16:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-10-26T06:16:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-10-26T06:16:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-10-26T06:16:38.172 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-26T06:16:38.173 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-10-26T06:16:38.173 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-26T06:16:38.173 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-26T06:16:38.173 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-26T06:16:38.173 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-10-26T06:16:38.173 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-10-26T06:16:38.174 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-10-26T06:16:38.174 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-10-26T06:16:38.174 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-10-26T06:16:38.174 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-10-26T06:16:38.174 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-10-26T06:16:38.174 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-10-26T06:16:38.175 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-10-26T06:16:38.175 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-10-26T06:16:38.175 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-10-26T06:16:38.175 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-10-26T06:16:38.175 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-10-26T06:16:38.175 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-10-26T06:16:38.176 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-26T06:16:38.176 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-10-26T06:16:38.176 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-10-26T06:16:38.176 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-26T06:16:38.176 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-26T06:16:38.176 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-10-26T06:16:38.177 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-26T06:16:38.177 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-10-26T06:16:38.177 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-10-26T06:16:38.177 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-26T06:16:38.177 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-10-26T06:16:38.177 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-10-26T06:16:38.178 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-26T06:16:38.178 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-26T06:16:38.178 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-26T06:16:38.178 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-26T06:16:38.178 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-26T06:16:38.179 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-26T06:16:38.179 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-26T06:16:38.179 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-10-26T06:16:38.179 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-26T06:16:38.179 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-26T06:16:38.179 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-26T06:16:38.180 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-26T06:16:38.180 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-26T06:16:38.180 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-10-26T06:16:38.180 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-10-26T06:16:38.180 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-26T06:16:38.180 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-26T06:16:38.181 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-26T06:16:38.181 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-10-26T06:16:38.181 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-10-26T06:16:38.181 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-26T06:16:38.181 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-26T06:16:38.182 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-10-26T06:16:38.182 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-26T06:16:38.182 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-10-26T06:16:38.182 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-10-26T06:16:38.182 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-26T06:16:38.182 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-10-26T06:16:38.183 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-10-26T06:16:38.183 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-10-26T06:16:38.183 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-10-26T06:16:38.183 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-10-26T06:16:38.183 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-10-26T06:16:38.183 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-26T06:16:38.184 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-10-26T06:16:38.184 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-10-26T06:16:38.184 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-10-26T06:16:38.184 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-10-26T06:16:38.184 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-10-26T06:16:38.185 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-26T06:16:38.185 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-26T06:16:38.185 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-10-26T06:16:38.185 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-10-26T06:16:38.185 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-10-26T06:16:38.186 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-10-26T06:16:38.186 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-10-26T06:16:38.186 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-10-26T06:16:38.186 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-26T06:16:38.186 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-26T06:16:38.186 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-26T06:16:38.187 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-26T06:16:38.187 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-26T06:16:38.187 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-26T06:16:38.187 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-26T06:16:38.187 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-26T06:16:38.187 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-26T06:16:38.188 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-26T06:16:38.188 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-26T06:16:38.188 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-10-26T06:16:38.188 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-10-26T06:16:38.188 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-26T06:16:38.188 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-26T06:16:38.189 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-26T06:16:38.189 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-10-26T06:16:38.189 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-10-26T06:16:38.189 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-26T06:16:38.189 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-26T06:16:38.189 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-10-26T06:16:38.190 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-10-26T06:16:38.190 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-26T06:16:38.190 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-26T06:16:38.190 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-10-26T06:16:38.190 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-10-26T06:16:38.190 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-10-26T06:16:38.191 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-26T06:16:38.191 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-26T06:16:38.191 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-26T06:16:38.191 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-26T06:16:38.191 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-10-26T06:16:38.192 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-26T06:16:38.192 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-10-26T06:16:38.192 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-26T06:16:38.192 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-26T06:16:38.192 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-26T06:16:38.192 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-10-26T06:16:38.193 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-10-26T06:16:38.193 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-10-26T06:16:38.193 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-26T06:16:38.193 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-10-26T06:16:38.193 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-10-26T06:16:38.193 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-10-26T06:16:38.194 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-10-26T06:16:38.194 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-26T06:16:38.194 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-10-26T06:16:38.194 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-10-26T06:16:38.194 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-26T06:16:38.194 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-10-26T06:16:38.195 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-10-26T06:16:38.195 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-10-26T06:16:38.195 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-10-26T06:16:38.195 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-10-26T06:16:38.195 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-10-26T06:16:38.196 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-10-26T06:16:38.196 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-10-26T06:16:38.196 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-10-26T06:16:38.196 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-10-26T06:16:38.196 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-10-26T06:16:38.196 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-10-26T06:16:38.197 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-10-26T06:16:38.197 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-10-26T06:16:38.197 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-10-26T06:16:38.197 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-10-26T06:16:38.197 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-10-26T06:16:38.197 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-10-26T06:16:38.198 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-10-26T06:16:38.198 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-10-26T06:16:38.198 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-10-26T06:16:38.198 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-26T06:16:38.198 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-10-26T06:16:38.199 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-10-26T06:16:38.199 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-10-26T06:16:38.199 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-10-26T06:16:38.199 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-10-26T06:16:38.199 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-10-26T06:16:38.199 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-10-26T06:16:38.199 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-10-26T06:16:38.200 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-10-26T06:16:38.200 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-10-26T06:16:38.200 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-10-26T06:16:38.200 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-10-26T06:16:38.200 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-10-26T06:16:38.201 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-10-26T06:16:38.201 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-10-26T06:16:38.201 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-10-26T06:16:38.201 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-10-26T06:16:38.201 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-10-26T06:16:38.201 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-10-26T06:16:38.202 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-10-26T06:16:38.202 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-10-26T06:16:38.202 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-10-26T06:16:38.202 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-10-26T06:16:38.202 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-10-26T06:16:38.202 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-10-26T06:16:38.203 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-10-26T06:16:38.203 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-10-26T06:16:38.203 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-10-26T06:16:38.203 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-10-26T06:16:38.203 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-26T06:16:38.203 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-10-26T06:16:38.204 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-26T06:16:38.204 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-10-26T06:16:38.204 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-26T06:16:38.204 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-10-26T06:16:38.204 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-10-26T06:16:38.205 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-10-26T06:16:38.205 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-10-26T06:16:38.205 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-10-26T06:16:38.205 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-10-26T06:16:38.205 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-10-26T06:16:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-10-26T06:16:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-10-26T06:16:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-10-26T06:16:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-10-26T06:16:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-10-26T06:16:38.206 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-10-26T06:16:38.207 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-10-26T06:16:38.207 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-10-26T06:16:38.207 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-10-26T06:16:38.207 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-10-26T06:16:38.207 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-10-26T06:16:38.207 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-10-26T06:16:38.208 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-10-26T06:16:38.208 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-10-26T06:16:38.208 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-10-26T06:16:38.208 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-10-26T06:16:38.208 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-10-26T06:16:38.209 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-10-26T06:16:38.209 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-10-26T06:16:38.209 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-10-26T06:16:38.209 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-10-26T06:16:38.209 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-10-26T06:16:38.209 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-10-26T06:16:38.210 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-10-26T06:16:38.210 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-10-26T06:16:38.210 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-10-26T06:16:38.210 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-10-26T06:16:38.210 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-10-26T06:16:38.210 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-10-26T06:16:38.211 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-10-26T06:16:38.211 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-10-26T06:16:38.211 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-10-26T06:16:38.211 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-10-26T06:16:38.211 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-10-26T06:16:38.211 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-10-26T06:16:38.212 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-10-26T06:16:38.212 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-10-26T06:16:38.212 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-10-26T06:16:38.212 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-10-26T06:16:38.212 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-10-26T06:16:38.213 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-10-26T06:16:38.213 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-10-26T06:16:38.213 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-10-26T06:16:38.213 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-10-26T06:16:38.213 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-10-26T06:16:38.213 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-10-26T06:16:38.213 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-10-26T06:16:38.214 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-10-26T06:16:38.214 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-10-26T06:16:38.214 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-10-26T06:16:38.214 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-10-26T06:16:38.214 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-10-26T06:16:38.215 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-26T06:16:38.215 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-10-26T06:16:38.215 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-10-26T06:16:38.215 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-10-26T06:16:38.215 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-26T06:16:38.215 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-10-26T06:16:38.216 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-26T06:16:38.216 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-10-26T06:16:38.216 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-26T06:16:38.216 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-10-26T06:16:38.216 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-26T06:16:38.216 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-26T06:16:38.217 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-26T06:16:38.217 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-10-26T06:16:38.217 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-10-26T06:16:38.217 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-10-26T06:16:38.217 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-10-26T06:16:38.217 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-10-26T06:16:38.218 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-10-26T06:16:38.218 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-10-26T06:16:38.218 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-10-26T06:16:38.218 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-10-26T06:16:38.218 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-10-26T06:16:38.218 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-10-26T06:16:38.219 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-10-26T06:16:38.219 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-10-26T06:16:38.219 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-10-26T06:16:38.219 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-10-26T06:16:38.219 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-10-26T06:16:38.219 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-10-26T06:16:38.220 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-10-26T06:16:38.220 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-10-26T06:16:38.220 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-10-26T06:16:38.220 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-10-26T06:16:38.220 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-10-26T06:16:38.220 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-10-26T06:16:38.221 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-10-26T06:16:38.221 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-10-26T06:16:38.221 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-10-26T06:16:38.221 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-10-26T06:16:38.221 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-10-26T06:16:38.221 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-10-26T06:16:38.222 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-26T06:16:38.222 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-10-26T06:16:38.222 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-10-26T06:16:38.222 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-10-26T06:16:38.222 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-10-26T06:16:38.222 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-10-26T06:16:38.223 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-10-26T06:16:38.223 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-10-26T06:16:38.223 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-10-26T06:16:38.223 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-10-26T06:16:38.223 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-10-26T06:16:38.223 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-10-26T06:16:38.224 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-10-26T06:16:38.224 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-10-26T06:16:38.224 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-10-26T06:16:38.224 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-10-26T06:16:38.224 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-10-26T06:16:38.224 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-10-26T06:16:38.225 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-26T06:16:38.225 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-10-26T06:16:38.225 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-10-26T06:16:38.225 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-10-26T06:16:38.225 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-10-26T06:16:38.225 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-10-26T06:16:38.226 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-10-26T06:16:38.226 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-10-26T06:16:38.226 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-10-26T06:16:38.226 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-10-26T06:16:38.226 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-10-26T06:16:38.226 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-10-26T06:16:38.227 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-10-26T06:16:38.227 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-10-26T06:16:38.227 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-26T06:16:38.227 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-10-26T06:16:38.227 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-10-26T06:16:38.227 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-10-26T06:16:38.228 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-10-26T06:16:38.228 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-10-26T06:16:38.228 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-10-26T06:16:38.228 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-10-26T06:16:38.228 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-10-26T06:16:38.228 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-10-26T06:16:38.228 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-10-26T06:16:38.229 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-10-26T06:16:38.229 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-10-26T06:16:38.229 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-10-26T06:16:38.229 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-10-26T06:16:38.229 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-10-26T06:16:38.229 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-10-26T06:16:38.230 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-10-26T06:16:38.230 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-10-26T06:16:38.230 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-10-26T06:16:38.230 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-10-26T06:16:38.230 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-10-26T06:16:38.231 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-10-26T06:16:38.231 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-26T06:16:38.231 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-26T06:16:38.231 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-26T06:16:38.231 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-26T06:16:38.231 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-26T06:16:38.231 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-26T06:16:38.232 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-26T06:16:38.232 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-26T06:16:38.232 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-26T06:16:38.232 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-26T06:16:38.232 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-26T06:16:38.232 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-26T06:16:38.233 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-26T06:16:38.233 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-26T06:16:38.233 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-26T06:16:38.233 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-26T06:16:38.233 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-26T06:16:38.233 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-26T06:16:38.234 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-26T06:16:38.234 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-26T06:16:38.234 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-10-26T06:16:38.234 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-26T06:16:38.234 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-26T06:16:38.234 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-26T06:16:38.235 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-10-26T06:16:38.235 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-10-26T06:16:38.235 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-10-26T06:16:38.235 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-10-26T06:16:38.235 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-10-26T06:16:38.235 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-10-26T06:16:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-10-26T06:16:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-10-26T06:16:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-10-26T06:16:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-10-26T06:16:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-10-26T06:16:38.236 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-10-26T06:16:38.237 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-10-26T06:16:38.237 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-10-26T06:16:38.237 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-10-26T06:16:38.237 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-10-26T06:16:38.237 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-10-26T06:16:38.237 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-10-26T06:16:38.238 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-10-26T06:16:38.238 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-10-26T06:16:38.238 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-26T06:16:38.238 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-10-26T06:16:38.238 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-26T06:16:38.239 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-10-26T06:16:38.239 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-10-26T06:16:38.239 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-10-26T06:16:38.239 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-10-26T06:16:38.239 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-10-26T06:16:38.239 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-10-26T06:16:38.240 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-10-26T06:16:38.240 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-10-26T06:16:38.240 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-10-26T06:16:38.240 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-10-26T06:16:38.240 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-10-26T06:16:38.240 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-10-26T06:16:38.241 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-26T06:16:38.241 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-26T06:16:38.241 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-26T06:16:38.241 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-26T06:16:38.241 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-26T06:16:38.241 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-26T06:16:38.242 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-26T06:16:38.242 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-26T06:16:38.242 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-26T06:16:38.242 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-26T06:16:38.242 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-26T06:16:38.243 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-26T06:16:38.243 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-26T06:16:38.243 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-26T06:16:38.243 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-26T06:16:38.243 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-26T06:16:38.243 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-26T06:16:38.244 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-26T06:16:38.244 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-26T06:16:38.244 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-26T06:16:38.244 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-26T06:16:38.244 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-10-26T06:16:38.244 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-10-26T06:16:38.245 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-10-26T06:16:38.245 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-10-26T06:16:38.245 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-10-26T06:16:38.245 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-10-26T06:16:38.245 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-10-26T06:16:38.245 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-10-26T06:16:38.246 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-10-26T06:16:38.246 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-10-26T06:16:38.246 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-10-26T06:16:38.246 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-10-26T06:16:38.246 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-10-26T06:16:38.247 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-10-26T06:16:38.247 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-10-26T06:16:38.247 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-10-26T06:16:38.247 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-26T06:16:38.247 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-10-26T06:16:38.247 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-10-26T06:16:38.248 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-10-26T06:16:38.248 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-10-26T06:16:38.248 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-10-26T06:16:38.248 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-26T06:16:38.248 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-10-26T06:16:38.248 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-10-26T06:16:38.249 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-10-26T06:16:38.249 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-26T06:16:38.249 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-26T06:16:38.249 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-26T06:16:38.249 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-26T06:16:38.249 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-26T06:16:38.250 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-26T06:16:38.250 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-26T06:16:38.250 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-26T06:16:38.250 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-26T06:16:38.250 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-26T06:16:38.250 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-26T06:16:38.251 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-26T06:16:38.251 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-26T06:16:38.251 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-26T06:16:38.251 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-26T06:16:38.251 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-26T06:16:38.251 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-26T06:16:38.252 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-26T06:16:38.252 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-26T06:16:38.252 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-26T06:16:38.252 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-26T06:16:38.252 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-26T06:16:38.253 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-26T06:16:38.253 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-26T06:16:38.253 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-10-26T06:16:38.253 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-10-26T06:16:38.253 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-26T06:16:38.253 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-10-26T06:16:38.254 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-10-26T06:16:38.254 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-10-26T06:16:38.254 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-10-26T06:16:38.254 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-26T06:16:38.254 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-10-26T06:16:38.254 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-10-26T06:16:38.255 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-10-26T06:16:38.255 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-10-26T06:16:38.255 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-10-26T06:16:38.255 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-10-26T06:16:38.255 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-10-26T06:16:38.256 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-10-26T06:16:38.256 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-10-26T06:16:38.256 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-10-26T06:16:38.256 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-10-26T06:16:38.256 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-10-26T06:16:38.256 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-10-26T06:16:38.257 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-10-26T06:16:38.257 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-26T06:16:38.257 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-26T06:16:38.257 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-26T06:16:38.257 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-26T06:16:38.257 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-26T06:16:38.258 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-26T06:16:38.258 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-26T06:16:38.258 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-26T06:16:38.258 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-26T06:16:38.258 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-26T06:16:38.258 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-26T06:16:38.259 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-26T06:16:38.259 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-26T06:16:38.259 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-26T06:16:38.259 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-26T06:16:38.259 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-10-26T06:16:38.259 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-26T06:16:38.260 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-26T06:16:38.260 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-26T06:16:38.260 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-26T06:16:38.260 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-26T06:16:38.260 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-26T06:16:38.260 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-26T06:16:38.261 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-26T06:16:38.261 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-26T06:16:38.261 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-26T06:16:38.261 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-26T06:16:38.261 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-26T06:16:38.261 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-26T06:16:38.262 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-26T06:16:38.262 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-10-26T06:16:38.262 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-10-26T06:16:38.262 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-26T06:16:38.262 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-26T06:16:38.263 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-26T06:16:38.263 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-26T06:16:38.263 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-26T06:16:38.263 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-26T06:16:38.263 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-26T06:16:38.263 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-26T06:16:38.264 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-26T06:16:38.264 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-26T06:16:38.264 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-26T06:16:38.264 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-26T06:16:38.264 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-26T06:16:38.264 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-10-26T06:16:38.265 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-26T06:16:38.265 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-10-26T06:16:38.265 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-10-26T06:16:38.265 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-26T06:16:38.265 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-26T06:16:38.265 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-10-26T06:16:38.266 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-10-26T06:16:38.266 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-26T06:16:38.266 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-26T06:16:38.266 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-10-26T06:16:38.266 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-10-26T06:16:38.267 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-26T06:16:38.267 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-10-26T06:16:38.267 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-26T06:16:38.267 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-26T06:16:38.267 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-10-26T06:16:38.267 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-10-26T06:16:38.267 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-10-26T06:16:38.268 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-26T06:16:38.268 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-10-26T06:16:38.268 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-26T06:16:38.268 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-10-26T06:16:38.268 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-10-26T06:16:38.268 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-10-26T06:16:38.269 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-10-26T06:16:38.269 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-10-26T06:16:38.269 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-10-26T06:16:38.269 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-10-26T06:16:38.269 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-10-26T06:16:38.269 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-10-26T06:16:38.270 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-10-26T06:16:38.270 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-10-26T06:16:38.270 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-10-26T06:16:38.270 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-10-26T06:16:38.270 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-10-26T06:16:38.271 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-26T06:16:38.271 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-26T06:16:38.271 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-26T06:16:38.271 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-10-26T06:16:38.271 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-10-26T06:16:38.271 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-10-26T06:16:38.272 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-10-26T06:16:38.272 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-26T06:16:38.272 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-10-26T06:16:38.272 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-10-26T06:16:38.272 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-26T06:16:38.272 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-10-26T06:16:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-10-26T06:16:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-26T06:16:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-10-26T06:16:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-26T06:16:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-10-26T06:16:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-26T06:16:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-26T06:16:38.274 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-10-26T06:16:38.274 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-10-26T06:16:38.274 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-10-26T06:16:38.274 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-10-26T06:16:38.274 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-10-26T06:16:38.275 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-10-26T06:16:38.275 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-10-26T06:16:38.275 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-10-26T06:16:38.275 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-10-26T06:16:38.275 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-10-26T06:16:38.275 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-10-26T06:16:38.275 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-10-26T06:16:38.276 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-10-26T06:16:38.276 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-10-26T06:16:38.276 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-10-26T06:16:38.276 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-10-26T06:16:38.276 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-10-26T06:16:38.276 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-10-26T06:16:38.277 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-10-26T06:16:38.277 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-10-26T06:16:38.277 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-10-26T06:16:38.277 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-10-26T06:16:38.277 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-10-26T06:16:38.277 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-10-26T06:16:38.278 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-10-26T06:16:38.278 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-10-26T06:16:38.278 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-10-26T06:16:38.278 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-10-26T06:16:38.278 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-10-26T06:16:38.278 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-10-26T06:16:38.279 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-10-26T06:16:38.279 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-10-26T06:16:38.279 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-10-26T06:16:38.279 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-10-26T06:16:38.279 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-10-26T06:16:38.279 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-10-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-10-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-10-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-10-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-10-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-10-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-10-26T06:16:38.280 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-10-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-10-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-10-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-10-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-10-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-10-26T06:16:38.281 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-10-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-10-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-10-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-10-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-10-26T06:16:38.282 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-10-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-10-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-10-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-10-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-10-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-10-26T06:16:38.283 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-10-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-10-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-10-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-10-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-10-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-10-26T06:16:38.284 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-10-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-10-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-10-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-10-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-10-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-10-26T06:16:38.285 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-10-26T06:16:38.286 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-10-26T06:16:38.286 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-10-26T06:16:38.286 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-10-26T06:16:38.286 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-10-26T06:16:38.286 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-10-26T06:16:38.286 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-10-26T06:16:38.287 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-10-26T06:16:38.287 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-10-26T06:16:38.287 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-10-26T06:16:38.287 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-10-26T06:16:38.287 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-10-26T06:16:38.287 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-10-26T06:16:38.288 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-10-26T06:16:38.288 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-10-26T06:16:38.288 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-10-26T06:16:38.288 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-10-26T06:16:38.288 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-10-26T06:16:38.288 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-10-26T06:16:38.288 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-10-26T06:16:38.289 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-10-26T06:16:38.289 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-10-26T06:16:38.289 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-10-26T06:16:38.289 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-10-26T06:16:38.289 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-10-26T06:16:38.289 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-10-26T06:16:38.290 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-10-26T06:16:38.290 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-10-26T06:16:38.290 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-10-26T06:16:38.290 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-10-26T06:16:38.290 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-10-26T06:16:38.290 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-10-26T06:16:38.291 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-10-26T06:16:38.291 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-10-26T06:16:38.291 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-10-26T06:16:38.291 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-10-26T06:16:38.291 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-10-26T06:16:38.291 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-10-26T06:16:38.292 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-10-26T06:16:38.292 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-10-26T06:16:38.292 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-10-26T06:16:38.292 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-10-26T06:16:38.292 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-10-26T06:16:38.292 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-10-26T06:16:38.293 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-10-26T06:16:38.293 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-26T06:16:38.293 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-26T06:16:38.293 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-10-26T06:16:38.293 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-10-26T06:16:38.293 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-26T06:16:38.294 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-10-26T06:16:38.294 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-10-26T06:16:38.294 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-10-26T06:16:38.294 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-10-26T06:16:38.294 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-10-26T06:16:38.294 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-10-26T06:16:38.295 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-26T06:16:38.295 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-26T06:16:38.295 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-10-26T06:16:38.295 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-26T06:16:38.295 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-26T06:16:38.295 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-10-26T06:16:38.296 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-26T06:16:38.296 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-10-26T06:16:38.296 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-10-26T06:16:38.296 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-10-26T06:16:38.296 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-26T06:16:38.296 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-10-26T06:16:38.297 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-10-26T06:16:38.297 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-10-26T06:16:38.297 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-10-26T06:16:38.297 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-10-26T06:16:38.297 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-10-26T06:16:38.297 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-10-26T06:16:38.298 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-10-26T06:16:38.298 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-10-26T06:16:38.298 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-10-26T06:16:38.298 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-10-26T06:16:38.298 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-10-26T06:16:38.298 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-10-26T06:16:38.299 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-10-26T06:16:38.299 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-10-26T06:16:38.299 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-10-26T06:16:38.299 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-10-26T06:16:38.300 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:38.300 DEBUG:teuthology.orchestra.run.smithi089:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-26T06:16:38.327 INFO:teuthology.orchestra.run.smithi089.stdout:postinstall scriptlet (using /bin/sh): 2023-10-26T06:16:38.328 INFO:teuthology.orchestra.run.smithi089.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-10-26T06:16:38.328 INFO:teuthology.orchestra.run.smithi089.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T06:16:38.328 INFO:teuthology.orchestra.run.smithi089.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T06:16:38.328 INFO:teuthology.orchestra.run.smithi089.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:38.328 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T06:16:38.329 INFO:teuthology.orchestra.run.smithi089.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T06:16:38.329 INFO:teuthology.orchestra.run.smithi089.stdout:fi 2023-10-26T06:16:38.329 INFO:teuthology.orchestra.run.smithi089.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-26T06:16:38.329 INFO:teuthology.orchestra.run.smithi089.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-26T06:16:38.329 INFO:teuthology.orchestra.run.smithi089.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-10-26T06:16:38.329 INFO:teuthology.orchestra.run.smithi089.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-26T06:16:38.330 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:38.330 INFO:teuthology.orchestra.run.smithi089.stdout:fi 2023-10-26T06:16:38.330 INFO:teuthology.orchestra.run.smithi089.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-26T06:16:38.330 INFO:teuthology.orchestra.run.smithi089.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-26T06:16:38.330 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:38.331 INFO:teuthology.orchestra.run.smithi089.stdout:fi 2023-10-26T06:16:38.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -qi grub2-tools 2023-10-26T06:16:38.409 INFO:teuthology.orchestra.run.smithi089.stdout:Name : grub2-tools 2023-10-26T06:16:38.409 INFO:teuthology.orchestra.run.smithi089.stdout:Epoch : 1 2023-10-26T06:16:38.409 INFO:teuthology.orchestra.run.smithi089.stdout:Version : 2.02 2023-10-26T06:16:38.409 INFO:teuthology.orchestra.run.smithi089.stdout:Release : 123.el8 2023-10-26T06:16:38.410 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: x86_64 2023-10-26T06:16:38.410 INFO:teuthology.orchestra.run.smithi089.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-26T06:16:38.410 INFO:teuthology.orchestra.run.smithi089.stdout:Group : System Environment/Base 2023-10-26T06:16:38.410 INFO:teuthology.orchestra.run.smithi089.stdout:Size : 9265450 2023-10-26T06:16:38.410 INFO:teuthology.orchestra.run.smithi089.stdout:License : GPLv3+ 2023-10-26T06:16:38.410 INFO:teuthology.orchestra.run.smithi089.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-26T06:16:38.411 INFO:teuthology.orchestra.run.smithi089.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-26T06:16:38.411 INFO:teuthology.orchestra.run.smithi089.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-26T06:16:38.411 INFO:teuthology.orchestra.run.smithi089.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-26T06:16:38.411 INFO:teuthology.orchestra.run.smithi089.stdout:Relocations : (not relocatable) 2023-10-26T06:16:38.411 INFO:teuthology.orchestra.run.smithi089.stdout:Packager : Red Hat, Inc. 2023-10-26T06:16:38.411 INFO:teuthology.orchestra.run.smithi089.stdout:Vendor : Red Hat, Inc. 2023-10-26T06:16:38.412 INFO:teuthology.orchestra.run.smithi089.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-26T06:16:38.412 INFO:teuthology.orchestra.run.smithi089.stdout:Summary : Support tools for GRUB. 2023-10-26T06:16:38.412 INFO:teuthology.orchestra.run.smithi089.stdout:Description : 2023-10-26T06:16:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:16:38.412 INFO:teuthology.orchestra.run.smithi089.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-26T06:16:38.413 INFO:teuthology.orchestra.run.smithi089.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-26T06:16:38.413 INFO:teuthology.orchestra.run.smithi089.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-26T06:16:38.413 INFO:teuthology.orchestra.run.smithi089.stdout:hardware devices. 2023-10-26T06:16:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:16:38.413 INFO:teuthology.orchestra.run.smithi089.stdout:This subpackage provides tools for support of all platforms. 2023-10-26T06:16:38.414 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-26T06:16:38.414 INFO:teuthology.task.kernel:Updating grub on smithi089 to boot 6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:38.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-26T06:16:45.456 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2023-10-26T06:16:47.505 INFO:teuthology.orchestra.run.smithi089.stderr:done 2023-10-26T06:16:47.507 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2023-10-26T06:16:47.527 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.QV1Q3Xc776 2023-10-26T06:16:47.527 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QV1Q3Xc776 2023-10-26T06:16:47.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.QV1Q3Xc776 2023-10-26T06:16:47.758 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.QV1Q3Xc776 is 6KB 2023-10-26T06:16:47.806 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.QV1Q3Xc776 2023-10-26T06:16:47.822 DEBUG:teuthology.orchestra.run.smithi089:> sudo /bin/ls /boot/loader/entries || true 2023-10-26T06:16:47.886 INFO:teuthology.orchestra.run.smithi089.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-26T06:16:47.886 INFO:teuthology.orchestra.run.smithi089.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-26T06:16:47.886 INFO:teuthology.orchestra.run.smithi089.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g5b73517e0e2f.conf 2023-10-26T06:16:47.887 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:16:48.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2023-10-26T06:17:18.048 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-26T06:17:18.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-26T06:17:18.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:18:18.052 DEBUG:teuthology.orchestra.remote:timed out 2023-10-26T06:18:27.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-26T06:18:27.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:18:30.454 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-10-26T06:18:30.829 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-26T06:18:30.829 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g5b73517e0e2f"... 2023-10-26T06:18:30.830 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2023-10-26T06:18:30.872 INFO:teuthology.orchestra.run.smithi089.stdout:6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:18:30.872 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-10-26T06:18:30.873 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-26T06:18:30.873 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-26T06:18:31.875 DEBUG:teuthology.parallel:result is None 2023-10-26T06:18:31.875 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-26T06:18:31.892 INFO:teuthology.task.internal:Creating test directory... 2023-10-26T06:18:31.893 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-26T06:18:31.899 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-26T06:18:31.904 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-26T06:18:31.938 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-26T06:18:31.961 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-26T06:18:31.993 INFO:teuthology.task.internal:Creating archive directory... 2023-10-26T06:18:31.994 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-26T06:18:32.000 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-26T06:18:32.004 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-26T06:18:32.057 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-26T06:18:32.069 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-26T06:18:32.069 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-26T06:18:32.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:18:32.095 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-26T06:18:32.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:18:32.117 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-26T06:18:32.122 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-26T06:18:32.139 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-26T06:18:32.183 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T06:18:32.208 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T06:18:32.209 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T06:18:32.241 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T06:18:32.347 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T06:18:32.379 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T06:18:32.383 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-26T06:18:32.405 INFO:teuthology.task.internal:Configuring sudo... 2023-10-26T06:18:32.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-26T06:18:32.412 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-26T06:18:32.415 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-26T06:18:32.476 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-26T06:18:32.489 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-26T06:18:32.489 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-26T06:18:32.496 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-26T06:18:32.499 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-26T06:18:32.533 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T06:18:32.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T06:18:32.732 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T06:18:32.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T06:18:32.891 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:18:32.892 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-26T06:18:32.969 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T06:18:33.021 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T06:18:33.075 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T06:18:33.168 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T06:18:33.221 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:18:33.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-26T06:18:33.320 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T06:18:33.377 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T06:18:33.440 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T06:18:33.495 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T06:18:33.539 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:18:33.540 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-26T06:18:33.612 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2023-10-26T06:18:33.618 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2023-10-26T06:18:33.623 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2023-10-26T06:18:33.683 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T06:18:33.729 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T06:18:33.733 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T06:18:34.156 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-26T06:18:34.168 INFO:teuthology.task.internal:Starting timer... 2023-10-26T06:18:34.169 INFO:teuthology.run_tasks:Running task pcp... 2023-10-26T06:18:34.210 INFO:teuthology.run_tasks:Running task selinux... 2023-10-26T06:18:34.288 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-26T06:18:34.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2023-10-26T06:18:34.404 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2023-10-26T06:18:34.408 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2023-10-26T06:18:34.465 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2023-10-26T06:18:34.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2023-10-26T06:18:34.534 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2023-10-26T06:18:34.536 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-26T06:18:34.537 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2023-10-26T06:18:34.585 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2023-10-26T06:18:34.586 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2023-10-26T06:18:34.613 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2023-10-26T06:18:34.614 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2023-10-26T06:18:34.642 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2023-10-26T06:18:34.643 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2023-10-26T06:18:34.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T06:18:34.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:18:34.720 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T06:18:34.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:18:34.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T06:18:34.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:18:34.780 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-26T06:18:34.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2023-10-26T06:18:34.849 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2023-10-26T06:18:34.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2023-10-26T06:18:34.904 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-26T06:18:34.914 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-26T06:18:34.914 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-26T06:18:34.932 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-26T06:18:34.933 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi135.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-26T06:30:43.230 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2023-10-26T06:30:43.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-26T06:30:43.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:30:43.357 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-10-26T06:30:43.464 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-26T06:30:43.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-10-26T06:30:43.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:30:43.533 DEBUG:teuthology.orchestra.run.smithi135:> true 2023-10-26T06:30:43.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-10-26T06:30:43.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-10-26T06:30:43.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T06:30:43.675 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-10-26T06:30:43.701 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-10-26T06:30:43.702 INFO:teuthology.run_tasks:Running task clock... 2023-10-26T06:30:43.713 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-26T06:30:43.713 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-26T06:30:43.714 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T06:30:43.718 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-26T06:30:43.719 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T06:30:43.721 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-26T06:30:43.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T06:30:43.755 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-26T06:30:43.771 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-26T06:30:43.775 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-26T06:30:43.788 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-26T06:30:43.790 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-26T06:30:43.822 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-26T06:30:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2023-10-26T06:30:43.860 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2023-10-26T06:30:43.866 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2023-10-26T06:30:43.872 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2023-10-26T06:30:43.881 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-26T06:30:43.886 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-26T06:30:43.895 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-26T06:30:43.901 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-26T06:30:43.910 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2023-10-26T06:30:43.939 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2023-10-26T06:30:43.968 INFO:teuthology.orchestra.run.smithi135.stderr:bash: ntpq: command not found 2023-10-26T06:30:43.972 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T06:30:43.973 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2023-10-26T06:30:43.973 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:43.973 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:43.973 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:43.974 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:43.975 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-26T06:30:43.991 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2023-10-26T06:30:43.994 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T06:30:43.994 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-10-26T06:30:43.994 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:43.995 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:43.995 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:43.995 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:44.007 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-26T06:30:44.100 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-10-26T06:30:44.107 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T06:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-10-26T06:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:44.108 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:44.109 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T06:30:44.110 INFO:teuthology.run_tasks:Running task pexec... 2023-10-26T06:30:44.120 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-26T06:30:44.121 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T06:30:44.121 DEBUG:teuthology.orchestra.run.smithi089:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-26T06:30:44.122 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi135.front.sepia.ceph.com 2023-10-26T06:30:44.122 DEBUG:teuthology.orchestra.run.smithi135:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-26T06:30:44.122 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi192.front.sepia.ceph.com 2023-10-26T06:30:44.123 DEBUG:teuthology.orchestra.run.smithi192:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-26T06:30:44.493 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-10-26T06:30:44.495 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-10-26T06:30:44.605 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-26T06:30:45.324 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:04:44 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:30:45.334 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:04:44 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:30:45.589 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:46 ago on Thu 26 Oct 2023 06:28:59 AM UTC. 2023-10-26T06:30:47.040 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-10-26T06:30:47.041 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-10-26T06:30:47.041 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2023-10-26T06:30:47.041 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-10-26T06:30:47.042 INFO:teuthology.orchestra.run.smithi135.stdout:Resetting modules: 2023-10-26T06:30:47.042 INFO:teuthology.orchestra.run.smithi135.stdout: container-tools 2023-10-26T06:30:47.042 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:30:47.042 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-10-26T06:30:47.042 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-10-26T06:30:47.043 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:30:47.214 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-10-26T06:30:47.215 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-10-26T06:30:47.216 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2023-10-26T06:30:47.216 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-10-26T06:30:47.216 INFO:teuthology.orchestra.run.smithi192.stdout:Resetting modules: 2023-10-26T06:30:47.216 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools 2023-10-26T06:30:47.217 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:30:47.217 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-10-26T06:30:47.217 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-10-26T06:30:47.218 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:30:47.428 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-10-26T06:30:47.429 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-10-26T06:30:47.429 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2023-10-26T06:30:47.430 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-10-26T06:30:47.430 INFO:teuthology.orchestra.run.smithi089.stdout:Resetting modules: 2023-10-26T06:30:47.430 INFO:teuthology.orchestra.run.smithi089.stdout: container-tools 2023-10-26T06:30:47.430 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:30:47.430 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-10-26T06:30:47.431 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-10-26T06:30:47.431 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:30:48.003 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:04:47 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:30:48.184 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:04:47 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:30:48.668 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:49 ago on Thu 26 Oct 2023 06:28:59 AM UTC. 2023-10-26T06:30:50.652 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-10-26T06:30:51.068 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-10-26T06:30:51.116 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-10-26T06:30:51.386 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-10-26T06:30:51.735 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-10-26T06:30:51.926 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:04:51 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:30:51.961 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-26T06:30:52.545 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:04:52 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:30:52.923 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:53 ago on Thu 26 Oct 2023 06:28:59 AM UTC. 2023-10-26T06:30:53.644 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-10-26T06:30:53.737 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================================ 2023-10-26T06:30:53.737 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-10-26T06:30:53.738 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================================ 2023-10-26T06:30:53.738 INFO:teuthology.orchestra.run.smithi135.stdout:Installing group/module packages: 2023-10-26T06:30:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-26T06:30:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-26T06:30:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-26T06:30:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-26T06:30:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-26T06:30:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-26T06:30:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-26T06:30:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-10-26T06:30:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-26T06:30:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-26T06:30:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-26T06:30:53.740 INFO:teuthology.orchestra.run.smithi135.stdout:Installing module profiles: 2023-10-26T06:30:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: container-tools/common 2023-10-26T06:30:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:30:53.741 INFO:teuthology.orchestra.run.smithi135.stdout:Enabling module streams: 2023-10-26T06:30:53.741 INFO:teuthology.orchestra.run.smithi135.stdout: container-tools rhel8 2023-10-26T06:30:53.741 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:30:53.741 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-10-26T06:30:53.741 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================================ 2023-10-26T06:30:53.742 INFO:teuthology.orchestra.run.smithi135.stdout:Install 10 Packages 2023-10-26T06:30:53.742 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:30:53.743 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 18 M 2023-10-26T06:30:53.743 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 65 M 2023-10-26T06:30:53.743 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-10-26T06:30:53.813 INFO:teuthology.orchestra.run.smithi135.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-10-26T06:30:53.839 INFO:teuthology.orchestra.run.smithi135.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 250 kB/s | 25 kB 00:00 2023-10-26T06:30:53.864 INFO:teuthology.orchestra.run.smithi135.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 328 kB/s | 41 kB 00:00 2023-10-26T06:30:53.890 INFO:teuthology.orchestra.run.smithi135.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 652 kB/s | 49 kB 00:00 2023-10-26T06:30:53.923 INFO:teuthology.orchestra.run.smithi135.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-26T06:30:53.948 INFO:teuthology.orchestra.run.smithi135.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-26T06:30:53.983 INFO:teuthology.orchestra.run.smithi135.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-10-26T06:30:54.095 INFO:teuthology.orchestra.run.smithi135.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-10-26T06:30:54.208 INFO:teuthology.orchestra.run.smithi135.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-10-26T06:30:54.316 INFO:teuthology.orchestra.run.smithi135.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-10-26T06:30:54.317 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-10-26T06:30:54.317 INFO:teuthology.orchestra.run.smithi135.stdout:Total 32 MB/s | 18 MB 00:00 2023-10-26T06:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-10-26T06:30:54.434 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================================ 2023-10-26T06:30:54.434 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-10-26T06:30:54.434 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================================ 2023-10-26T06:30:54.435 INFO:teuthology.orchestra.run.smithi192.stdout:Installing group/module packages: 2023-10-26T06:30:54.435 INFO:teuthology.orchestra.run.smithi192.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-26T06:30:54.435 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-26T06:30:54.435 INFO:teuthology.orchestra.run.smithi192.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-26T06:30:54.436 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-26T06:30:54.436 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-26T06:30:54.436 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-26T06:30:54.436 INFO:teuthology.orchestra.run.smithi192.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-26T06:30:54.436 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-10-26T06:30:54.437 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-26T06:30:54.437 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-26T06:30:54.437 INFO:teuthology.orchestra.run.smithi192.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-26T06:30:54.437 INFO:teuthology.orchestra.run.smithi192.stdout:Installing module profiles: 2023-10-26T06:30:54.438 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools/common 2023-10-26T06:30:54.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:30:54.438 INFO:teuthology.orchestra.run.smithi192.stdout:Enabling module streams: 2023-10-26T06:30:54.438 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools rhel8 2023-10-26T06:30:54.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:30:54.439 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-10-26T06:30:54.439 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================================ 2023-10-26T06:30:54.439 INFO:teuthology.orchestra.run.smithi192.stdout:Install 10 Packages 2023-10-26T06:30:54.439 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:30:54.440 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 18 M 2023-10-26T06:30:54.440 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 65 M 2023-10-26T06:30:54.440 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-10-26T06:30:54.491 INFO:teuthology.orchestra.run.smithi192.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 456 kB/s | 25 kB 00:00 2023-10-26T06:30:54.516 INFO:teuthology.orchestra.run.smithi192.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 512 kB/s | 41 kB 00:00 2023-10-26T06:30:54.517 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-10-26T06:30:54.535 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-10-26T06:30:54.536 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-10-26T06:30:54.541 INFO:teuthology.orchestra.run.smithi192.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 894 kB/s | 94 kB 00:00 2023-10-26T06:30:54.567 INFO:teuthology.orchestra.run.smithi192.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-10-26T06:30:54.600 INFO:teuthology.orchestra.run.smithi192.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-26T06:30:54.626 INFO:teuthology.orchestra.run.smithi192.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-26T06:30:54.651 INFO:teuthology.orchestra.run.smithi192.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-26T06:30:54.716 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-10-26T06:30:54.718 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-10-26T06:30:54.722 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-10-26T06:30:54.759 INFO:teuthology.orchestra.run.smithi192.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-10-26T06:30:54.812 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================================================ 2023-10-26T06:30:54.813 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-10-26T06:30:54.813 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================================================ 2023-10-26T06:30:54.813 INFO:teuthology.orchestra.run.smithi089.stdout:Installing group/module packages: 2023-10-26T06:30:54.813 INFO:teuthology.orchestra.run.smithi089.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-26T06:30:54.814 INFO:teuthology.orchestra.run.smithi089.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-26T06:30:54.814 INFO:teuthology.orchestra.run.smithi089.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-26T06:30:54.814 INFO:teuthology.orchestra.run.smithi089.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-26T06:30:54.814 INFO:teuthology.orchestra.run.smithi089.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-26T06:30:54.814 INFO:teuthology.orchestra.run.smithi089.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-26T06:30:54.814 INFO:teuthology.orchestra.run.smithi089.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-26T06:30:54.815 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-10-26T06:30:54.815 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-26T06:30:54.815 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-26T06:30:54.815 INFO:teuthology.orchestra.run.smithi089.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-26T06:30:54.815 INFO:teuthology.orchestra.run.smithi089.stdout:Installing module profiles: 2023-10-26T06:30:54.816 INFO:teuthology.orchestra.run.smithi089.stdout: container-tools/common 2023-10-26T06:30:54.816 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:30:54.816 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling module streams: 2023-10-26T06:30:54.816 INFO:teuthology.orchestra.run.smithi089.stdout: container-tools rhel8 2023-10-26T06:30:54.816 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:30:54.817 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-10-26T06:30:54.817 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================================================ 2023-10-26T06:30:54.817 INFO:teuthology.orchestra.run.smithi089.stdout:Install 10 Packages 2023-10-26T06:30:54.817 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:30:54.818 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 18 M 2023-10-26T06:30:54.818 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 65 M 2023-10-26T06:30:54.818 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-10-26T06:30:54.876 INFO:teuthology.orchestra.run.smithi192.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-10-26T06:30:54.888 INFO:teuthology.orchestra.run.smithi089.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-10-26T06:30:54.913 INFO:teuthology.orchestra.run.smithi089.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 256 kB/s | 25 kB 00:00 2023-10-26T06:30:54.938 INFO:teuthology.orchestra.run.smithi089.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 333 kB/s | 41 kB 00:00 2023-10-26T06:30:54.964 INFO:teuthology.orchestra.run.smithi089.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 651 kB/s | 49 kB 00:00 2023-10-26T06:30:54.997 INFO:teuthology.orchestra.run.smithi089.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-26T06:30:55.001 INFO:teuthology.orchestra.run.smithi192.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 17 MB/s | 8.0 MB 00:00 2023-10-26T06:30:55.002 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-10-26T06:30:55.002 INFO:teuthology.orchestra.run.smithi192.stdout:Total 32 MB/s | 18 MB 00:00 2023-10-26T06:30:55.056 INFO:teuthology.orchestra.run.smithi089.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.6 MB/s | 149 kB 00:00 2023-10-26T06:30:55.081 INFO:teuthology.orchestra.run.smithi089.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.5 MB/s | 213 kB 00:00 2023-10-26T06:30:55.198 INFO:teuthology.orchestra.run.smithi089.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-10-26T06:30:55.199 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-10-26T06:30:55.217 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-10-26T06:30:55.218 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-10-26T06:30:55.237 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-10-26T06:30:55.357 INFO:teuthology.orchestra.run.smithi089.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 23 MB/s | 7.0 MB 00:00 2023-10-26T06:30:55.399 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-10-26T06:30:55.403 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-10-26T06:30:55.431 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-26T06:30:55.449 INFO:teuthology.orchestra.run.smithi089.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-10-26T06:30:55.449 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-10-26T06:30:55.450 INFO:teuthology.orchestra.run.smithi089.stdout:Total 29 MB/s | 18 MB 00:00 2023-10-26T06:30:55.611 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-26T06:30:55.786 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-10-26T06:30:55.797 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-26T06:30:55.809 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-10-26T06:30:55.809 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-10-26T06:30:55.901 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-10-26T06:30:56.025 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-26T06:30:56.078 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-26T06:30:56.287 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-26T06:30:56.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-26T06:30:56.366 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-10-26T06:30:56.371 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-10-26T06:30:56.481 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-26T06:30:56.700 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-26T06:30:56.916 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-10-26T06:30:57.057 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-26T06:30:57.097 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-26T06:30:57.153 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-26T06:30:57.306 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-26T06:30:57.523 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-26T06:30:57.727 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-26T06:30:57.820 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-26T06:30:58.017 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-26T06:30:58.126 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-26T06:30:58.257 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-26T06:30:58.410 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-26T06:30:58.587 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-26T06:30:58.693 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-26T06:30:58.844 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-26T06:30:58.844 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-26T06:30:58.844 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-26T06:30:58.844 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-26T06:30:58.845 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-26T06:30:58.845 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-26T06:30:58.845 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-26T06:30:58.845 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-26T06:30:58.845 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-26T06:30:58.846 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-26T06:30:58.933 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-26T06:30:58.936 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-26T06:30:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-26T06:30:59.372 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-26T06:30:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-26T06:30:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-26T06:30:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-26T06:30:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-26T06:30:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-26T06:30:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-26T06:30:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-26T06:30:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-26T06:30:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-26T06:30:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-26T06:30:59.838 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-26T06:31:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-26T06:31:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-26T06:31:00.413 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-26T06:31:00.414 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:04:59 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:31:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-26T06:31:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-26T06:31:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-26T06:31:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-26T06:31:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-26T06:31:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-26T06:31:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-26T06:31:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-26T06:31:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-26T06:31:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-26T06:31:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-26T06:31:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-26T06:31:00.864 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:05:00 ago on Thu 26 Oct 2023 06:26:00 AM UTC. 2023-10-26T06:31:02.285 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-26T06:31:02.285 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:02 ago on Thu 26 Oct 2023 06:28:59 AM UTC. 2023-10-26T06:31:03.078 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-10-26T06:31:03.283 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:31:03.284 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-10-26T06:31:03.284 INFO:teuthology.orchestra.run.smithi135.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.284 INFO:teuthology.orchestra.run.smithi135.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-26T06:31:03.284 INFO:teuthology.orchestra.run.smithi135.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.285 INFO:teuthology.orchestra.run.smithi135.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-26T06:31:03.285 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-26T06:31:03.285 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-26T06:31:03.285 INFO:teuthology.orchestra.run.smithi135.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.285 INFO:teuthology.orchestra.run.smithi135.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.286 INFO:teuthology.orchestra.run.smithi135.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-26T06:31:03.286 INFO:teuthology.orchestra.run.smithi135.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-26T06:31:03.286 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:31:03.286 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-10-26T06:31:03.491 DEBUG:teuthology.parallel:result is None 2023-10-26T06:31:03.667 INFO:teuthology.orchestra.run.smithi192.stdout:Installed products updated. 2023-10-26T06:31:03.870 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:31:03.870 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-10-26T06:31:03.870 INFO:teuthology.orchestra.run.smithi192.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.871 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-26T06:31:03.871 INFO:teuthology.orchestra.run.smithi192.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-26T06:31:03.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-26T06:31:03.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-26T06:31:03.872 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.872 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:03.872 INFO:teuthology.orchestra.run.smithi192.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-26T06:31:03.872 INFO:teuthology.orchestra.run.smithi192.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-26T06:31:03.872 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:31:03.873 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-10-26T06:31:04.081 DEBUG:teuthology.parallel:result is None 2023-10-26T06:31:05.303 INFO:teuthology.orchestra.run.smithi089.stdout:Installed products updated. 2023-10-26T06:31:05.509 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:31:05.509 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-10-26T06:31:05.509 INFO:teuthology.orchestra.run.smithi089.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:05.509 INFO:teuthology.orchestra.run.smithi089.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-26T06:31:05.510 INFO:teuthology.orchestra.run.smithi089.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:05.510 INFO:teuthology.orchestra.run.smithi089.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-26T06:31:05.510 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-26T06:31:05.510 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-26T06:31:05.511 INFO:teuthology.orchestra.run.smithi089.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:05.511 INFO:teuthology.orchestra.run.smithi089.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-26T06:31:05.511 INFO:teuthology.orchestra.run.smithi089.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-26T06:31:05.511 INFO:teuthology.orchestra.run.smithi089.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-26T06:31:05.511 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:31:05.512 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-10-26T06:31:05.713 DEBUG:teuthology.parallel:result is None 2023-10-26T06:31:05.713 INFO:teuthology.run_tasks:Running task install... 2023-10-26T06:31:05.722 DEBUG:teuthology.task.install:project ceph 2023-10-26T06:31:05.723 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}} 2023-10-26T06:31:05.723 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'} 2023-10-26T06:31:05.723 INFO:teuthology.task.install:Using flavor: default 2023-10-26T06:31:05.729 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-26T06:31:05.730 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-26T06:31:05.730 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'tag': None, 'wait_for_package': False} 2023-10-26T06:31:05.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:31:05.733 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'tag': None, 'wait_for_package': False} 2023-10-26T06:31:05.733 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:31:05.734 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'tag': None, 'wait_for_package': False} 2023-10-26T06:31:05.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:31:05.898 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/ 2023-10-26T06:31:05.899 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6863.g38499368 2023-10-26T06:31:05.969 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/ 2023-10-26T06:31:05.970 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6863.g38499368 2023-10-26T06:31:05.975 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/ 2023-10-26T06:31:05.976 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6863.g38499368 2023-10-26T06:31:06.070 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-26T06:31:06.071 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:31:06.071 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-26T06:31:06.078 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-26T06:31:06.078 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:31:06.079 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-26T06:31:06.106 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-26T06:31:06.107 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/384993681abceb8cb8affea10538d607052db4b4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-26T06:31:06.139 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-26T06:31:06.140 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/384993681abceb8cb8affea10538d607052db4b4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-26T06:31:06.143 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-26T06:31:06.144 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:31:06.144 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-26T06:31:06.170 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-26T06:31:06.170 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/384993681abceb8cb8affea10538d607052db4b4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-26T06:31:06.174 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-26T06:31:06.237 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-26T06:31:06.250 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-26T06:31:06.252 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-26T06:31:06.315 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-26T06:31:06.319 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2023-10-26T06:31:06.321 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2023-10-26T06:31:06.345 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-26T06:31:06.380 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2023-10-26T06:31:06.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2023-10-26T06:31:06.413 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2023-10-26T06:31:06.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2023-10-26T06:31:06.716 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-10-26T06:31:06.782 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-10-26T06:31:06.862 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-26T06:31:07.015 INFO:teuthology.orchestra.run.smithi135.stdout:44 files removed 2023-10-26T06:31:07.045 INFO:teuthology.orchestra.run.smithi192.stdout:44 files removed 2023-10-26T06:31:07.077 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-26T06:31:07.100 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-26T06:31:07.182 INFO:teuthology.orchestra.run.smithi089.stdout:44 files removed 2023-10-26T06:31:07.238 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-26T06:31:07.474 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-10-26T06:31:07.520 INFO:teuthology.orchestra.run.smithi192.stdout:Updating Subscription Management repositories. 2023-10-26T06:31:07.686 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-10-26T06:31:08.210 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 257 kB/s | 81 kB 00:00 2023-10-26T06:31:08.409 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 183 kB/s | 81 kB 00:00 2023-10-26T06:31:08.604 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-10-26T06:31:08.619 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 200 kB/s | 81 kB 00:00 2023-10-26T06:31:08.676 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2023-10-26T06:31:08.936 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-10-26T06:31:08.937 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2023-10-26T06:31:08.980 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-10-26T06:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-10-26T06:31:09.131 INFO:teuthology.orchestra.run.smithi135.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-26T06:31:09.303 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2023-10-26T06:31:09.455 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-26T06:31:11.201 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2023-10-26T06:31:13.558 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 16 MB 00:04 2023-10-26T06:31:14.964 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 216 kB/s | 24 kB 00:00 2023-10-26T06:31:15.937 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-10-26T06:31:17.124 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 524 kB/s | 24 kB 00:00 2023-10-26T06:31:18.012 INFO:teuthology.orchestra.run.smithi135.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-10-26T06:31:23.869 INFO:teuthology.orchestra.run.smithi192.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 53 MB 00:00 2023-10-26T06:31:25.877 INFO:teuthology.orchestra.run.smithi135.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-10-26T06:31:35.053 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-26T06:31:35.054 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-26T06:31:35.069 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-26T06:31:35.070 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-26T06:31:35.071 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-26T06:31:35.072 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-26T06:31:35.074 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-26T06:31:35.077 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-26T06:31:35.078 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-26T06:31:35.213 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-10-26T06:31:35.225 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-10-26T06:31:35.225 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-10-26T06:31:35.226 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-10-26T06:31:35.226 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2023-10-26T06:31:35.226 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-26T06:31:35.226 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-26T06:31:35.226 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:18.0.0-6863.g38499368.el8 ceph 6.4 k 2023-10-26T06:31:35.227 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:18.0.0-6863.g38499368.el8 ceph 5.4 M 2023-10-26T06:31:35.227 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:18.0.0-6863.g38499368.el8 ceph 918 k 2023-10-26T06:31:35.227 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6863.g38499368.el8 ceph 167 k 2023-10-26T06:31:35.227 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:18.0.0-6863.g38499368.el8 ceph 1.6 M 2023-10-26T06:31:35.227 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 132 k 2023-10-26T06:31:35.228 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 1.8 M 2023-10-26T06:31:35.228 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 7.4 M 2023-10-26T06:31:35.228 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 48 k 2023-10-26T06:31:35.228 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:18.0.0-6863.g38499368.el8 ceph 14 M 2023-10-26T06:31:35.228 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:18.0.0-6863.g38499368.el8 ceph 49 M 2023-10-26T06:31:35.228 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 261 k 2023-10-26T06:31:35.229 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 191 k 2023-10-26T06:31:35.229 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:18.0.0-6863.g38499368.el8 ceph 261 k 2023-10-26T06:31:35.229 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 18 k 2023-10-26T06:31:35.229 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-26T06:31:35.229 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-26T06:31:35.230 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-26T06:31:35.230 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:18.0.0-6863.g38499368.el8 ceph 31 k 2023-10-26T06:31:35.230 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 840 k 2023-10-26T06:31:35.230 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:18.0.0-6863.g38499368.el8 ceph 139 k 2023-10-26T06:31:35.230 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-26T06:31:35.231 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-26T06:31:35.231 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-26T06:31:35.231 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-26T06:31:35.231 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-26T06:31:35.231 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-26T06:31:35.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:18.0.0-6863.g38499368.el8 ceph 197 k 2023-10-26T06:31:35.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:18.0.0-6863.g38499368.el8 ceph 371 k 2023-10-26T06:31:35.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:18.0.0-6863.g38499368.el8 ceph 364 k 2023-10-26T06:31:35.232 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:18.0.0-6863.g38499368.el8 ceph 107 k 2023-10-26T06:31:35.232 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:18.0.0-6863.g38499368.el8 ceph 94 k 2023-10-26T06:31:35.232 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.6 M 2023-10-26T06:31:35.233 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:18.0.0-6863.g38499368.el8 ceph 179 k 2023-10-26T06:31:35.233 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-26T06:31:35.233 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2023-10-26T06:31:35.233 INFO:teuthology.orchestra.run.smithi192.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-26T06:31:35.233 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.7 M 2023-10-26T06:31:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.7 M 2023-10-26T06:31:35.234 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-10-26T06:31:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-26T06:31:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:18.0.0-6863.g38499368.el8 ceph 23 M 2023-10-26T06:31:35.234 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 23 k 2023-10-26T06:31:35.235 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:18.0.0-6863.g38499368.el8 ceph 2.3 M 2023-10-26T06:31:35.235 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 242 k 2023-10-26T06:31:35.235 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:18.0.0-6863.g38499368.el8 ceph 4.6 M 2023-10-26T06:31:35.235 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:18.0.0-6863.g38499368.el8 ceph 18 M 2023-10-26T06:31:35.235 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 15 k 2023-10-26T06:31:35.235 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:18.0.0-6863.g38499368.el8 ceph 24 k 2023-10-26T06:31:35.236 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-26T06:31:35.236 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-26T06:31:35.236 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-26T06:31:35.236 INFO:teuthology.orchestra.run.smithi192.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-26T06:31:35.236 INFO:teuthology.orchestra.run.smithi192.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-26T06:31:35.237 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:18.0.0-6863.g38499368.el8 ceph 184 k 2023-10-26T06:31:35.237 INFO:teuthology.orchestra.run.smithi192.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-26T06:31:35.237 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-26T06:31:35.237 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-26T06:31:35.237 INFO:teuthology.orchestra.run.smithi192.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-26T06:31:35.238 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-26T06:31:35.238 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-26T06:31:35.238 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:18.0.0-6863.g38499368.el8 ceph 518 k 2023-10-26T06:31:35.238 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-26T06:31:35.238 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 6.9 M 2023-10-26T06:31:35.239 INFO:teuthology.orchestra.run.smithi192.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-26T06:31:35.239 INFO:teuthology.orchestra.run.smithi192.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-26T06:31:35.239 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-26T06:31:35.239 INFO:teuthology.orchestra.run.smithi192.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-26T06:31:35.239 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-26T06:31:35.240 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-26T06:31:35.240 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-26T06:31:35.240 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-26T06:31:35.240 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-26T06:31:35.240 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-26T06:31:35.241 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-26T06:31:35.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-26T06:31:35.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-26T06:31:35.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-26T06:31:35.241 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-26T06:31:35.242 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-26T06:31:35.242 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:18.0.0-6863.g38499368.el8 ceph 45 k 2023-10-26T06:31:35.242 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:18.0.0-6863.g38499368.el8 ceph 125 k 2023-10-26T06:31:35.242 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-26T06:31:35.242 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-26T06:31:35.243 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-26T06:31:35.243 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-26T06:31:35.243 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-26T06:31:35.243 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-26T06:31:35.243 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-26T06:31:35.243 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-26T06:31:35.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-26T06:31:35.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-26T06:31:35.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-26T06:31:35.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-26T06:31:35.244 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-26T06:31:35.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-26T06:31:35.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-26T06:31:35.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-26T06:31:35.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-26T06:31:35.245 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-26T06:31:35.246 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-26T06:31:35.246 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-26T06:31:35.246 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-26T06:31:35.246 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-26T06:31:35.246 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-26T06:31:35.246 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-26T06:31:35.247 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-26T06:31:35.247 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-26T06:31:35.247 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-26T06:31:35.247 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-26T06:31:35.247 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-26T06:31:35.248 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-26T06:31:35.248 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-26T06:31:35.248 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-26T06:31:35.248 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-26T06:31:35.248 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-26T06:31:35.249 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-26T06:31:35.249 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-26T06:31:35.249 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-26T06:31:35.249 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-26T06:31:35.249 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-26T06:31:35.249 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-26T06:31:35.250 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-26T06:31:35.250 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-26T06:31:35.250 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-26T06:31:35.250 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-26T06:31:35.250 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-26T06:31:35.251 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-26T06:31:35.251 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-26T06:31:35.251 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-26T06:31:35.251 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-26T06:31:35.251 INFO:teuthology.orchestra.run.smithi192.stdout:Enabling module streams: 2023-10-26T06:31:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql 10 2023-10-26T06:31:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:31:35.252 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-10-26T06:31:35.252 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================================================== 2023-10-26T06:31:35.252 INFO:teuthology.orchestra.run.smithi192.stdout:Install 128 Packages 2023-10-26T06:31:35.252 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 3 Packages 2023-10-26T06:31:35.253 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:31:35.253 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 209 M 2023-10-26T06:31:35.254 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-10-26T06:31:35.628 INFO:teuthology.orchestra.run.smithi192.stdout:(1/131): ceph-18.0.0-6863.g38499368.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-10-26T06:31:36.428 INFO:teuthology.orchestra.run.smithi192.stdout:(2/131): ceph-fuse-18.0.0-6863.g38499368.el8.x8 1.1 MB/s | 918 kB 00:00 2023-10-26T06:31:36.556 INFO:teuthology.orchestra.run.smithi192.stdout:(3/131): ceph-immutable-object-cache-18.0.0-686 1.3 MB/s | 167 kB 00:00 2023-10-26T06:31:37.031 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-26T06:31:37.032 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-26T06:31:37.046 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-26T06:31:37.047 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-26T06:31:37.048 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-26T06:31:37.049 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-26T06:31:37.051 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-26T06:31:37.054 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-26T06:31:37.056 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-26T06:31:37.188 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-10-26T06:31:37.200 INFO:teuthology.orchestra.run.smithi135.stdout:=========================================================================================================================== 2023-10-26T06:31:37.201 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-10-26T06:31:37.201 INFO:teuthology.orchestra.run.smithi135.stdout:=========================================================================================================================== 2023-10-26T06:31:37.201 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-10-26T06:31:37.201 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-26T06:31:37.202 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-26T06:31:37.202 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:18.0.0-6863.g38499368.el8 ceph 6.4 k 2023-10-26T06:31:37.202 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:18.0.0-6863.g38499368.el8 ceph 5.4 M 2023-10-26T06:31:37.203 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:18.0.0-6863.g38499368.el8 ceph 918 k 2023-10-26T06:31:37.203 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6863.g38499368.el8 ceph 167 k 2023-10-26T06:31:37.203 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:18.0.0-6863.g38499368.el8 ceph 1.6 M 2023-10-26T06:31:37.204 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 132 k 2023-10-26T06:31:37.204 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 1.8 M 2023-10-26T06:31:37.204 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 7.4 M 2023-10-26T06:31:37.204 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 48 k 2023-10-26T06:31:37.205 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:18.0.0-6863.g38499368.el8 ceph 14 M 2023-10-26T06:31:37.205 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:18.0.0-6863.g38499368.el8 ceph 49 M 2023-10-26T06:31:37.206 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 261 k 2023-10-26T06:31:37.206 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 191 k 2023-10-26T06:31:37.206 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:18.0.0-6863.g38499368.el8 ceph 261 k 2023-10-26T06:31:37.207 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 18 k 2023-10-26T06:31:37.207 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-26T06:31:37.207 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-26T06:31:37.207 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-26T06:31:37.208 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:18.0.0-6863.g38499368.el8 ceph 31 k 2023-10-26T06:31:37.208 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 840 k 2023-10-26T06:31:37.208 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:18.0.0-6863.g38499368.el8 ceph 139 k 2023-10-26T06:31:37.208 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-26T06:31:37.208 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-26T06:31:37.208 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-26T06:31:37.209 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-26T06:31:37.209 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-26T06:31:37.209 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-26T06:31:37.209 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:18.0.0-6863.g38499368.el8 ceph 197 k 2023-10-26T06:31:37.209 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:18.0.0-6863.g38499368.el8 ceph 371 k 2023-10-26T06:31:37.210 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:18.0.0-6863.g38499368.el8 ceph 364 k 2023-10-26T06:31:37.210 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:18.0.0-6863.g38499368.el8 ceph 107 k 2023-10-26T06:31:37.210 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:18.0.0-6863.g38499368.el8 ceph 94 k 2023-10-26T06:31:37.210 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.6 M 2023-10-26T06:31:37.210 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:18.0.0-6863.g38499368.el8 ceph 179 k 2023-10-26T06:31:37.211 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-26T06:31:37.211 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2023-10-26T06:31:37.212 INFO:teuthology.orchestra.run.smithi135.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-26T06:31:37.212 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.7 M 2023-10-26T06:31:37.212 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.7 M 2023-10-26T06:31:37.212 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-10-26T06:31:37.213 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-26T06:31:37.213 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:18.0.0-6863.g38499368.el8 ceph 23 M 2023-10-26T06:31:37.213 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 23 k 2023-10-26T06:31:37.214 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:18.0.0-6863.g38499368.el8 ceph 2.3 M 2023-10-26T06:31:37.214 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 242 k 2023-10-26T06:31:37.214 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:18.0.0-6863.g38499368.el8 ceph 4.6 M 2023-10-26T06:31:37.215 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:18.0.0-6863.g38499368.el8 ceph 18 M 2023-10-26T06:31:37.215 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 15 k 2023-10-26T06:31:37.215 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:18.0.0-6863.g38499368.el8 ceph 24 k 2023-10-26T06:31:37.215 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-26T06:31:37.215 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-26T06:31:37.216 INFO:teuthology.orchestra.run.smithi135.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-26T06:31:37.216 INFO:teuthology.orchestra.run.smithi135.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-26T06:31:37.216 INFO:teuthology.orchestra.run.smithi135.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-26T06:31:37.217 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:18.0.0-6863.g38499368.el8 ceph 184 k 2023-10-26T06:31:37.217 INFO:teuthology.orchestra.run.smithi135.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-26T06:31:37.217 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-26T06:31:37.217 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-26T06:31:37.217 INFO:teuthology.orchestra.run.smithi135.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-26T06:31:37.217 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-26T06:31:37.218 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-26T06:31:37.218 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:18.0.0-6863.g38499368.el8 ceph 518 k 2023-10-26T06:31:37.218 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-26T06:31:37.218 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 6.9 M 2023-10-26T06:31:37.218 INFO:teuthology.orchestra.run.smithi135.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-26T06:31:37.219 INFO:teuthology.orchestra.run.smithi135.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-26T06:31:37.219 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-26T06:31:37.219 INFO:teuthology.orchestra.run.smithi135.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-26T06:31:37.219 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-26T06:31:37.219 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-26T06:31:37.220 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-26T06:31:37.220 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-26T06:31:37.220 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-26T06:31:37.220 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-26T06:31:37.221 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-26T06:31:37.221 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-26T06:31:37.221 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-26T06:31:37.221 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-26T06:31:37.222 INFO:teuthology.orchestra.run.smithi135.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-26T06:31:37.222 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-26T06:31:37.222 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:18.0.0-6863.g38499368.el8 ceph 45 k 2023-10-26T06:31:37.222 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:18.0.0-6863.g38499368.el8 ceph 125 k 2023-10-26T06:31:37.222 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-26T06:31:37.223 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-26T06:31:37.223 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-26T06:31:37.223 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-26T06:31:37.223 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-26T06:31:37.223 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-26T06:31:37.223 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-26T06:31:37.224 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-26T06:31:37.224 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-26T06:31:37.224 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-26T06:31:37.224 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-26T06:31:37.224 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-26T06:31:37.225 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-26T06:31:37.225 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-26T06:31:37.225 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-26T06:31:37.225 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-26T06:31:37.225 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-26T06:31:37.226 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-26T06:31:37.226 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-26T06:31:37.226 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-26T06:31:37.226 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-26T06:31:37.226 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-26T06:31:37.227 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-26T06:31:37.227 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-26T06:31:37.227 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-26T06:31:37.228 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-26T06:31:37.228 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-26T06:31:37.228 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-26T06:31:37.228 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-26T06:31:37.229 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-26T06:31:37.229 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-26T06:31:37.229 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-26T06:31:37.229 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-26T06:31:37.229 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-26T06:31:37.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-26T06:31:37.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-26T06:31:37.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-26T06:31:37.230 INFO:teuthology.orchestra.run.smithi135.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-26T06:31:37.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-26T06:31:37.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-26T06:31:37.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-26T06:31:37.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-26T06:31:37.231 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-26T06:31:37.232 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-26T06:31:37.232 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-26T06:31:37.232 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-26T06:31:37.233 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-26T06:31:37.233 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-26T06:31:37.233 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-26T06:31:37.233 INFO:teuthology.orchestra.run.smithi135.stdout:Enabling module streams: 2023-10-26T06:31:37.233 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql 10 2023-10-26T06:31:37.234 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:31:37.234 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-10-26T06:31:37.234 INFO:teuthology.orchestra.run.smithi135.stdout:=========================================================================================================================== 2023-10-26T06:31:37.234 INFO:teuthology.orchestra.run.smithi135.stdout:Install 128 Packages 2023-10-26T06:31:37.235 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 3 Packages 2023-10-26T06:31:37.235 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:31:37.236 INFO:teuthology.orchestra.run.smithi192.stdout:(4/131): ceph-mds-18.0.0-6863.g38499368.el8.x86 3.5 MB/s | 2.3 MB 00:00 2023-10-26T06:31:37.236 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 209 M 2023-10-26T06:31:37.236 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-10-26T06:31:37.296 INFO:teuthology.orchestra.run.smithi192.stdout:(5/131): ceph-base-18.0.0-6863.g38499368.el8.x8 2.8 MB/s | 5.4 MB 00:01 2023-10-26T06:31:37.404 INFO:teuthology.orchestra.run.smithi135.stdout:(1/131): ceph-18.0.0-6863.g38499368.el8.x86_64. 48 kB/s | 6.4 kB 00:00 2023-10-26T06:31:37.605 INFO:teuthology.orchestra.run.smithi192.stdout:(6/131): ceph-mgr-18.0.0-6863.g38499368.el8.x86 4.2 MB/s | 1.6 MB 00:00 2023-10-26T06:31:38.029 INFO:teuthology.orchestra.run.smithi135.stdout:(2/131): ceph-fuse-18.0.0-6863.g38499368.el8.x8 1.4 MB/s | 918 kB 00:00 2023-10-26T06:31:38.030 INFO:teuthology.orchestra.run.smithi192.stdout:(7/131): ceph-mon-18.0.0-6863.g38499368.el8.x86 6.3 MB/s | 4.6 MB 00:00 2023-10-26T06:31:38.213 INFO:teuthology.orchestra.run.smithi135.stdout:(3/131): ceph-immutable-object-cache-18.0.0-686 914 kB/s | 167 kB 00:00 2023-10-26T06:31:38.605 INFO:teuthology.orchestra.run.smithi192.stdout:(8/131): ceph-common-18.0.0-6863.g38499368.el8. 7.0 MB/s | 23 MB 00:03 2023-10-26T06:31:38.688 INFO:teuthology.orchestra.run.smithi192.stdout:(9/131): ceph-selinux-18.0.0-6863.g38499368.el8 293 kB/s | 24 kB 00:00 2023-10-26T06:31:39.739 INFO:teuthology.orchestra.run.smithi135.stdout:(4/131): ceph-mds-18.0.0-6863.g38499368.el8.x86 1.5 MB/s | 2.3 MB 00:01 2023-10-26T06:31:39.788 INFO:teuthology.orchestra.run.smithi192.stdout:(10/131): ceph-radosgw-18.0.0-6863.g38499368.el 7.9 MB/s | 14 MB 00:01 2023-10-26T06:31:39.889 INFO:teuthology.orchestra.run.smithi192.stdout:(11/131): cephfs-mirror-18.0.0-6863.g38499368.e 2.6 MB/s | 261 kB 00:00 2023-10-26T06:31:39.972 INFO:teuthology.orchestra.run.smithi192.stdout:(12/131): libcephfs-devel-18.0.0-6863.g38499368 373 kB/s | 31 kB 00:00 2023-10-26T06:31:39.987 INFO:teuthology.orchestra.run.smithi135.stdout:(5/131): ceph-base-18.0.0-6863.g38499368.el8.x8 2.0 MB/s | 5.4 MB 00:02 2023-10-26T06:31:40.164 INFO:teuthology.orchestra.run.smithi192.stdout:(13/131): libcephfs2-18.0.0-6863.g38499368.el8. 4.3 MB/s | 840 kB 00:00 2023-10-26T06:31:40.264 INFO:teuthology.orchestra.run.smithi192.stdout:(14/131): libcephsqlite-18.0.0-6863.g38499368.e 1.8 MB/s | 184 kB 00:00 2023-10-26T06:31:40.365 INFO:teuthology.orchestra.run.smithi192.stdout:(15/131): librados-devel-18.0.0-6863.g38499368. 1.4 MB/s | 139 kB 00:00 2023-10-26T06:31:40.582 INFO:teuthology.orchestra.run.smithi135.stdout:(6/131): ceph-mgr-18.0.0-6863.g38499368.el8.x86 1.9 MB/s | 1.6 MB 00:00 2023-10-26T06:31:40.615 INFO:teuthology.orchestra.run.smithi192.stdout:(16/131): ceph-osd-18.0.0-6863.g38499368.el8.x8 6.0 MB/s | 18 MB 00:03 2023-10-26T06:31:40.657 INFO:teuthology.orchestra.run.smithi192.stdout:(17/131): libradosstriper1-18.0.0-6863.g3849936 1.7 MB/s | 518 kB 00:00 2023-10-26T06:31:40.757 INFO:teuthology.orchestra.run.smithi192.stdout:(18/131): python3-ceph-argparse-18.0.0-6863.g38 447 kB/s | 45 kB 00:00 2023-10-26T06:31:40.882 INFO:teuthology.orchestra.run.smithi192.stdout:(19/131): python3-ceph-common-18.0.0-6863.g3849 1.0 MB/s | 125 kB 00:00 2023-10-26T06:31:41.000 INFO:teuthology.orchestra.run.smithi192.stdout:(20/131): python3-cephfs-18.0.0-6863.g38499368. 1.6 MB/s | 197 kB 00:00 2023-10-26T06:31:41.108 INFO:teuthology.orchestra.run.smithi192.stdout:(21/131): python3-rados-18.0.0-6863.g38499368.e 3.4 MB/s | 371 kB 00:00 2023-10-26T06:31:41.250 INFO:teuthology.orchestra.run.smithi192.stdout:(22/131): python3-rbd-18.0.0-6863.g38499368.el8 2.5 MB/s | 364 kB 00:00 2023-10-26T06:31:41.342 INFO:teuthology.orchestra.run.smithi192.stdout:(23/131): python3-rgw-18.0.0-6863.g38499368.el8 1.1 MB/s | 107 kB 00:00 2023-10-26T06:31:41.356 INFO:teuthology.orchestra.run.smithi135.stdout:(7/131): ceph-mon-18.0.0-6863.g38499368.el8.x86 3.4 MB/s | 4.6 MB 00:01 2023-10-26T06:31:41.442 INFO:teuthology.orchestra.run.smithi192.stdout:(24/131): rbd-fuse-18.0.0-6863.g38499368.el8.x8 942 kB/s | 94 kB 00:00 2023-10-26T06:31:41.901 INFO:teuthology.orchestra.run.smithi192.stdout:(25/131): librgw2-18.0.0-6863.g38499368.el8.x86 5.4 MB/s | 6.9 MB 00:01 2023-10-26T06:31:41.952 INFO:teuthology.orchestra.run.smithi192.stdout:(26/131): rbd-mirror-18.0.0-6863.g38499368.el8. 7.0 MB/s | 3.6 MB 00:00 2023-10-26T06:31:41.977 INFO:teuthology.orchestra.run.smithi192.stdout:(27/131): rbd-nbd-18.0.0-6863.g38499368.el8.x86 2.3 MB/s | 179 kB 00:00 2023-10-26T06:31:42.027 INFO:teuthology.orchestra.run.smithi192.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6863.g 307 kB/s | 23 kB 00:00 2023-10-26T06:31:42.044 INFO:teuthology.orchestra.run.smithi192.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6863.g3849936 1.9 MB/s | 132 kB 00:00 2023-10-26T06:31:42.278 INFO:teuthology.orchestra.run.smithi192.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6863.g38499 7.1 MB/s | 1.8 MB 00:00 2023-10-26T06:31:42.386 INFO:teuthology.orchestra.run.smithi192.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6863.g38 2.2 MB/s | 242 kB 00:00 2023-10-26T06:31:42.453 INFO:teuthology.orchestra.run.smithi192.stdout:(32/131): ceph-mgr-rook-18.0.0-6863.g38499368.e 714 kB/s | 48 kB 00:00 2023-10-26T06:31:42.528 INFO:teuthology.orchestra.run.smithi192.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6863.g3 200 kB/s | 15 kB 00:00 2023-10-26T06:31:42.654 INFO:teuthology.orchestra.run.smithi192.stdout:(34/131): ceph-volume-18.0.0-6863.g38499368.el8 2.0 MB/s | 261 kB 00:00 2023-10-26T06:31:42.754 INFO:teuthology.orchestra.run.smithi192.stdout:(35/131): cephadm-18.0.0-6863.g38499368.el8.noa 1.9 MB/s | 191 kB 00:00 2023-10-26T06:31:42.821 INFO:teuthology.orchestra.run.smithi192.stdout:(36/131): cephfs-top-18.0.0-6863.g38499368.el8. 271 kB/s | 18 kB 00:00 2023-10-26T06:31:43.196 INFO:teuthology.orchestra.run.smithi192.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 817 kB/s | 306 kB 00:00 2023-10-26T06:31:43.288 INFO:teuthology.orchestra.run.smithi192.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 5.9 MB/s | 7.4 MB 00:01 2023-10-26T06:31:43.314 INFO:teuthology.orchestra.run.smithi192.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 503 kB/s | 59 kB 00:00 2023-10-26T06:31:43.439 INFO:teuthology.orchestra.run.smithi192.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2023-10-26T06:31:43.506 INFO:teuthology.orchestra.run.smithi192.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 655 kB/s | 44 kB 00:00 2023-10-26T06:31:43.564 INFO:teuthology.orchestra.run.smithi192.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 272 kB/s | 75 kB 00:00 2023-10-26T06:31:43.590 INFO:teuthology.orchestra.run.smithi192.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-10-26T06:31:43.624 INFO:teuthology.orchestra.run.smithi135.stdout:(8/131): ceph-common-18.0.0-6863.g38499368.el8. 3.6 MB/s | 23 MB 00:06 2023-10-26T06:31:43.657 INFO:teuthology.orchestra.run.smithi192.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 358 kB/s | 33 kB 00:00 2023-10-26T06:31:43.673 INFO:teuthology.orchestra.run.smithi135.stdout:(9/131): ceph-selinux-18.0.0-6863.g38499368.el8 495 kB/s | 24 kB 00:00 2023-10-26T06:31:43.682 INFO:teuthology.orchestra.run.smithi192.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 176 kB/s | 16 kB 00:00 2023-10-26T06:31:43.766 INFO:teuthology.orchestra.run.smithi192.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-10-26T06:31:43.791 INFO:teuthology.orchestra.run.smithi192.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-10-26T06:31:43.858 INFO:teuthology.orchestra.run.smithi192.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 884 kB/s | 81 kB 00:00 2023-10-26T06:31:43.883 INFO:teuthology.orchestra.run.smithi192.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-10-26T06:31:43.926 INFO:teuthology.orchestra.run.smithi192.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-10-26T06:31:44.009 INFO:teuthology.orchestra.run.smithi192.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 584 kB/s | 49 kB 00:00 2023-10-26T06:31:44.084 INFO:teuthology.orchestra.run.smithi192.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 784 kB/s | 59 kB 00:00 2023-10-26T06:31:44.118 INFO:teuthology.orchestra.run.smithi192.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-10-26T06:31:44.168 INFO:teuthology.orchestra.run.smithi192.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 722 kB/s | 60 kB 00:00 2023-10-26T06:31:44.193 INFO:teuthology.orchestra.run.smithi192.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-10-26T06:31:44.235 INFO:teuthology.orchestra.run.smithi192.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-10-26T06:31:44.240 INFO:teuthology.orchestra.run.smithi135.stdout:(10/131): ceph-radosgw-18.0.0-6863.g38499368.el 4.8 MB/s | 14 MB 00:02 2023-10-26T06:31:44.261 INFO:teuthology.orchestra.run.smithi192.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2023-10-26T06:31:44.303 INFO:teuthology.orchestra.run.smithi192.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-10-26T06:31:44.316 INFO:teuthology.orchestra.run.smithi135.stdout:(11/131): cephfs-mirror-18.0.0-6863.g38499368.e 3.4 MB/s | 261 kB 00:00 2023-10-26T06:31:44.328 INFO:teuthology.orchestra.run.smithi192.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 962 kB/s | 64 kB 00:00 2023-10-26T06:31:44.370 INFO:teuthology.orchestra.run.smithi192.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2023-10-26T06:31:44.383 INFO:teuthology.orchestra.run.smithi135.stdout:(12/131): libcephfs-devel-18.0.0-6863.g38499368 466 kB/s | 31 kB 00:00 2023-10-26T06:31:44.395 INFO:teuthology.orchestra.run.smithi192.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 358 kB/s | 24 kB 00:00 2023-10-26T06:31:44.437 INFO:teuthology.orchestra.run.smithi192.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 436 kB/s | 29 kB 00:00 2023-10-26T06:31:44.462 INFO:teuthology.orchestra.run.smithi192.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2023-10-26T06:31:44.521 INFO:teuthology.orchestra.run.smithi192.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-10-26T06:31:44.546 INFO:teuthology.orchestra.run.smithi192.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2023-10-26T06:31:44.583 INFO:teuthology.orchestra.run.smithi135.stdout:(13/131): libcephfs2-18.0.0-6863.g38499368.el8. 4.1 MB/s | 840 kB 00:00 2023-10-26T06:31:44.597 INFO:teuthology.orchestra.run.smithi192.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 814 kB/s | 61 kB 00:00 2023-10-26T06:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-26T06:31:44.658 INFO:teuthology.orchestra.run.smithi135.stdout:(14/131): libcephsqlite-18.0.0-6863.g38499368.e 2.4 MB/s | 184 kB 00:00 2023-10-26T06:31:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 343 kB/s | 23 kB 00:00 2023-10-26T06:31:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-26T06:31:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-10-26T06:31:44.751 INFO:teuthology.orchestra.run.smithi135.stdout:(15/131): librados-devel-18.0.0-6863.g38499368. 1.5 MB/s | 139 kB 00:00 2023-10-26T06:31:44.806 INFO:teuthology.orchestra.run.smithi192.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-10-26T06:31:44.840 INFO:teuthology.orchestra.run.smithi192.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-26T06:31:44.867 INFO:teuthology.orchestra.run.smithi135.stdout:(16/131): libradosstriper1-18.0.0-6863.g3849936 4.3 MB/s | 518 kB 00:00 2023-10-26T06:31:44.882 INFO:teuthology.orchestra.run.smithi192.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-10-26T06:31:44.924 INFO:teuthology.orchestra.run.smithi192.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-10-26T06:31:44.958 INFO:teuthology.orchestra.run.smithi192.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-10-26T06:31:45.008 INFO:teuthology.orchestra.run.smithi192.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-10-26T06:31:45.083 INFO:teuthology.orchestra.run.smithi192.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-10-26T06:31:45.225 INFO:teuthology.orchestra.run.smithi192.stdout:(78/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-10-26T06:31:45.250 INFO:teuthology.orchestra.run.smithi192.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 941 kB/s | 157 kB 00:00 2023-10-26T06:31:45.276 INFO:teuthology.orchestra.run.smithi192.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-26T06:31:45.310 INFO:teuthology.orchestra.run.smithi192.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-10-26T06:31:45.352 INFO:teuthology.orchestra.run.smithi192.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 641 kB/s | 27 kB 00:00 2023-10-26T06:31:45.393 INFO:teuthology.orchestra.run.smithi192.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-26T06:31:45.436 INFO:teuthology.orchestra.run.smithi192.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-10-26T06:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-10-26T06:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-10-26T06:31:45.636 INFO:teuthology.orchestra.run.smithi192.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-10-26T06:31:45.661 INFO:teuthology.orchestra.run.smithi192.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-10-26T06:31:45.687 INFO:teuthology.orchestra.run.smithi192.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-26T06:31:46.093 INFO:teuthology.orchestra.run.smithi135.stdout:(17/131): librgw2-18.0.0-6863.g38499368.el8.x86 5.7 MB/s | 6.9 MB 00:01 2023-10-26T06:31:46.112 INFO:teuthology.orchestra.run.smithi192.stdout:(90/131): ceph-test-18.0.0-6863.g38499368.el8.x 6.6 MB/s | 49 MB 00:07 2023-10-26T06:31:46.146 INFO:teuthology.orchestra.run.smithi192.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 44 kB/s | 20 kB 00:00 2023-10-26T06:31:46.179 INFO:teuthology.orchestra.run.smithi192.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 244 kB/s | 126 kB 00:00 2023-10-26T06:31:46.193 INFO:teuthology.orchestra.run.smithi135.stdout:(18/131): python3-ceph-argparse-18.0.0-6863.g38 449 kB/s | 45 kB 00:00 2023-10-26T06:31:46.196 INFO:teuthology.orchestra.run.smithi192.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.2 MB/s | 103 kB 00:00 2023-10-26T06:31:46.230 INFO:teuthology.orchestra.run.smithi192.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 8.9 MB/s | 709 kB 00:00 2023-10-26T06:31:46.247 INFO:teuthology.orchestra.run.smithi192.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-10-26T06:31:46.252 INFO:teuthology.orchestra.run.smithi135.stdout:(19/131): python3-ceph-common-18.0.0-6863.g3849 2.1 MB/s | 125 kB 00:00 2023-10-26T06:31:46.297 INFO:teuthology.orchestra.run.smithi192.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-10-26T06:31:46.419 INFO:teuthology.orchestra.run.smithi135.stdout:(20/131): ceph-osd-18.0.0-6863.g38499368.el8.x8 3.1 MB/s | 18 MB 00:05 2023-10-26T06:31:46.444 INFO:teuthology.orchestra.run.smithi135.stdout:(21/131): python3-cephfs-18.0.0-6863.g38499368. 1.0 MB/s | 197 kB 00:00 2023-10-26T06:31:46.511 INFO:teuthology.orchestra.run.smithi135.stdout:(22/131): python3-rados-18.0.0-6863.g38499368.e 4.0 MB/s | 371 kB 00:00 2023-10-26T06:31:46.533 INFO:teuthology.orchestra.run.smithi192.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2023-10-26T06:31:46.545 INFO:teuthology.orchestra.run.smithi135.stdout:(23/131): python3-rbd-18.0.0-6863.g38499368.el8 3.5 MB/s | 364 kB 00:00 2023-10-26T06:31:46.595 INFO:teuthology.orchestra.run.smithi135.stdout:(24/131): python3-rgw-18.0.0-6863.g38499368.el8 1.2 MB/s | 107 kB 00:00 2023-10-26T06:31:46.629 INFO:teuthology.orchestra.run.smithi135.stdout:(25/131): rbd-fuse-18.0.0-6863.g38499368.el8.x8 1.1 MB/s | 94 kB 00:00 2023-10-26T06:31:46.696 INFO:teuthology.orchestra.run.smithi135.stdout:(26/131): rbd-nbd-18.0.0-6863.g38499368.el8.x86 2.6 MB/s | 179 kB 00:00 2023-10-26T06:31:46.739 INFO:teuthology.orchestra.run.smithi192.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 8.7 MB/s | 4.9 MB 00:00 2023-10-26T06:31:46.763 INFO:teuthology.orchestra.run.smithi135.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6863.g 345 kB/s | 23 kB 00:00 2023-10-26T06:31:46.815 INFO:teuthology.orchestra.run.smithi192.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 8.1 MB/s | 4.7 MB 00:00 2023-10-26T06:31:46.831 INFO:teuthology.orchestra.run.smithi135.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6863.g3849936 1.9 MB/s | 132 kB 00:00 2023-10-26T06:31:46.865 INFO:teuthology.orchestra.run.smithi192.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-10-26T06:31:46.924 INFO:teuthology.orchestra.run.smithi192.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-10-26T06:31:46.949 INFO:teuthology.orchestra.run.smithi192.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 561 kB/s | 47 kB 00:00 2023-10-26T06:31:46.974 INFO:teuthology.orchestra.run.smithi192.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-10-26T06:31:47.000 INFO:teuthology.orchestra.run.smithi192.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-10-26T06:31:47.025 INFO:teuthology.orchestra.run.smithi192.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-26T06:31:47.050 INFO:teuthology.orchestra.run.smithi192.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 958 kB/s | 48 kB 00:00 2023-10-26T06:31:47.075 INFO:teuthology.orchestra.run.smithi192.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-26T06:31:47.101 INFO:teuthology.orchestra.run.smithi192.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-10-26T06:31:47.126 INFO:teuthology.orchestra.run.smithi192.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-26T06:31:47.151 INFO:teuthology.orchestra.run.smithi192.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-26T06:31:47.227 INFO:teuthology.orchestra.run.smithi192.stdout:(111/131): postgresql-server-10.21-2.module+el8 7.3 MB/s | 5.1 MB 00:00 2023-10-26T06:31:47.252 INFO:teuthology.orchestra.run.smithi192.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 187 kB/s | 19 kB 00:00 2023-10-26T06:31:47.264 INFO:teuthology.orchestra.run.smithi135.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6863.g38499 4.1 MB/s | 1.8 MB 00:00 2023-10-26T06:31:47.277 INFO:teuthology.orchestra.run.smithi192.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-10-26T06:31:47.311 INFO:teuthology.orchestra.run.smithi192.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.0 MB/s | 560 kB 00:00 2023-10-26T06:31:47.336 INFO:teuthology.orchestra.run.smithi192.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-10-26T06:31:47.370 INFO:teuthology.orchestra.run.smithi192.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-10-26T06:31:47.395 INFO:teuthology.orchestra.run.smithi192.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.2 MB/s | 605 kB 00:00 2023-10-26T06:31:47.429 INFO:teuthology.orchestra.run.smithi192.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-10-26T06:31:47.439 INFO:teuthology.orchestra.run.smithi135.stdout:(30/131): rbd-mirror-18.0.0-6863.g38499368.el8. 4.2 MB/s | 3.6 MB 00:00 2023-10-26T06:31:47.454 INFO:teuthology.orchestra.run.smithi192.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-10-26T06:31:47.512 INFO:teuthology.orchestra.run.smithi192.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2023-10-26T06:31:47.515 INFO:teuthology.orchestra.run.smithi135.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6863.g38 3.2 MB/s | 242 kB 00:00 2023-10-26T06:31:47.538 INFO:teuthology.orchestra.run.smithi192.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 717 kB/s | 60 kB 00:00 2023-10-26T06:31:47.565 INFO:teuthology.orchestra.run.smithi135.stdout:(32/131): ceph-mgr-rook-18.0.0-6863.g38499368.e 954 kB/s | 48 kB 00:00 2023-10-26T06:31:47.571 INFO:teuthology.orchestra.run.smithi192.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.6 MB/s | 98 kB 00:00 2023-10-26T06:31:47.597 INFO:teuthology.orchestra.run.smithi192.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 984 kB/s | 58 kB 00:00 2023-10-26T06:31:47.622 INFO:teuthology.orchestra.run.smithi192.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-10-26T06:31:47.624 INFO:teuthology.orchestra.run.smithi135.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6863.g3 258 kB/s | 15 kB 00:00 2023-10-26T06:31:47.656 INFO:teuthology.orchestra.run.smithi192.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-10-26T06:31:47.681 INFO:teuthology.orchestra.run.smithi192.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-10-26T06:31:47.706 INFO:teuthology.orchestra.run.smithi192.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-26T06:31:47.740 INFO:teuthology.orchestra.run.smithi135.stdout:(34/131): ceph-volume-18.0.0-6863.g38499368.el8 2.2 MB/s | 261 kB 00:00 2023-10-26T06:31:47.756 INFO:teuthology.orchestra.run.smithi192.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-10-26T06:31:47.823 INFO:teuthology.orchestra.run.smithi192.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-10-26T06:31:47.824 INFO:teuthology.orchestra.run.smithi135.stdout:(35/131): cephadm-18.0.0-6863.g38499368.el8.noa 2.2 MB/s | 191 kB 00:00 2023-10-26T06:31:47.883 INFO:teuthology.orchestra.run.smithi135.stdout:(36/131): cephfs-top-18.0.0-6863.g38499368.el8. 309 kB/s | 18 kB 00:00 2023-10-26T06:31:48.159 INFO:teuthology.orchestra.run.smithi135.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-10-26T06:31:48.218 INFO:teuthology.orchestra.run.smithi135.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-10-26T06:31:48.276 INFO:teuthology.orchestra.run.smithi135.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-10-26T06:31:48.315 INFO:teuthology.orchestra.run.smithi192.stdout:(130/131): librbd1-18.0.0-6863.g38499368.el8.x8 6.1 MB/s | 3.7 MB 00:00 2023-10-26T06:31:48.369 INFO:teuthology.orchestra.run.smithi135.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 5.1 MB/s | 476 kB 00:00 2023-10-26T06:31:48.427 INFO:teuthology.orchestra.run.smithi135.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 750 kB/s | 44 kB 00:00 2023-10-26T06:31:48.440 INFO:teuthology.orchestra.run.smithi192.stdout:(131/131): librados2-18.0.0-6863.g38499368.el8. 4.9 MB/s | 3.7 MB 00:00 2023-10-26T06:31:48.443 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-10-26T06:31:48.444 INFO:teuthology.orchestra.run.smithi192.stdout:Total 16 MB/s | 209 MB 00:13 2023-10-26T06:31:48.537 INFO:teuthology.orchestra.run.smithi135.stdout:(42/131): ceph-mgr-diskprediction-local-18.0.0- 5.8 MB/s | 7.4 MB 00:01 2023-10-26T06:31:48.562 INFO:teuthology.orchestra.run.smithi135.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-10-26T06:31:48.622 INFO:teuthology.orchestra.run.smithi135.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 278 kB/s | 16 kB 00:00 2023-10-26T06:31:48.679 INFO:teuthology.orchestra.run.smithi135.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-10-26T06:31:48.707 INFO:teuthology.orchestra.run.smithi135.stdout:(46/131): python3-cachetools-3.1.1-4.el8.noarch 197 kB/s | 33 kB 00:00 2023-10-26T06:31:48.780 INFO:teuthology.orchestra.run.smithi135.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2023-10-26T06:31:48.806 INFO:teuthology.orchestra.run.smithi135.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 808 kB/s | 81 kB 00:00 2023-10-26T06:31:48.822 INFO:teuthology.orchestra.run.smithi135.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 264 kB/s | 11 kB 00:00 2023-10-26T06:31:48.888 INFO:teuthology.orchestra.run.smithi135.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 214 kB/s | 18 kB 00:00 2023-10-26T06:31:48.940 INFO:teuthology.orchestra.run.smithi135.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 943 kB/s | 49 kB 00:00 2023-10-26T06:31:48.999 INFO:teuthology.orchestra.run.smithi135.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 6.6 MB/s | 1.2 MB 00:00 2023-10-26T06:31:49.016 INFO:teuthology.orchestra.run.smithi135.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 779 kB/s | 59 kB 00:00 2023-10-26T06:31:49.049 INFO:teuthology.orchestra.run.smithi135.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-26T06:31:49.100 INFO:teuthology.orchestra.run.smithi135.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-10-26T06:31:49.134 INFO:teuthology.orchestra.run.smithi135.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-10-26T06:31:49.158 INFO:teuthology.orchestra.run.smithi135.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 573 kB/s | 34 kB 00:00 2023-10-26T06:31:49.192 INFO:teuthology.orchestra.run.smithi135.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-10-26T06:31:49.217 INFO:teuthology.orchestra.run.smithi135.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-10-26T06:31:49.243 INFO:teuthology.orchestra.run.smithi135.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-10-26T06:31:49.271 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-10-26T06:31:49.277 INFO:teuthology.orchestra.run.smithi135.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 407 kB/s | 24 kB 00:00 2023-10-26T06:31:49.302 INFO:teuthology.orchestra.run.smithi135.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 498 kB/s | 29 kB 00:00 2023-10-26T06:31:49.338 INFO:teuthology.orchestra.run.smithi135.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 470 kB/s | 27 kB 00:00 2023-10-26T06:31:49.362 INFO:teuthology.orchestra.run.smithi135.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2023-10-26T06:31:49.392 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-10-26T06:31:49.392 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-10-26T06:31:49.397 INFO:teuthology.orchestra.run.smithi135.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-10-26T06:31:49.424 INFO:teuthology.orchestra.run.smithi135.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-10-26T06:31:49.479 INFO:teuthology.orchestra.run.smithi135.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-26T06:31:49.504 INFO:teuthology.orchestra.run.smithi135.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 274 kB/s | 23 kB 00:00 2023-10-26T06:31:49.530 INFO:teuthology.orchestra.run.smithi135.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-26T06:31:49.680 INFO:teuthology.orchestra.run.smithi135.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 8.6 MB/s | 1.7 MB 00:00 2023-10-26T06:31:49.731 INFO:teuthology.orchestra.run.smithi135.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-10-26T06:31:49.765 INFO:teuthology.orchestra.run.smithi135.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-26T06:31:49.807 INFO:teuthology.orchestra.run.smithi135.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-10-26T06:31:49.956 INFO:teuthology.orchestra.run.smithi135.stdout:(74/131): python3-scipy-1.0.0-21.module+el8.5.0 32 MB/s | 14 MB 00:00 2023-10-26T06:31:49.982 INFO:teuthology.orchestra.run.smithi135.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2023-10-26T06:31:50.035 INFO:teuthology.orchestra.run.smithi135.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 496 kB/s | 39 kB 00:00 2023-10-26T06:31:50.074 INFO:teuthology.orchestra.run.smithi135.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2023-10-26T06:31:50.116 INFO:teuthology.orchestra.run.smithi135.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 6.5 MB/s | 538 kB 00:00 2023-10-26T06:31:50.133 INFO:teuthology.orchestra.run.smithi135.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-26T06:31:50.159 INFO:teuthology.orchestra.run.smithi135.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.3 MB/s | 187 kB 00:00 2023-10-26T06:31:50.192 INFO:teuthology.orchestra.run.smithi135.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-10-26T06:31:50.226 INFO:teuthology.orchestra.run.smithi135.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 798 kB/s | 27 kB 00:00 2023-10-26T06:31:50.269 INFO:teuthology.orchestra.run.smithi135.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-26T06:31:50.302 INFO:teuthology.orchestra.run.smithi135.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-26T06:31:50.385 INFO:teuthology.orchestra.run.smithi135.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-10-26T06:31:50.427 INFO:teuthology.orchestra.run.smithi135.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-10-26T06:31:50.470 INFO:teuthology.orchestra.run.smithi135.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-10-26T06:31:50.520 INFO:teuthology.orchestra.run.smithi135.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-10-26T06:31:50.527 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-10-26T06:31:50.532 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-10-26T06:31:50.545 INFO:teuthology.orchestra.run.smithi135.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 847 kB/s | 64 kB 00:00 2023-10-26T06:31:50.562 INFO:teuthology.orchestra.run.smithi135.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-10-26T06:31:50.613 INFO:teuthology.orchestra.run.smithi135.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 297 kB/s | 20 kB 00:00 2023-10-26T06:31:50.638 INFO:teuthology.orchestra.run.smithi135.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-10-26T06:31:50.672 INFO:teuthology.orchestra.run.smithi135.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-10-26T06:31:50.714 INFO:teuthology.orchestra.run.smithi135.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-26T06:31:50.806 INFO:teuthology.orchestra.run.smithi135.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-10-26T06:31:50.840 INFO:teuthology.orchestra.run.smithi135.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 5.9 MB/s | 202 kB 00:00 2023-10-26T06:31:50.924 INFO:teuthology.orchestra.run.smithi135.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-10-26T06:31:50.949 INFO:teuthology.orchestra.run.smithi135.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-10-26T06:31:51.008 INFO:teuthology.orchestra.run.smithi135.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 13 MB/s | 808 kB 00:00 2023-10-26T06:31:51.166 INFO:teuthology.orchestra.run.smithi135.stdout:(100/131): postgresql-server-10.21-2.module+el8 21 MB/s | 5.1 MB 00:00 2023-10-26T06:31:51.342 INFO:teuthology.orchestra.run.smithi135.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 4.5 MB/s | 1.5 MB 00:00 2023-10-26T06:31:51.367 INFO:teuthology.orchestra.run.smithi135.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 233 kB/s | 47 kB 00:00 2023-10-26T06:31:51.392 INFO:teuthology.orchestra.run.smithi135.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-10-26T06:31:51.418 INFO:teuthology.orchestra.run.smithi135.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-10-26T06:31:51.443 INFO:teuthology.orchestra.run.smithi135.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-26T06:31:51.477 INFO:teuthology.orchestra.run.smithi135.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 820 kB/s | 48 kB 00:00 2023-10-26T06:31:51.502 INFO:teuthology.orchestra.run.smithi135.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2023-10-26T06:31:51.528 INFO:teuthology.orchestra.run.smithi135.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 846 kB/s | 43 kB 00:00 2023-10-26T06:31:51.544 INFO:teuthology.orchestra.run.smithi135.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-10-26T06:31:51.578 INFO:teuthology.orchestra.run.smithi135.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-26T06:31:52.021 INFO:teuthology.orchestra.run.smithi135.stdout:(111/131): ceph-test-18.0.0-6863.g38499368.el8. 5.9 MB/s | 49 MB 00:08 2023-10-26T06:31:52.046 INFO:teuthology.orchestra.run.smithi135.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 40 kB/s | 19 kB 00:00 2023-10-26T06:31:52.080 INFO:teuthology.orchestra.run.smithi135.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 1.0 MB/s | 560 kB 00:00 2023-10-26T06:31:52.105 INFO:teuthology.orchestra.run.smithi135.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2023-10-26T06:31:52.131 INFO:teuthology.orchestra.run.smithi135.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 354 kB/s | 18 kB 00:00 2023-10-26T06:31:52.164 INFO:teuthology.orchestra.run.smithi135.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.9 MB/s | 559 kB 00:00 2023-10-26T06:31:52.198 INFO:teuthology.orchestra.run.smithi135.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-10-26T06:31:52.223 INFO:teuthology.orchestra.run.smithi135.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-10-26T06:31:52.249 INFO:teuthology.orchestra.run.smithi135.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-10-26T06:31:52.274 INFO:teuthology.orchestra.run.smithi135.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-26T06:31:52.291 INFO:teuthology.orchestra.run.smithi135.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-10-26T06:31:52.316 INFO:teuthology.orchestra.run.smithi135.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-10-26T06:31:52.342 INFO:teuthology.orchestra.run.smithi135.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-26T06:31:52.358 INFO:teuthology.orchestra.run.smithi135.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 920 kB/s | 39 kB 00:00 2023-10-26T06:31:52.392 INFO:teuthology.orchestra.run.smithi135.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 12 MB/s | 644 kB 00:00 2023-10-26T06:31:52.426 INFO:teuthology.orchestra.run.smithi135.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-10-26T06:31:52.451 INFO:teuthology.orchestra.run.smithi135.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2023-10-26T06:31:52.531 INFO:teuthology.orchestra.run.smithi135.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.4 MB/s | 2.3 MB 00:00 2023-10-26T06:31:52.602 INFO:teuthology.orchestra.run.smithi135.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 11 MB/s | 798 kB 00:00 2023-10-26T06:31:53.036 INFO:teuthology.orchestra.run.smithi135.stdout:(130/131): librados2-18.0.0-6863.g38499368.el8. 6.1 MB/s | 3.7 MB 00:00 2023-10-26T06:31:53.161 INFO:teuthology.orchestra.run.smithi135.stdout:(131/131): librbd1-18.0.0-6863.g38499368.el8.x8 5.2 MB/s | 3.7 MB 00:00 2023-10-26T06:31:53.164 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-10-26T06:31:53.165 INFO:teuthology.orchestra.run.smithi135.stdout:Total 13 MB/s | 209 MB 00:15 2023-10-26T06:31:53.986 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-10-26T06:31:54.106 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-10-26T06:31:54.107 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-10-26T06:31:55.234 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-10-26T06:31:55.237 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-10-26T06:31:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-10-26T06:31:55.772 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-26T06:31:55.932 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-26T06:31:56.181 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-26T06:31:56.358 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-26T06:31:56.471 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-26T06:31:56.724 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-26T06:31:57.119 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-26T06:31:57.332 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:31:57.743 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:31:57.924 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:31:58.037 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:31:58.224 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:31:58.304 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:31:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 9/134 2023-10-26T06:31:58.735 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-26T06:31:58.952 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-26T06:31:59.326 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-26T06:31:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-26T06:31:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 14/134 2023-10-26T06:31:59.853 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 14/134 2023-10-26T06:32:00.007 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 15/134 2023-10-26T06:32:00.097 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 15/134 2023-10-26T06:32:00.215 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-26T06:32:00.356 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-26T06:32:00.380 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-10-26T06:32:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-26T06:32:00.546 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-26T06:32:00.733 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-26T06:32:00.929 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-26T06:32:01.081 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-26T06:32:01.123 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-26T06:32:01.188 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-26T06:32:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-26T06:32:01.464 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-26T06:32:01.859 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-26T06:32:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-26T06:32:02.097 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-26T06:32:02.508 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:32:02.831 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-26T06:32:02.980 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:32:03.094 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:32:03.130 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-26T06:32:03.322 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:32:03.358 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-26T06:32:03.419 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:32:03.594 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-26T06:32:03.605 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 9/134 2023-10-26T06:32:03.795 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-26T06:32:03.866 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-26T06:32:04.003 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-26T06:32:04.066 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-26T06:32:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-26T06:32:04.413 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:18.0.0-6863.g38499368.el8. 26/134 2023-10-26T06:32:04.437 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-26T06:32:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 27/134 2023-10-26T06:32:04.662 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-26T06:32:04.797 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 14/134 2023-10-26T06:32:04.888 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 14/134 2023-10-26T06:32:05.038 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 15/134 2023-10-26T06:32:05.112 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 15/134 2023-10-26T06:32:05.230 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-26T06:32:05.370 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-26T06:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_6 28/134 2023-10-26T06:32:05.563 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-26T06:32:05.804 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-26T06:32:06.074 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-26T06:32:06.183 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-26T06:32:06.346 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-26T06:32:06.347 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-26T06:32:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-26T06:32:06.941 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-26T06:32:07.121 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-26T06:32:07.814 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-26T06:32:08.120 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-26T06:32:08.308 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-26T06:32:08.331 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-26T06:32:08.558 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-26T06:32:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-26T06:32:08.752 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-26T06:32:08.850 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-26T06:32:08.952 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-26T06:32:09.040 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-26T06:32:09.198 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-26T06:32:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 36/134 2023-10-26T06:32:09.369 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:18.0.0-6863.g38499368.el8. 26/134 2023-10-26T06:32:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:18.0.0-6863.g38499368.el8.x86_6 37/134 2023-10-26T06:32:09.507 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 27/134 2023-10-26T06:32:09.587 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-26T06:32:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-26T06:32:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-26T06:32:10.125 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-26T06:32:10.352 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-26T06:32:10.413 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_6 28/134 2023-10-26T06:32:10.560 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-26T06:32:10.717 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-26T06:32:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-26T06:32:11.007 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-26T06:32:11.242 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-26T06:32:11.279 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-26T06:32:11.440 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-26T06:32:11.496 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-26T06:32:11.666 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-26T06:32:11.962 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-26T06:32:12.288 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-26T06:32:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-26T06:32:12.705 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-26T06:32:12.935 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-26T06:32:13.126 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-26T06:32:13.274 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-26T06:32:13.297 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-26T06:32:13.588 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-26T06:32:13.648 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-26T06:32:13.816 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-26T06:32:13.909 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-26T06:32:14.006 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-26T06:32:14.168 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-26T06:32:14.192 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 36/134 2023-10-26T06:32:14.382 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:18.0.0-6863.g38499368.el8.x86_6 37/134 2023-10-26T06:32:14.436 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-26T06:32:14.527 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-26T06:32:14.603 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-26T06:32:14.706 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-26T06:32:14.768 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-26T06:32:14.906 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-26T06:32:14.929 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-26T06:32:14.998 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-26T06:32:15.034 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-26T06:32:15.242 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-26T06:32:15.270 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-26T06:32:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-26T06:32:15.498 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-26T06:32:15.638 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-26T06:32:15.718 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-26T06:32:15.861 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-26T06:32:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-26T06:32:16.114 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-26T06:32:16.148 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-26T06:32:16.304 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-26T06:32:16.355 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-26T06:32:16.499 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-26T06:32:16.556 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-26T06:32:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-26T06:32:16.841 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-26T06:32:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-26T06:32:17.100 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-26T06:32:17.144 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-26T06:32:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-26T06:32:17.344 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-26T06:32:17.480 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-26T06:32:17.529 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-26T06:32:17.676 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-26T06:32:17.724 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-26T06:32:17.885 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-26T06:32:17.915 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-26T06:32:18.073 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-26T06:32:18.091 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-26T06:32:18.244 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-26T06:32:18.370 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-26T06:32:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-26T06:32:18.616 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-26T06:32:18.849 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-26T06:32:19.085 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-26T06:32:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-26T06:32:19.243 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-26T06:32:19.310 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-26T06:32:19.388 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-26T06:32:19.558 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-26T06:32:19.586 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-26T06:32:19.657 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-26T06:32:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-26T06:32:19.901 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-26T06:32:19.953 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-26T06:32:20.089 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-26T06:32:20.188 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-26T06:32:20.261 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-26T06:32:20.346 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-26T06:32:20.565 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-26T06:32:20.762 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-26T06:32:20.902 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-26T06:32:21.073 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-26T06:32:21.297 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-26T06:32:21.372 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-26T06:32:21.576 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-26T06:32:21.591 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-26T06:32:21.715 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-26T06:32:21.853 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-26T06:32:21.854 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-26T06:32:22.062 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-26T06:32:22.178 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-26T06:32:22.264 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-26T06:32:22.439 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-26T06:32:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-26T06:32:22.639 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-26T06:32:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-26T06:32:22.775 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-26T06:32:22.992 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-26T06:32:23.248 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-26T06:32:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-26T06:32:23.508 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-26T06:32:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:18.0.0-6863.g38499368.el8.x86_64 92/134 2023-10-26T06:32:23.729 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:18.0.0-6863.g38499368.el8.x86_64 92/134 2023-10-26T06:32:23.799 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-26T06:32:23.896 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 93/134 2023-10-26T06:32:24.001 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-26T06:32:24.168 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-26T06:32:24.428 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-26T06:32:25.518 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-26T06:32:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-26T06:32:26.060 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-26T06:32:26.362 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-26T06:32:26.694 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-26T06:32:26.896 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-26T06:32:27.615 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:32:27.701 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-26T06:32:27.876 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:18.0.0-6863.g38499368.el8.x86_64 92/134 2023-10-26T06:32:27.878 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:32:27.954 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:18.0.0-6863.g38499368.el8.x86_64 92/134 2023-10-26T06:32:28.119 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 93/134 2023-10-26T06:32:28.469 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:32:28.740 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 95/134 2023-10-26T06:32:28.809 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 95/134 2023-10-26T06:32:28.986 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 96/134 2023-10-26T06:32:31.844 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:32:32.076 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:32:32.660 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:32:32.914 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 95/134 2023-10-26T06:32:32.994 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 95/134 2023-10-26T06:32:33.159 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 96/134 2023-10-26T06:32:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 96/134 2023-10-26T06:32:47.898 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2023-10-26T06:32:47.898 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2023-10-26T06:32:47.898 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2023-10-26T06:32:47.898 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /run 2023-10-26T06:32:47.898 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2023-10-26T06:32:47.899 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2023-10-26T06:32:47.899 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2023-10-26T06:32:47.899 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2023-10-26T06:32:47.899 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2023-10-26T06:32:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 97/134 2023-10-26T06:32:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 97/134 2023-10-26T06:32:50.028 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 98/134 2023-10-26T06:32:50.249 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 98/134 2023-10-26T06:32:50.425 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 99/134 2023-10-26T06:32:50.891 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 99/134 2023-10-26T06:32:51.058 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 100/134 2023-10-26T06:32:51.075 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 100/134 2023-10-26T06:32:52.191 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 96/134 2023-10-26T06:32:52.192 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2023-10-26T06:32:52.192 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2023-10-26T06:32:52.192 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /dev 2023-10-26T06:32:52.192 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /run 2023-10-26T06:32:52.192 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2023-10-26T06:32:52.193 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2023-10-26T06:32:52.193 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2023-10-26T06:32:52.193 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2023-10-26T06:32:52.193 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /dev 2023-10-26T06:32:52.193 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:32:52.548 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 97/134 2023-10-26T06:32:52.788 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:18.0.0-6863.g38499368.el8.noarch 101/134 2023-10-26T06:32:52.980 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:18.0.0-6863.g38499368.el8.noarch 101/134 2023-10-26T06:32:53.090 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 102/134 2023-10-26T06:32:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 103/134 2023-10-26T06:32:53.499 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 104/134 2023-10-26T06:32:53.721 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 104/134 2023-10-26T06:32:54.045 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 105/134 2023-10-26T06:32:54.149 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 97/134 2023-10-26T06:32:54.335 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 98/134 2023-10-26T06:32:54.554 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 98/134 2023-10-26T06:32:54.756 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 99/134 2023-10-26T06:32:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 105/134 2023-10-26T06:32:55.207 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 106/134 2023-10-26T06:32:55.221 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 99/134 2023-10-26T06:32:55.297 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 106/134 2023-10-26T06:32:55.389 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 100/134 2023-10-26T06:32:55.406 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 100/134 2023-10-26T06:32:55.654 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 107/134 2023-10-26T06:32:55.928 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 108/134 2023-10-26T06:32:55.964 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 108/134 2023-10-26T06:32:56.132 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 109/134 2023-10-26T06:32:56.143 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 109/134 2023-10-26T06:32:57.098 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.0.0-6863.g38499368.el8.noarch 101/134 2023-10-26T06:32:57.275 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:18.0.0-6863.g38499368.el8.noarch 101/134 2023-10-26T06:32:57.387 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 102/134 2023-10-26T06:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:18.0.0-6863.g38499368.el8.x86_64 110/134 2023-10-26T06:32:57.631 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 103/134 2023-10-26T06:32:57.636 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 111/134 2023-10-26T06:32:57.670 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 111/134 2023-10-26T06:32:57.815 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6863.g38499 112/134 2023-10-26T06:32:57.823 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 104/134 2023-10-26T06:32:57.857 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6863.g38499 112/134 2023-10-26T06:32:58.008 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 113/134 2023-10-26T06:32:58.043 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 104/134 2023-10-26T06:32:58.335 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 105/134 2023-10-26T06:32:58.367 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 113/134 2023-10-26T06:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 114/134 2023-10-26T06:32:59.305 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 105/134 2023-10-26T06:32:59.522 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 106/134 2023-10-26T06:32:59.612 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 106/134 2023-10-26T06:32:59.960 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 107/134 2023-10-26T06:33:00.206 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 108/134 2023-10-26T06:33:00.242 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 108/134 2023-10-26T06:33:00.405 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 109/134 2023-10-26T06:33:00.416 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 109/134 2023-10-26T06:33:01.745 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:18.0.0-6863.g38499368.el8.x86_64 110/134 2023-10-26T06:33:01.962 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 111/134 2023-10-26T06:33:01.996 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 111/134 2023-10-26T06:33:02.122 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6863.g38499 112/134 2023-10-26T06:33:02.163 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6863.g38499 112/134 2023-10-26T06:33:02.305 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 113/134 2023-10-26T06:33:02.665 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 113/134 2023-10-26T06:33:02.822 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 114/134 2023-10-26T06:33:03.204 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 114/134 2023-10-26T06:33:03.488 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 115/134 2023-10-26T06:33:03.664 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-26T06:33:03.862 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-26T06:33:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-26T06:33:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-26T06:33:05.233 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 119/134 2023-10-26T06:33:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-26T06:33:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_ 121/134 2023-10-26T06:33:05.798 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-26T06:33:07.481 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 114/134 2023-10-26T06:33:07.816 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 115/134 2023-10-26T06:33:07.971 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-26T06:33:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:33:08.167 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-26T06:33:08.272 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:33:08.289 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:33:08.447 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 124/134 2023-10-26T06:33:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 125/134 2023-10-26T06:33:09.074 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-26T06:33:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 126/134 2023-10-26T06:33:09.382 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-26T06:33:09.391 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-26T06:33:09.522 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-26T06:33:09.530 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-26T06:33:09.531 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 119/134 2023-10-26T06:33:09.691 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-26T06:33:09.704 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-26T06:33:09.849 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_ 121/134 2023-10-26T06:33:09.905 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-26T06:33:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-26T06:33:10.013 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-26T06:33:10.230 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-26T06:33:10.243 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-26T06:33:10.243 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:33:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:33:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-26T06:33:10.665 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-26T06:33:10.854 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:33:11.371 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:33:11.372 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:18.0.0-6863.g38499368.el8.x86_64 1/134 2023-10-26T06:33:11.372 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 2/134 2023-10-26T06:33:11.372 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 3/134 2023-10-26T06:33:11.372 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 4/134 2023-10-26T06:33:11.372 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6863.g38499 5/134 2023-10-26T06:33:11.373 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:33:11.373 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:33:11.373 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:33:11.373 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 9/134 2023-10-26T06:33:11.374 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 10/134 2023-10-26T06:33:11.374 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 11/134 2023-10-26T06:33:11.374 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 12/134 2023-10-26T06:33:11.374 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 13/134 2023-10-26T06:33:11.374 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_ 14/134 2023-10-26T06:33:11.375 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 15/134 2023-10-26T06:33:11.375 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 16/134 2023-10-26T06:33:11.375 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:18.0.0-6863.g38499368.el8.x86_6 17/134 2023-10-26T06:33:11.375 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 18/134 2023-10-26T06:33:11.376 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:18.0.0-6863.g38499368.el8.x86_64 19/134 2023-10-26T06:33:11.376 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 20/134 2023-10-26T06:33:11.376 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:18.0.0-6863.g38499368.el8. 21/134 2023-10-26T06:33:11.376 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_6 22/134 2023-10-26T06:33:11.376 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 23/134 2023-10-26T06:33:11.377 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 24/134 2023-10-26T06:33:11.377 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 25/134 2023-10-26T06:33:11.377 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 26/134 2023-10-26T06:33:11.377 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 27/134 2023-10-26T06:33:11.378 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 28/134 2023-10-26T06:33:11.378 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 29/134 2023-10-26T06:33:11.392 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 30/134 2023-10-26T06:33:11.392 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 31/134 2023-10-26T06:33:11.393 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 32/134 2023-10-26T06:33:11.393 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 33/134 2023-10-26T06:33:11.393 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 34/134 2023-10-26T06:33:11.393 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 35/134 2023-10-26T06:33:11.393 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 36/134 2023-10-26T06:33:11.394 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:18.0.0-6863.g38499368.el8.noarch 37/134 2023-10-26T06:33:11.394 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 38/134 2023-10-26T06:33:11.394 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-26T06:33:11.394 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-26T06:33:11.394 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-26T06:33:11.395 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-26T06:33:11.395 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-26T06:33:11.395 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-26T06:33:11.395 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-26T06:33:11.395 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-26T06:33:11.395 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-26T06:33:11.396 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-26T06:33:11.396 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-26T06:33:11.396 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-26T06:33:11.396 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-26T06:33:11.396 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-26T06:33:11.397 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-26T06:33:11.397 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-26T06:33:11.397 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-26T06:33:11.397 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-26T06:33:11.397 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-26T06:33:11.397 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-26T06:33:11.398 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-26T06:33:11.398 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-26T06:33:11.398 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-26T06:33:11.398 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-26T06:33:11.398 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-26T06:33:11.399 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-26T06:33:11.399 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-26T06:33:11.399 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-26T06:33:11.399 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-26T06:33:11.399 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-26T06:33:11.400 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-26T06:33:11.400 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-26T06:33:11.400 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-26T06:33:11.400 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-26T06:33:11.400 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-26T06:33:11.400 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-26T06:33:11.401 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-26T06:33:11.401 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-26T06:33:11.401 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-26T06:33:11.401 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-26T06:33:11.401 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-26T06:33:11.402 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-26T06:33:11.402 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-26T06:33:11.402 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-26T06:33:11.402 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-26T06:33:11.402 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-26T06:33:11.402 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-26T06:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-26T06:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-26T06:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-26T06:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-26T06:33:11.403 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-26T06:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-26T06:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-26T06:33:11.404 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-26T06:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-26T06:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-26T06:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-26T06:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-26T06:33:11.405 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-26T06:33:11.406 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-26T06:33:11.406 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-26T06:33:11.406 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-26T06:33:11.406 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-26T06:33:11.407 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-26T06:33:11.407 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-26T06:33:11.407 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-26T06:33:11.407 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-26T06:33:11.407 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-26T06:33:11.408 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-26T06:33:11.408 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-26T06:33:11.408 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-26T06:33:11.408 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-26T06:33:11.409 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-26T06:33:11.409 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-26T06:33:11.409 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-26T06:33:11.409 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-26T06:33:11.410 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-26T06:33:11.410 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-26T06:33:11.410 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-26T06:33:11.410 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-26T06:33:11.411 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-26T06:33:11.411 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-26T06:33:11.411 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-26T06:33:11.412 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-26T06:33:11.412 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-26T06:33:11.412 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-26T06:33:11.412 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-26T06:33:11.413 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-26T06:33:11.413 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-26T06:33:11.413 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:18.0.0-6863.g38499368.el8.x86_64 129/134 2023-10-26T06:33:11.414 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-26T06:33:11.414 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:18.0.0-6863.g38499368.el8.x86_64 131/134 2023-10-26T06:33:11.414 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:33:11.415 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-26T06:33:12.168 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:33:12.437 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:33:12.455 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:33:12.604 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 124/134 2023-10-26T06:33:12.911 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 125/134 2023-10-26T06:33:13.188 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 126/134 2023-10-26T06:33:13.555 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-26T06:33:13.695 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-26T06:33:13.703 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-26T06:33:13.880 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-26T06:33:13.895 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:33:13.895 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:50 ago on Thu 26 Oct 2023 06:31:23 AM UTC. 2023-10-26T06:33:14.094 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-26T06:33:14.175 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-26T06:33:14.370 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-26T06:33:14.382 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-26T06:33:14.382 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:33:14.606 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:33:14.606 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-26T06:33:14.851 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-26T06:33:15.044 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:33:15.565 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:33:15.566 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:18.0.0-6863.g38499368.el8.x86_64 1/134 2023-10-26T06:33:15.566 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 2/134 2023-10-26T06:33:15.566 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 3/134 2023-10-26T06:33:15.567 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 4/134 2023-10-26T06:33:15.567 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6863.g38499 5/134 2023-10-26T06:33:15.567 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:33:15.567 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:33:15.567 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:33:15.568 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 9/134 2023-10-26T06:33:15.568 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 10/134 2023-10-26T06:33:15.568 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 11/134 2023-10-26T06:33:15.568 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 12/134 2023-10-26T06:33:15.568 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 13/134 2023-10-26T06:33:15.569 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_ 14/134 2023-10-26T06:33:15.569 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 15/134 2023-10-26T06:33:15.569 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 16/134 2023-10-26T06:33:15.569 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:18.0.0-6863.g38499368.el8.x86_6 17/134 2023-10-26T06:33:15.569 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 18/134 2023-10-26T06:33:15.570 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:18.0.0-6863.g38499368.el8.x86_64 19/134 2023-10-26T06:33:15.570 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 20/134 2023-10-26T06:33:15.570 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:18.0.0-6863.g38499368.el8. 21/134 2023-10-26T06:33:15.570 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_6 22/134 2023-10-26T06:33:15.571 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 23/134 2023-10-26T06:33:15.571 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 24/134 2023-10-26T06:33:15.571 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 25/134 2023-10-26T06:33:15.571 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 26/134 2023-10-26T06:33:15.571 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 27/134 2023-10-26T06:33:15.572 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 28/134 2023-10-26T06:33:15.572 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 29/134 2023-10-26T06:33:15.581 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 30/134 2023-10-26T06:33:15.582 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 31/134 2023-10-26T06:33:15.582 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 32/134 2023-10-26T06:33:15.582 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 33/134 2023-10-26T06:33:15.582 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 34/134 2023-10-26T06:33:15.582 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 35/134 2023-10-26T06:33:15.583 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 36/134 2023-10-26T06:33:15.583 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:18.0.0-6863.g38499368.el8.noarch 37/134 2023-10-26T06:33:15.583 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 38/134 2023-10-26T06:33:15.583 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-26T06:33:15.583 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-26T06:33:15.584 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-26T06:33:15.584 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-26T06:33:15.584 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-26T06:33:15.584 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-26T06:33:15.584 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-26T06:33:15.585 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-26T06:33:15.585 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-26T06:33:15.585 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-26T06:33:15.585 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-26T06:33:15.585 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-26T06:33:15.586 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-26T06:33:15.586 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-26T06:33:15.586 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-26T06:33:15.586 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-26T06:33:15.586 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-26T06:33:15.587 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-26T06:33:15.587 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-26T06:33:15.587 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-26T06:33:15.587 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-26T06:33:15.587 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-26T06:33:15.587 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-26T06:33:15.588 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-26T06:33:15.588 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-26T06:33:15.588 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-26T06:33:15.588 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-26T06:33:15.588 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-26T06:33:15.589 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-26T06:33:15.589 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-26T06:33:15.589 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-26T06:33:15.589 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-26T06:33:15.589 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-26T06:33:15.589 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-26T06:33:15.590 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-26T06:33:15.590 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-26T06:33:15.590 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-26T06:33:15.590 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-26T06:33:15.590 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-26T06:33:15.591 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-26T06:33:15.591 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-26T06:33:15.591 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-26T06:33:15.591 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-26T06:33:15.591 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-26T06:33:15.592 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-26T06:33:15.592 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-26T06:33:15.592 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-26T06:33:15.592 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-26T06:33:15.592 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-26T06:33:15.593 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-26T06:33:15.593 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-26T06:33:15.593 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-26T06:33:15.593 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-26T06:33:15.593 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-26T06:33:15.593 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-26T06:33:15.594 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-26T06:33:15.594 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-26T06:33:15.594 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-26T06:33:15.594 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-26T06:33:15.594 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-26T06:33:15.595 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-26T06:33:15.595 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-26T06:33:15.595 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-26T06:33:15.595 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-26T06:33:15.596 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-26T06:33:15.596 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-26T06:33:15.596 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-26T06:33:15.596 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-26T06:33:15.596 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-26T06:33:15.596 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-26T06:33:15.597 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-26T06:33:15.597 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-26T06:33:15.597 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-26T06:33:15.597 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-26T06:33:15.597 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-26T06:33:15.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-26T06:33:15.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-26T06:33:15.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-26T06:33:15.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-26T06:33:15.598 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-26T06:33:15.599 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-26T06:33:15.599 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-26T06:33:15.599 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-26T06:33:15.599 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-26T06:33:15.599 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-26T06:33:15.599 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-26T06:33:15.600 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-26T06:33:15.600 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-26T06:33:15.600 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-26T06:33:15.600 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-26T06:33:15.600 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:18.0.0-6863.g38499368.el8.x86_64 129/134 2023-10-26T06:33:15.601 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-26T06:33:15.601 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:18.0.0-6863.g38499368.el8.x86_64 131/134 2023-10-26T06:33:15.601 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:33:15.601 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-26T06:33:16.721 INFO:teuthology.orchestra.run.smithi192.stdout:Installed products updated. 2023-10-26T06:33:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:33:16.935 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2023-10-26T06:33:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-26T06:33:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.936 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-10-26T06:33:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-26T06:33:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.1-7.el8.noarch 2023-10-26T06:33:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.0.4-10.el8.x86_64 2023-10-26T06:33:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-26T06:33:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.1-9.el8.x86_64 2023-10-26T06:33:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-26T06:33:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-26T06:33:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-3.el8.x86_64 2023-10-26T06:33:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-26T06:33:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-26T06:33:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-26T06:33:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-26T06:33:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-26T06:33:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-26T06:33:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: libpq-13.5-1.el8.x86_64 2023-10-26T06:33:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-26T06:33:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-26T06:33:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-26T06:33:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-26T06:33:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-26T06:33:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-26T06:33:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-26T06:33:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-26T06:33:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-26T06:33:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-26T06:33:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-26T06:33:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-26T06:33:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-26T06:33:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-26T06:33:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-26T06:33:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-26T06:33:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-26T06:33:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:33:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:33:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:33:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-26T06:33:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-26T06:33:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-26T06:33:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-26T06:33:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-26T06:33:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-26T06:33:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-26T06:33:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-26T06:33:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-26T06:33:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-26T06:33:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-26T06:33:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-26T06:33:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-26T06:33:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-26T06:33:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-26T06:33:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-26T06:33:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-26T06:33:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-26T06:33:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-26T06:33:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-26T06:33:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-26T06:33:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-26T06:33:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-26T06:33:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-26T06:33:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-26T06:33:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-26T06:33:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-26T06:33:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-2.6-1.el8.noarch 2023-10-26T06:33:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-26T06:33:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-26T06:33:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-26T06:33:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-26T06:33:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-26T06:33:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-26T06:33:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-26T06:33:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-26T06:33:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-26T06:33:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-26T06:33:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-26T06:33:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-26T06:33:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-26T06:33:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-26T06:33:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-26T06:33:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-26T06:33:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-26T06:33:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-26T06:33:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-26T06:33:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-26T06:33:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-26T06:33:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-26T06:33:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-26T06:33:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-26T06:33:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-26T06:33:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-26T06:33:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-26T06:33:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:33:16.962 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-10-26T06:33:17.142 DEBUG:teuthology.parallel:result is None 2023-10-26T06:33:18.177 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:33:18.177 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:53 ago on Thu 26 Oct 2023 06:31:25 AM UTC. 2023-10-26T06:33:19.584 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 125 kB/s | 16 MB 02:10 2023-10-26T06:33:20.605 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-10-26T06:33:20.824 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:33:20.824 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2023-10-26T06:33:20.824 INFO:teuthology.orchestra.run.smithi135.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-26T06:33:20.825 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.825 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.825 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-10-26T06:33:20.825 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-26T06:33:20.825 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.1-7.el8.noarch 2023-10-26T06:33:20.826 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.0.4-10.el8.x86_64 2023-10-26T06:33:20.826 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.826 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.826 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.826 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.827 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.827 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.827 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.827 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.827 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.828 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.828 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.828 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.828 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.828 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.829 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.829 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.829 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.829 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.829 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.829 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.830 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.830 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.830 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:33:20.830 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-26T06:33:20.830 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.1-9.el8.x86_64 2023-10-26T06:33:20.831 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-26T06:33:20.831 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-26T06:33:20.831 INFO:teuthology.orchestra.run.smithi135.stdout: jq-1.6-3.el8.x86_64 2023-10-26T06:33:20.831 INFO:teuthology.orchestra.run.smithi135.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-26T06:33:20.831 INFO:teuthology.orchestra.run.smithi135.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-26T06:33:20.832 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-26T06:33:20.832 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.832 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.832 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.832 INFO:teuthology.orchestra.run.smithi135.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-26T06:33:20.832 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-26T06:33:20.833 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-26T06:33:20.833 INFO:teuthology.orchestra.run.smithi135.stdout: libpq-13.5-1.el8.x86_64 2023-10-26T06:33:20.833 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-26T06:33:20.833 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-26T06:33:20.833 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.834 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.834 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-26T06:33:20.834 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.834 INFO:teuthology.orchestra.run.smithi135.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-26T06:33:20.834 INFO:teuthology.orchestra.run.smithi135.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-26T06:33:20.835 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-26T06:33:20.835 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-26T06:33:20.835 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-26T06:33:20.835 INFO:teuthology.orchestra.run.smithi135.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-26T06:33:20.835 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-26T06:33:20.836 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-26T06:33:20.836 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-26T06:33:20.836 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-26T06:33:20.836 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-26T06:33:20.836 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-26T06:33:20.837 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-26T06:33:20.837 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-26T06:33:20.837 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:33:20.837 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:33:20.837 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:33:20.838 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-26T06:33:20.838 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-26T06:33:20.838 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-26T06:33:20.838 INFO:teuthology.orchestra.run.smithi135.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-26T06:33:20.838 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-26T06:33:20.838 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.839 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.839 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.839 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-26T06:33:20.839 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-26T06:33:20.839 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-26T06:33:20.840 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-26T06:33:20.840 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-26T06:33:20.840 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-26T06:33:20.840 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-26T06:33:20.840 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-26T06:33:20.841 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-26T06:33:20.841 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-26T06:33:20.841 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-26T06:33:20.841 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-26T06:33:20.841 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-26T06:33:20.842 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-26T06:33:20.842 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-26T06:33:20.842 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-26T06:33:20.842 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-26T06:33:20.843 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-26T06:33:20.843 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-26T06:33:20.843 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-26T06:33:20.843 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-26T06:33:20.843 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-26T06:33:20.843 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-2.6-1.el8.noarch 2023-10-26T06:33:20.844 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-26T06:33:20.844 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-26T06:33:20.844 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-26T06:33:20.844 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-26T06:33:20.844 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-26T06:33:20.845 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-26T06:33:20.845 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.845 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.845 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-26T06:33:20.845 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-26T06:33:20.846 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.846 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-26T06:33:20.846 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-26T06:33:20.846 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-26T06:33:20.846 INFO:teuthology.orchestra.run.smithi135.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-26T06:33:20.847 INFO:teuthology.orchestra.run.smithi135.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-26T06:33:20.847 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-26T06:33:20.847 INFO:teuthology.orchestra.run.smithi135.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-26T06:33:20.847 INFO:teuthology.orchestra.run.smithi135.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-26T06:33:20.847 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-26T06:33:20.847 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-26T06:33:20.848 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-26T06:33:20.848 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-26T06:33:20.848 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-26T06:33:20.848 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.848 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.849 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:33:20.849 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-26T06:33:20.849 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-26T06:33:20.849 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-26T06:33:20.849 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-26T06:33:20.849 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-26T06:33:20.850 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-26T06:33:20.850 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-10-26T06:33:20.850 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-10-26T06:33:21.001 DEBUG:teuthology.parallel:result is None 2023-10-26T06:33:24.034 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 58 kB/s | 24 kB 00:00 2023-10-26T06:33:25.144 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 53 MB 00:01 2023-10-26T06:33:33.945 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 47 MB 00:01 2023-10-26T06:33:45.588 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-26T06:33:45.589 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-26T06:33:45.604 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-26T06:33:45.605 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-26T06:33:45.606 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-26T06:33:45.608 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-26T06:33:45.610 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-26T06:33:45.613 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-26T06:33:45.614 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-26T06:33:45.752 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-10-26T06:33:45.764 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-10-26T06:33:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-10-26T06:33:45.765 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-10-26T06:33:45.765 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-10-26T06:33:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-26T06:33:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-26T06:33:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:18.0.0-6863.g38499368.el8 ceph 6.4 k 2023-10-26T06:33:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:18.0.0-6863.g38499368.el8 ceph 5.4 M 2023-10-26T06:33:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:18.0.0-6863.g38499368.el8 ceph 918 k 2023-10-26T06:33:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6863.g38499368.el8 ceph 167 k 2023-10-26T06:33:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:18.0.0-6863.g38499368.el8 ceph 1.6 M 2023-10-26T06:33:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 132 k 2023-10-26T06:33:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 1.8 M 2023-10-26T06:33:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 7.4 M 2023-10-26T06:33:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 48 k 2023-10-26T06:33:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:18.0.0-6863.g38499368.el8 ceph 14 M 2023-10-26T06:33:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:18.0.0-6863.g38499368.el8 ceph 49 M 2023-10-26T06:33:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 261 k 2023-10-26T06:33:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 191 k 2023-10-26T06:33:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:18.0.0-6863.g38499368.el8 ceph 261 k 2023-10-26T06:33:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 18 k 2023-10-26T06:33:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-26T06:33:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-26T06:33:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-26T06:33:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:18.0.0-6863.g38499368.el8 ceph 31 k 2023-10-26T06:33:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 840 k 2023-10-26T06:33:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:18.0.0-6863.g38499368.el8 ceph 139 k 2023-10-26T06:33:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-26T06:33:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-26T06:33:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-26T06:33:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-26T06:33:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-26T06:33:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-26T06:33:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:18.0.0-6863.g38499368.el8 ceph 197 k 2023-10-26T06:33:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:18.0.0-6863.g38499368.el8 ceph 371 k 2023-10-26T06:33:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:18.0.0-6863.g38499368.el8 ceph 364 k 2023-10-26T06:33:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:18.0.0-6863.g38499368.el8 ceph 107 k 2023-10-26T06:33:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:18.0.0-6863.g38499368.el8 ceph 94 k 2023-10-26T06:33:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.6 M 2023-10-26T06:33:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:18.0.0-6863.g38499368.el8 ceph 179 k 2023-10-26T06:33:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-26T06:33:45.773 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-10-26T06:33:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-26T06:33:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.7 M 2023-10-26T06:33:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:18.0.0-6863.g38499368.el8 ceph 3.7 M 2023-10-26T06:33:45.774 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-10-26T06:33:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-26T06:33:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:18.0.0-6863.g38499368.el8 ceph 23 M 2023-10-26T06:33:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 23 k 2023-10-26T06:33:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:18.0.0-6863.g38499368.el8 ceph 2.3 M 2023-10-26T06:33:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 242 k 2023-10-26T06:33:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:18.0.0-6863.g38499368.el8 ceph 4.6 M 2023-10-26T06:33:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:18.0.0-6863.g38499368.el8 ceph 18 M 2023-10-26T06:33:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6863.g38499368.el8 ceph-noarch 15 k 2023-10-26T06:33:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:18.0.0-6863.g38499368.el8 ceph 24 k 2023-10-26T06:33:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-26T06:33:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-26T06:33:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-26T06:33:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-26T06:33:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-26T06:33:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:18.0.0-6863.g38499368.el8 ceph 184 k 2023-10-26T06:33:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-26T06:33:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-26T06:33:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-26T06:33:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-26T06:33:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-26T06:33:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-26T06:33:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:18.0.0-6863.g38499368.el8 ceph 518 k 2023-10-26T06:33:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-26T06:33:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:18.0.0-6863.g38499368.el8 ceph 6.9 M 2023-10-26T06:33:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-26T06:33:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-26T06:33:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-26T06:33:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-26T06:33:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-26T06:33:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-26T06:33:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-26T06:33:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-26T06:33:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-26T06:33:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-26T06:33:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-26T06:33:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-26T06:33:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-26T06:33:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-26T06:33:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-26T06:33:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-26T06:33:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:18.0.0-6863.g38499368.el8 ceph 45 k 2023-10-26T06:33:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:18.0.0-6863.g38499368.el8 ceph 125 k 2023-10-26T06:33:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-26T06:33:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-26T06:33:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-26T06:33:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-26T06:33:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-26T06:33:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-26T06:33:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-26T06:33:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-26T06:33:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-26T06:33:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-26T06:33:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-26T06:33:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-26T06:33:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-26T06:33:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-26T06:33:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-26T06:33:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-26T06:33:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-26T06:33:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-26T06:33:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-26T06:33:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-26T06:33:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-26T06:33:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-26T06:33:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-26T06:33:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-26T06:33:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-26T06:33:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-26T06:33:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-26T06:33:45.787 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-26T06:33:45.787 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-26T06:33:45.787 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-26T06:33:45.787 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-26T06:33:45.787 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-26T06:33:45.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-26T06:33:45.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-26T06:33:45.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-26T06:33:45.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-26T06:33:45.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-26T06:33:45.788 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-26T06:33:45.789 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-26T06:33:45.789 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-26T06:33:45.789 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-26T06:33:45.789 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-26T06:33:45.789 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-26T06:33:45.790 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-26T06:33:45.790 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-26T06:33:45.790 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-26T06:33:45.790 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-26T06:33:45.790 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-26T06:33:45.791 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-26T06:33:45.791 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling module streams: 2023-10-26T06:33:45.791 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql 10 2023-10-26T06:33:45.791 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:33:45.791 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-10-26T06:33:45.791 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-10-26T06:33:45.792 INFO:teuthology.orchestra.run.smithi089.stdout:Install 128 Packages 2023-10-26T06:33:45.792 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 3 Packages 2023-10-26T06:33:45.792 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:33:45.793 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 209 M 2023-10-26T06:33:45.793 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-10-26T06:33:46.212 INFO:teuthology.orchestra.run.smithi089.stdout:(1/131): ceph-18.0.0-6863.g38499368.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-10-26T06:33:46.829 INFO:teuthology.orchestra.run.smithi089.stdout:(2/131): ceph-fuse-18.0.0-6863.g38499368.el8.x8 1.5 MB/s | 918 kB 00:00 2023-10-26T06:33:46.938 INFO:teuthology.orchestra.run.smithi089.stdout:(3/131): ceph-immutable-object-cache-18.0.0-686 1.5 MB/s | 167 kB 00:00 2023-10-26T06:33:47.288 INFO:teuthology.orchestra.run.smithi089.stdout:(4/131): ceph-base-18.0.0-6863.g38499368.el8.x8 3.8 MB/s | 5.4 MB 00:01 2023-10-26T06:33:47.430 INFO:teuthology.orchestra.run.smithi089.stdout:(5/131): ceph-mds-18.0.0-6863.g38499368.el8.x86 4.7 MB/s | 2.3 MB 00:00 2023-10-26T06:33:47.522 INFO:teuthology.orchestra.run.smithi089.stdout:(6/131): ceph-mgr-18.0.0-6863.g38499368.el8.x86 7.0 MB/s | 1.6 MB 00:00 2023-10-26T06:33:48.164 INFO:teuthology.orchestra.run.smithi089.stdout:(7/131): ceph-common-18.0.0-6863.g38499368.el8. 9.8 MB/s | 23 MB 00:02 2023-10-26T06:33:48.514 INFO:teuthology.orchestra.run.smithi089.stdout:(8/131): ceph-mon-18.0.0-6863.g38499368.el8.x86 4.3 MB/s | 4.6 MB 00:01 2023-10-26T06:33:48.864 INFO:teuthology.orchestra.run.smithi089.stdout:(9/131): ceph-selinux-18.0.0-6863.g38499368.el8 69 kB/s | 24 kB 00:00 2023-10-26T06:33:49.148 INFO:teuthology.orchestra.run.smithi089.stdout:(10/131): ceph-radosgw-18.0.0-6863.g38499368.el 14 MB/s | 14 MB 00:00 2023-10-26T06:33:49.748 INFO:teuthology.orchestra.run.smithi089.stdout:(11/131): ceph-osd-18.0.0-6863.g38499368.el8.x8 8.1 MB/s | 18 MB 00:02 2023-10-26T06:33:49.781 INFO:teuthology.orchestra.run.smithi089.stdout:(12/131): cephfs-mirror-18.0.0-6863.g38499368.e 418 kB/s | 261 kB 00:00 2023-10-26T06:33:49.807 INFO:teuthology.orchestra.run.smithi089.stdout:(13/131): libcephfs-devel-18.0.0-6863.g38499368 531 kB/s | 31 kB 00:00 2023-10-26T06:33:49.899 INFO:teuthology.orchestra.run.smithi089.stdout:(14/131): libcephfs2-18.0.0-6863.g38499368.el8. 6.5 MB/s | 840 kB 00:00 2023-10-26T06:33:50.049 INFO:teuthology.orchestra.run.smithi089.stdout:(15/131): librados-devel-18.0.0-6863.g38499368. 927 kB/s | 139 kB 00:00 2023-10-26T06:33:50.075 INFO:teuthology.orchestra.run.smithi089.stdout:(16/131): libcephsqlite-18.0.0-6863.g38499368.e 691 kB/s | 184 kB 00:00 2023-10-26T06:33:50.183 INFO:teuthology.orchestra.run.smithi089.stdout:(17/131): libradosstriper1-18.0.0-6863.g3849936 3.8 MB/s | 518 kB 00:00 2023-10-26T06:33:50.641 INFO:teuthology.orchestra.run.smithi089.stdout:(18/131): python3-ceph-argparse-18.0.0-6863.g38 99 kB/s | 45 kB 00:00 2023-10-26T06:33:50.726 INFO:teuthology.orchestra.run.smithi089.stdout:(19/131): librgw2-18.0.0-6863.g38499368.el8.x86 11 MB/s | 6.9 MB 00:00 2023-10-26T06:33:50.751 INFO:teuthology.orchestra.run.smithi089.stdout:(20/131): python3-ceph-common-18.0.0-6863.g3849 1.0 MB/s | 125 kB 00:00 2023-10-26T06:33:50.926 INFO:teuthology.orchestra.run.smithi089.stdout:(21/131): python3-cephfs-18.0.0-6863.g38499368. 986 kB/s | 197 kB 00:00 2023-10-26T06:33:50.952 INFO:teuthology.orchestra.run.smithi089.stdout:(22/131): python3-rados-18.0.0-6863.g38499368.e 1.8 MB/s | 371 kB 00:00 2023-10-26T06:33:51.102 INFO:teuthology.orchestra.run.smithi089.stdout:(23/131): python3-rgw-18.0.0-6863.g38499368.el8 714 kB/s | 107 kB 00:00 2023-10-26T06:33:51.127 INFO:teuthology.orchestra.run.smithi089.stdout:(24/131): python3-rbd-18.0.0-6863.g38499368.el8 1.8 MB/s | 364 kB 00:00 2023-10-26T06:33:51.227 INFO:teuthology.orchestra.run.smithi089.stdout:(25/131): rbd-fuse-18.0.0-6863.g38499368.el8.x8 752 kB/s | 94 kB 00:00 2023-10-26T06:33:51.361 INFO:teuthology.orchestra.run.smithi089.stdout:(26/131): rbd-nbd-18.0.0-6863.g38499368.el8.x86 1.3 MB/s | 179 kB 00:00 2023-10-26T06:33:51.428 INFO:teuthology.orchestra.run.smithi089.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6863.g 345 kB/s | 23 kB 00:00 2023-10-26T06:33:51.504 INFO:teuthology.orchestra.run.smithi089.stdout:(28/131): rbd-mirror-18.0.0-6863.g38499368.el8. 9.5 MB/s | 3.6 MB 00:00 2023-10-26T06:33:51.528 INFO:teuthology.orchestra.run.smithi089.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6863.g3849936 1.3 MB/s | 132 kB 00:00 2023-10-26T06:33:51.737 INFO:teuthology.orchestra.run.smithi089.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6863.g38499 7.6 MB/s | 1.8 MB 00:00 2023-10-26T06:33:51.829 INFO:teuthology.orchestra.run.smithi089.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6863.g38 2.6 MB/s | 242 kB 00:00 2023-10-26T06:33:51.904 INFO:teuthology.orchestra.run.smithi089.stdout:(32/131): ceph-mgr-rook-18.0.0-6863.g38499368.e 634 kB/s | 48 kB 00:00 2023-10-26T06:33:52.009 INFO:teuthology.orchestra.run.smithi089.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6863.g3 144 kB/s | 15 kB 00:00 2023-10-26T06:33:52.105 INFO:teuthology.orchestra.run.smithi089.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 13 MB/s | 7.4 MB 00:00 2023-10-26T06:33:52.139 INFO:teuthology.orchestra.run.smithi089.stdout:(35/131): ceph-volume-18.0.0-6863.g38499368.el8 2.0 MB/s | 261 kB 00:00 2023-10-26T06:33:52.166 INFO:teuthology.orchestra.run.smithi089.stdout:(36/131): cephadm-18.0.0-6863.g38499368.el8.noa 3.2 MB/s | 191 kB 00:00 2023-10-26T06:33:52.197 INFO:teuthology.orchestra.run.smithi089.stdout:(37/131): cephfs-top-18.0.0-6863.g38499368.el8. 310 kB/s | 18 kB 00:00 2023-10-26T06:33:52.531 INFO:teuthology.orchestra.run.smithi089.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 176 kB/s | 59 kB 00:00 2023-10-26T06:33:52.665 INFO:teuthology.orchestra.run.smithi089.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 612 kB/s | 306 kB 00:00 2023-10-26T06:33:52.773 INFO:teuthology.orchestra.run.smithi089.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 310 kB/s | 75 kB 00:00 2023-10-26T06:33:52.932 INFO:teuthology.orchestra.run.smithi089.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 275 kB/s | 44 kB 00:00 2023-10-26T06:33:53.041 INFO:teuthology.orchestra.run.smithi089.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-10-26T06:33:53.166 INFO:teuthology.orchestra.run.smithi089.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 264 kB/s | 33 kB 00:00 2023-10-26T06:33:53.192 INFO:teuthology.orchestra.run.smithi089.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 716 kB/s | 185 kB 00:00 2023-10-26T06:33:53.442 INFO:teuthology.orchestra.run.smithi089.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 690 kB/s | 173 kB 00:00 2023-10-26T06:33:53.467 INFO:teuthology.orchestra.run.smithi089.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 54 kB/s | 16 kB 00:00 2023-10-26T06:33:53.600 INFO:teuthology.orchestra.run.smithi089.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 610 kB/s | 81 kB 00:00 2023-10-26T06:33:53.726 INFO:teuthology.orchestra.run.smithi089.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-10-26T06:33:53.842 INFO:teuthology.orchestra.run.smithi089.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 960 kB/s | 384 kB 00:00 2023-10-26T06:33:53.868 INFO:teuthology.orchestra.run.smithi089.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 124 kB/s | 18 kB 00:00 2023-10-26T06:33:54.002 INFO:teuthology.orchestra.run.smithi089.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2023-10-26T06:33:54.527 INFO:teuthology.orchestra.run.smithi089.stdout:(52/131): ceph-test-18.0.0-6863.g38499368.el8.x 8.6 MB/s | 49 MB 00:05 2023-10-26T06:33:54.552 INFO:teuthology.orchestra.run.smithi089.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 107 kB/s | 59 kB 00:00 2023-10-26T06:33:54.636 INFO:teuthology.orchestra.run.smithi089.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 1.5 MB/s | 1.2 MB 00:00 2023-10-26T06:33:54.753 INFO:teuthology.orchestra.run.smithi089.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-10-26T06:33:54.831 INFO:teuthology.orchestra.run.smithi089.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 80 kB/s | 16 kB 00:00 2023-10-26T06:33:54.947 INFO:teuthology.orchestra.run.smithi089.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 173 kB/s | 34 kB 00:00 2023-10-26T06:33:55.021 INFO:teuthology.orchestra.run.smithi089.stdout:(58/131): python3-natsort-7.1.1-2.el8.noarch.rp 122 kB/s | 60 kB 00:00 2023-10-26T06:33:55.046 INFO:teuthology.orchestra.run.smithi089.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 653 kB/s | 64 kB 00:00 2023-10-26T06:33:55.071 INFO:teuthology.orchestra.run.smithi089.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 820 kB/s | 196 kB 00:00 2023-10-26T06:33:55.130 INFO:teuthology.orchestra.run.smithi089.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 171 kB/s | 19 kB 00:00 2023-10-26T06:33:55.155 INFO:teuthology.orchestra.run.smithi089.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-10-26T06:33:55.180 INFO:teuthology.orchestra.run.smithi089.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 268 kB/s | 29 kB 00:00 2023-10-26T06:33:55.230 INFO:teuthology.orchestra.run.smithi089.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-10-26T06:33:55.339 INFO:teuthology.orchestra.run.smithi089.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 565 kB/s | 61 kB 00:00 2023-10-26T06:33:55.365 INFO:teuthology.orchestra.run.smithi089.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-10-26T06:33:55.390 INFO:teuthology.orchestra.run.smithi089.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-10-26T06:33:55.449 INFO:teuthology.orchestra.run.smithi089.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 785 kB/s | 85 kB 00:00 2023-10-26T06:33:55.474 INFO:teuthology.orchestra.run.smithi089.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2023-10-26T06:33:55.499 INFO:teuthology.orchestra.run.smithi089.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-10-26T06:33:55.558 INFO:teuthology.orchestra.run.smithi089.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 562 kB/s | 47 kB 00:00 2023-10-26T06:33:55.591 INFO:teuthology.orchestra.run.smithi089.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-10-26T06:33:55.616 INFO:teuthology.orchestra.run.smithi089.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 732 kB/s | 43 kB 00:00 2023-10-26T06:33:55.642 INFO:teuthology.orchestra.run.smithi089.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-26T06:33:55.667 INFO:teuthology.orchestra.run.smithi089.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 962 kB/s | 48 kB 00:00 2023-10-26T06:33:55.692 INFO:teuthology.orchestra.run.smithi089.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-26T06:33:55.726 INFO:teuthology.orchestra.run.smithi089.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 728 kB/s | 43 kB 00:00 2023-10-26T06:33:55.751 INFO:teuthology.orchestra.run.smithi089.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2023-10-26T06:33:55.785 INFO:teuthology.orchestra.run.smithi089.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-10-26T06:33:55.818 INFO:teuthology.orchestra.run.smithi089.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-10-26T06:33:55.835 INFO:teuthology.orchestra.run.smithi089.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-10-26T06:33:55.860 INFO:teuthology.orchestra.run.smithi089.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-10-26T06:33:55.894 INFO:teuthology.orchestra.run.smithi089.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.4 MB/s | 559 kB 00:00 2023-10-26T06:33:55.911 INFO:teuthology.orchestra.run.smithi089.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 356 kB/s | 18 kB 00:00 2023-10-26T06:33:55.953 INFO:teuthology.orchestra.run.smithi089.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-10-26T06:33:55.978 INFO:teuthology.orchestra.run.smithi089.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-10-26T06:33:56.046 INFO:teuthology.orchestra.run.smithi089.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2023-10-26T06:33:56.088 INFO:teuthology.orchestra.run.smithi089.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-10-26T06:33:56.138 INFO:teuthology.orchestra.run.smithi089.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-10-26T06:33:56.180 INFO:teuthology.orchestra.run.smithi089.stdout:(90/131): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2023-10-26T06:33:56.205 INFO:teuthology.orchestra.run.smithi089.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 511 kB/s | 60 kB 00:00 2023-10-26T06:33:56.255 INFO:teuthology.orchestra.run.smithi089.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 839 kB/s | 98 kB 00:00 2023-10-26T06:33:56.281 INFO:teuthology.orchestra.run.smithi089.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 555 kB/s | 39 kB 00:00 2023-10-26T06:33:56.306 INFO:teuthology.orchestra.run.smithi089.stdout:(94/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 459 kB/s | 58 kB 00:00 2023-10-26T06:33:56.331 INFO:teuthology.orchestra.run.smithi089.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 171 kB 00:00 2023-10-26T06:33:56.356 INFO:teuthology.orchestra.run.smithi089.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-26T06:33:56.398 INFO:teuthology.orchestra.run.smithi089.stdout:(97/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.4 MB/s | 644 kB 00:00 2023-10-26T06:33:56.424 INFO:teuthology.orchestra.run.smithi089.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-10-26T06:33:56.449 INFO:teuthology.orchestra.run.smithi089.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-26T06:33:56.483 INFO:teuthology.orchestra.run.smithi089.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-10-26T06:33:56.516 INFO:teuthology.orchestra.run.smithi089.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-26T06:33:56.541 INFO:teuthology.orchestra.run.smithi089.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-10-26T06:33:56.583 INFO:teuthology.orchestra.run.smithi089.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-26T06:33:56.617 INFO:teuthology.orchestra.run.smithi089.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-10-26T06:33:56.642 INFO:teuthology.orchestra.run.smithi089.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-26T06:33:56.676 INFO:teuthology.orchestra.run.smithi089.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-26T06:33:56.726 INFO:teuthology.orchestra.run.smithi089.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.4 MB/s | 713 kB 00:00 2023-10-26T06:33:56.760 INFO:teuthology.orchestra.run.smithi089.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 801 kB/s | 27 kB 00:00 2023-10-26T06:33:56.802 INFO:teuthology.orchestra.run.smithi089.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-26T06:33:56.835 INFO:teuthology.orchestra.run.smithi089.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-10-26T06:33:56.986 INFO:teuthology.orchestra.run.smithi089.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-10-26T06:33:57.044 INFO:teuthology.orchestra.run.smithi089.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-10-26T06:33:57.120 INFO:teuthology.orchestra.run.smithi089.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2023-10-26T06:33:57.145 INFO:teuthology.orchestra.run.smithi089.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2023-10-26T06:33:57.162 INFO:teuthology.orchestra.run.smithi089.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-10-26T06:33:57.195 INFO:teuthology.orchestra.run.smithi089.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-26T06:33:57.221 INFO:teuthology.orchestra.run.smithi089.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 340 kB/s | 20 kB 00:00 2023-10-26T06:33:57.246 INFO:teuthology.orchestra.run.smithi089.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-26T06:33:57.297 INFO:teuthology.orchestra.run.smithi089.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-10-26T06:33:57.338 INFO:teuthology.orchestra.run.smithi089.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-10-26T06:33:57.514 INFO:teuthology.orchestra.run.smithi089.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 12 MB/s | 14 MB 00:01 2023-10-26T06:33:57.556 INFO:teuthology.orchestra.run.smithi089.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-26T06:33:57.598 INFO:teuthology.orchestra.run.smithi089.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-26T06:33:57.698 INFO:teuthology.orchestra.run.smithi089.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 11 MB/s | 4.9 MB 00:00 2023-10-26T06:33:57.756 INFO:teuthology.orchestra.run.smithi089.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-10-26T06:33:57.823 INFO:teuthology.orchestra.run.smithi089.stdout:(126/131): openblas-0.3.15-3.el8.x86_64.rpm 9.8 MB/s | 4.7 MB 00:00 2023-10-26T06:33:57.890 INFO:teuthology.orchestra.run.smithi089.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-10-26T06:33:57.982 INFO:teuthology.orchestra.run.smithi089.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-10-26T06:33:58.033 INFO:teuthology.orchestra.run.smithi089.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-26T06:33:58.658 INFO:teuthology.orchestra.run.smithi089.stdout:(130/131): librbd1-18.0.0-6863.g38499368.el8.x8 4.8 MB/s | 3.7 MB 00:00 2023-10-26T06:33:58.849 INFO:teuthology.orchestra.run.smithi089.stdout:(131/131): librados2-18.0.0-6863.g38499368.el8. 3.6 MB/s | 3.7 MB 00:01 2023-10-26T06:33:58.852 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-10-26T06:33:58.852 INFO:teuthology.orchestra.run.smithi089.stdout:Total 16 MB/s | 209 MB 00:13 2023-10-26T06:34:00.528 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-10-26T06:34:00.662 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-10-26T06:34:00.662 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-10-26T06:34:02.254 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-10-26T06:34:02.258 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-10-26T06:34:07.400 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-10-26T06:34:07.592 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-26T06:34:07.769 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-26T06:34:07.935 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-26T06:34:08.086 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-26T06:34:08.327 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-26T06:34:08.808 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-26T06:34:09.060 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:18.0.0-6863.g38499368.el8.x86_64 5/134 2023-10-26T06:34:09.537 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:18.0.0-6863.g38499368.el8.x86_64 5/134 2023-10-26T06:34:09.751 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:34:09.910 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:34:10.110 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:34:10.226 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:34:10.691 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:34:10.867 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-26T06:34:11.106 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-26T06:34:11.359 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-26T06:34:11.752 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-26T06:34:12.027 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-26T06:34:12.176 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 14/134 2023-10-26T06:34:12.309 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 14/134 2023-10-26T06:34:12.459 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 15/134 2023-10-26T06:34:12.639 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 15/134 2023-10-26T06:34:12.883 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-26T06:34:13.052 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-26T06:34:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-26T06:34:13.417 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-26T06:34:13.583 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-26T06:34:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-26T06:34:14.397 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-26T06:34:14.559 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-26T06:34:15.221 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-26T06:34:15.467 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-26T06:34:16.310 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-26T06:34:16.567 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-26T06:34:16.753 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-26T06:34:17.030 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-26T06:34:17.199 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:18.0.0-6863.g38499368.el8. 26/134 2023-10-26T06:34:17.322 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 27/134 2023-10-26T06:34:17.569 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_6 28/134 2023-10-26T06:34:17.750 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-26T06:34:19.816 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-26T06:34:20.970 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-26T06:34:21.346 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-26T06:34:21.607 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-26T06:34:21.900 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-26T06:34:22.103 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-26T06:34:22.304 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-26T06:34:22.532 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 36/134 2023-10-26T06:34:22.716 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:18.0.0-6863.g38499368.el8.x86_6 37/134 2023-10-26T06:34:22.867 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-26T06:34:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-26T06:34:23.232 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-26T06:34:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-26T06:34:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-26T06:34:23.755 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-26T06:34:23.941 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-26T06:34:24.035 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-26T06:34:24.260 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-26T06:34:24.412 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-26T06:34:24.599 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-26T06:34:24.769 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-26T06:34:24.948 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-26T06:34:25.079 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-26T06:34:25.327 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-26T06:34:25.536 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-26T06:34:26.298 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-26T06:34:26.623 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-26T06:34:26.872 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-26T06:34:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-26T06:34:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-26T06:34:27.724 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-26T06:34:27.942 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-26T06:34:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-26T06:34:28.384 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-26T06:34:28.556 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-26T06:34:28.720 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-26T06:34:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-26T06:34:29.387 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-26T06:34:29.680 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-26T06:34:29.893 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-26T06:34:30.083 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-26T06:34:30.297 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-26T06:34:30.572 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-26T06:34:30.781 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-26T06:34:30.953 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-26T06:34:31.111 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-26T06:34:31.387 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-26T06:34:31.589 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-26T06:34:31.813 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-26T06:34:31.985 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-26T06:34:32.130 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-26T06:34:32.344 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-26T06:34:32.562 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-26T06:34:32.755 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-26T06:34:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-26T06:34:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-26T06:34:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-26T06:34:33.604 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-26T06:34:34.936 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-26T06:34:35.268 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-26T06:34:35.536 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-26T06:34:35.929 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-26T06:34:36.247 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-26T06:34:36.463 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-26T06:34:37.338 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-26T06:34:37.489 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:18.0.0-6863.g38499368.el8.x86_64 92/134 2023-10-26T06:34:37.589 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:18.0.0-6863.g38499368.el8.x86_64 92/134 2023-10-26T06:34:37.756 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 93/134 2023-10-26T06:34:42.667 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:34:42.914 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:34:43.596 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 94/134 2023-10-26T06:34:43.881 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 95/134 2023-10-26T06:34:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 95/134 2023-10-26T06:34:44.155 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 96/134 2023-10-26T06:35:04.513 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 96/134 2023-10-26T06:35:04.513 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2023-10-26T06:35:04.513 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2023-10-26T06:35:04.514 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-10-26T06:35:04.514 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /run 2023-10-26T06:35:04.514 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2023-10-26T06:35:04.514 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2023-10-26T06:35:04.514 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2023-10-26T06:35:04.515 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2023-10-26T06:35:04.515 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-10-26T06:35:04.515 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:35:04.935 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 97/134 2023-10-26T06:35:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 97/134 2023-10-26T06:35:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 98/134 2023-10-26T06:35:07.112 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 98/134 2023-10-26T06:35:07.293 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 99/134 2023-10-26T06:35:07.827 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 99/134 2023-10-26T06:35:08.019 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 100/134 2023-10-26T06:35:08.060 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 100/134 2023-10-26T06:35:10.830 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:18.0.0-6863.g38499368.el8.noarch 101/134 2023-10-26T06:35:11.014 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:18.0.0-6863.g38499368.el8.noarch 101/134 2023-10-26T06:35:11.134 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 102/134 2023-10-26T06:35:11.366 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 103/134 2023-10-26T06:35:11.550 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 104/134 2023-10-26T06:35:11.831 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 104/134 2023-10-26T06:35:12.093 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 105/134 2023-10-26T06:35:13.180 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 105/134 2023-10-26T06:35:13.392 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 106/134 2023-10-26T06:35:13.523 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 106/134 2023-10-26T06:35:13.888 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 107/134 2023-10-26T06:35:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 108/134 2023-10-26T06:35:14.196 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 108/134 2023-10-26T06:35:14.375 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 109/134 2023-10-26T06:35:14.404 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 109/134 2023-10-26T06:35:15.814 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:18.0.0-6863.g38499368.el8.x86_64 110/134 2023-10-26T06:35:16.032 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 111/134 2023-10-26T06:35:16.094 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 111/134 2023-10-26T06:35:16.275 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6863.g38499 112/134 2023-10-26T06:35:16.346 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6863.g38499 112/134 2023-10-26T06:35:16.500 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 113/134 2023-10-26T06:35:16.921 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 113/134 2023-10-26T06:35:17.083 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 114/134 2023-10-26T06:35:22.120 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 114/134 2023-10-26T06:35:23.351 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 115/134 2023-10-26T06:35:23.717 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-26T06:35:23.897 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 117/134 2023-10-26T06:35:24.155 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-26T06:35:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-26T06:35:24.603 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-26T06:35:24.845 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-26T06:35:25.021 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_ 121/134 2023-10-26T06:35:25.158 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-26T06:35:28.641 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:35:28.932 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:35:28.975 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-26T06:35:29.216 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 124/134 2023-10-26T06:35:29.432 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 125/134 2023-10-26T06:35:29.809 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 126/134 2023-10-26T06:35:30.118 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-26T06:35:30.331 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-26T06:35:30.437 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-26T06:35:30.631 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-26T06:35:30.660 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-26T06:35:30.847 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-26T06:35:30.870 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-26T06:35:31.076 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-26T06:35:31.076 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:35:31.304 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:35:31.305 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-26T06:35:31.571 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-26T06:35:31.789 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:35:32.046 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:35:32.046 INFO:teuthology.orchestra.run.smithi089.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-26T06:35:32.046 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:35:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:18.0.0-6863.g38499368.el8.x86_64 1/134 2023-10-26T06:35:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 2/134 2023-10-26T06:35:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 3/134 2023-10-26T06:35:32.693 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 4/134 2023-10-26T06:35:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6863.g38499 5/134 2023-10-26T06:35:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 6/134 2023-10-26T06:35:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 7/134 2023-10-26T06:35:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 8/134 2023-10-26T06:35:32.694 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 9/134 2023-10-26T06:35:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 10/134 2023-10-26T06:35:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 11/134 2023-10-26T06:35:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 12/134 2023-10-26T06:35:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 13/134 2023-10-26T06:35:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_ 14/134 2023-10-26T06:35:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 15/134 2023-10-26T06:35:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 16/134 2023-10-26T06:35:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:18.0.0-6863.g38499368.el8.x86_6 17/134 2023-10-26T06:35:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:18.0.0-6863.g38499368.el8.x86 18/134 2023-10-26T06:35:32.696 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:18.0.0-6863.g38499368.el8.x86_64 19/134 2023-10-26T06:35:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 20/134 2023-10-26T06:35:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:18.0.0-6863.g38499368.el8. 21/134 2023-10-26T06:35:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_6 22/134 2023-10-26T06:35:32.697 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 23/134 2023-10-26T06:35:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 24/134 2023-10-26T06:35:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 25/134 2023-10-26T06:35:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 26/134 2023-10-26T06:35:32.698 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 27/134 2023-10-26T06:35:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 28/134 2023-10-26T06:35:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 29/134 2023-10-26T06:35:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noa 30/134 2023-10-26T06:35:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.n 31/134 2023-10-26T06:35:32.699 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 32/134 2023-10-26T06:35:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 33/134 2023-10-26T06:35:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 34/134 2023-10-26T06:35:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 35/134 2023-10-26T06:35:32.700 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 36/134 2023-10-26T06:35:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:18.0.0-6863.g38499368.el8.noarch 37/134 2023-10-26T06:35:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 38/134 2023-10-26T06:35:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-26T06:35:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-26T06:35:32.701 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-26T06:35:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-26T06:35:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-26T06:35:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-26T06:35:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-26T06:35:32.702 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-26T06:35:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-26T06:35:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-26T06:35:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-26T06:35:32.703 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-26T06:35:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-26T06:35:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-26T06:35:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-26T06:35:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-26T06:35:32.704 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-26T06:35:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-26T06:35:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-26T06:35:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-26T06:35:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-26T06:35:32.705 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-26T06:35:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-26T06:35:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-26T06:35:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-26T06:35:32.706 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-26T06:35:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-26T06:35:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-26T06:35:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-26T06:35:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-26T06:35:32.707 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-26T06:35:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-26T06:35:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-26T06:35:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-26T06:35:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-26T06:35:32.708 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-26T06:35:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-26T06:35:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-26T06:35:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-26T06:35:32.709 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-26T06:35:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-26T06:35:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-26T06:35:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-26T06:35:32.710 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-26T06:35:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-26T06:35:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-26T06:35:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-26T06:35:32.711 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-26T06:35:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-26T06:35:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-26T06:35:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-26T06:35:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-26T06:35:32.712 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-26T06:35:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-26T06:35:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-26T06:35:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-26T06:35:32.713 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-26T06:35:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-26T06:35:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-26T06:35:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-26T06:35:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-26T06:35:32.714 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-26T06:35:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-26T06:35:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-26T06:35:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-26T06:35:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-26T06:35:32.715 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-26T06:35:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-26T06:35:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-26T06:35:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-26T06:35:32.716 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-26T06:35:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-26T06:35:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-26T06:35:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-26T06:35:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-26T06:35:32.717 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-26T06:35:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-26T06:35:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-26T06:35:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-26T06:35:32.718 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-26T06:35:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-26T06:35:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-26T06:35:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-26T06:35:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-26T06:35:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-26T06:35:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-26T06:35:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-26T06:35:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-26T06:35:32.720 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-26T06:35:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-26T06:35:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:18.0.0-6863.g38499368.el8.x86_64 129/134 2023-10-26T06:35:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-26T06:35:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:18.0.0-6863.g38499368.el8.x86_64 131/134 2023-10-26T06:35:32.721 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-26T06:35:32.722 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-26T06:35:35.612 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-26T06:35:35.613 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:02:02 ago on Thu 26 Oct 2023 06:33:33 AM UTC. 2023-10-26T06:35:38.187 INFO:teuthology.orchestra.run.smithi089.stdout:Installed products updated. 2023-10-26T06:35:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:35:38.426 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-10-26T06:35:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-26T06:35:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.427 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-10-26T06:35:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-26T06:35:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.1-7.el8.noarch 2023-10-26T06:35:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.0.4-10.el8.x86_64 2023-10-26T06:35:38.429 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.429 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.429 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.429 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.430 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.430 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.430 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.430 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.431 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.431 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.431 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.431 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.432 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.432 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.432 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.432 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.433 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.433 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.433 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.433 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.433 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.434 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.434 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:18.0.0-6863.g38499368.el8.noarch 2023-10-26T06:35:38.434 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-26T06:35:38.434 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.1-9.el8.x86_64 2023-10-26T06:35:38.434 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-26T06:35:38.435 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-26T06:35:38.435 INFO:teuthology.orchestra.run.smithi089.stdout: jq-1.6-3.el8.x86_64 2023-10-26T06:35:38.435 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-26T06:35:38.435 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-26T06:35:38.435 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-26T06:35:38.436 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.436 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.436 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.436 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-26T06:35:38.436 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-26T06:35:38.437 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-26T06:35:38.437 INFO:teuthology.orchestra.run.smithi089.stdout: libpq-13.5-1.el8.x86_64 2023-10-26T06:35:38.437 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-26T06:35:38.437 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-26T06:35:38.437 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.438 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.438 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-26T06:35:38.438 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.438 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-26T06:35:38.438 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-26T06:35:38.439 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-26T06:35:38.439 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-26T06:35:38.439 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-26T06:35:38.439 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-26T06:35:38.440 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-26T06:35:38.440 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-26T06:35:38.440 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-26T06:35:38.440 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-26T06:35:38.440 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-26T06:35:38.441 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-26T06:35:38.441 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-26T06:35:38.441 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-26T06:35:38.441 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:35:38.441 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:35:38.442 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-26T06:35:38.442 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-26T06:35:38.442 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-26T06:35:38.442 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-26T06:35:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-26T06:35:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-26T06:35:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.443 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-26T06:35:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-26T06:35:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-26T06:35:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-26T06:35:38.444 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-26T06:35:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-26T06:35:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-26T06:35:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-26T06:35:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-26T06:35:38.445 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-26T06:35:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-26T06:35:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-26T06:35:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-26T06:35:38.446 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-26T06:35:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-26T06:35:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-26T06:35:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-26T06:35:38.450 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-26T06:35:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-26T06:35:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-26T06:35:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-26T06:35:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-26T06:35:38.451 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-2.6-1.el8.noarch 2023-10-26T06:35:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-26T06:35:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-26T06:35:38.452 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-26T06:35:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-26T06:35:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-26T06:35:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-26T06:35:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.453 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-26T06:35:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-26T06:35:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.454 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-26T06:35:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-26T06:35:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-26T06:35:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-26T06:35:38.455 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-26T06:35:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-26T06:35:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-26T06:35:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-26T06:35:38.456 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-26T06:35:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-26T06:35:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-26T06:35:38.457 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-26T06:35:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-26T06:35:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.458 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:18.0.0-6863.g38499368.el8.x86_64 2023-10-26T06:35:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-26T06:35:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-26T06:35:38.459 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-26T06:35:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-26T06:35:38.460 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-26T06:35:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-26T06:35:38.461 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:35:38.461 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-10-26T06:35:39.420 DEBUG:teuthology.parallel:result is None 2023-10-26T06:35:39.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:35:39.565 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-26T06:35:39.622 INFO:teuthology.orchestra.run.smithi089.stdout:18.0.0-6863.g38499368.el8 2023-10-26T06:35:39.623 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6863.g38499368.el8 2023-10-26T06:35:39.623 INFO:teuthology.task.install:The correct ceph version 18.0.0-6863.g38499368 is installed. 2023-10-26T06:35:39.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:35:39.770 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-26T06:35:39.800 INFO:teuthology.orchestra.run.smithi135.stdout:18.0.0-6863.g38499368.el8 2023-10-26T06:35:39.800 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6863.g38499368.el8 2023-10-26T06:35:39.801 INFO:teuthology.task.install:The correct ceph version 18.0.0-6863.g38499368 is installed. 2023-10-26T06:35:39.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:35:40.056 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-26T06:35:40.086 INFO:teuthology.orchestra.run.smithi192.stdout:18.0.0-6863.g38499368.el8 2023-10-26T06:35:40.086 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6863.g38499368.el8 2023-10-26T06:35:40.086 INFO:teuthology.task.install:The correct ceph version 18.0.0-6863.g38499368 is installed. 2023-10-26T06:35:40.087 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-26T06:35:40.087 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:35:40.087 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-26T06:35:40.178 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:35:40.178 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-26T06:35:40.227 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:35:40.227 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-26T06:35:40.275 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-26T06:35:40.275 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:35:40.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2023-10-26T06:35:40.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-26T06:35:40.451 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:35:40.452 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2023-10-26T06:35:40.484 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-26T06:35:40.556 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:35:40.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2023-10-26T06:35:40.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-26T06:35:40.660 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-26T06:35:40.660 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:35:40.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-26T06:35:40.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-26T06:35:40.836 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:35:40.836 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-26T06:35:40.868 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-26T06:35:40.940 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:35:40.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-26T06:35:40.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-26T06:35:41.044 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-26T06:35:41.045 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:35:41.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2023-10-26T06:35:41.113 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-26T06:35:41.220 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:35:41.220 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2023-10-26T06:35:41.252 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-26T06:35:41.324 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:35:41.324 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2023-10-26T06:35:41.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-26T06:35:41.426 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-26T06:35:41.515 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-10-26T06:35:41.515 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:35:41.516 INFO:tasks.cephadm:Cluster fsid is e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:35:41.517 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-26T06:35:41.517 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.89', 'mon.b': '172.21.15.135', 'mon.c': '172.21.15.192'} 2023-10-26T06:35:41.518 INFO:tasks.cephadm:First mon is mon.a on smithi089 2023-10-26T06:35:41.518 INFO:tasks.cephadm:First mgr is x 2023-10-26T06:35:41.522 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-26T06:35:41.522 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2023-10-26T06:35:41.592 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname $(hostname -s) 2023-10-26T06:35:41.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2023-10-26T06:35:41.658 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-26T06:35:41.658 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:35:41.806 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73553/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6863-g38499368', 'node_name': '172.21.4.64+confusa02', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6863.g38499368'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-24 19:42:09.877327', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20231024.144545', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73554/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6863-g38499368', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6863.g38499368'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-24 17:51:36.100178', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20231024.144545', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-10-26T06:35:42.028 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20231024.144545, sha1 384993681abceb8cb8affea10538d607052db4b4 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:35:42.029 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/x86_64/flavors/default/cephadm 2023-10-26T06:35:42.029 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:42.578 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 26 06:35 /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:42.578 DEBUG:teuthology.orchestra.run.smithi135:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:42.955 INFO:teuthology.orchestra.run.smithi135.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 26 06:35 /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:42.956 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:43.481 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 26 06:35 /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:43.481 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:43.533 DEBUG:teuthology.orchestra.run.smithi135:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:43.555 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-26T06:35:43.619 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 on all hosts... 2023-10-26T06:35:43.620 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 pull 2023-10-26T06:35:43.624 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 pull 2023-10-26T06:35:43.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 pull 2023-10-26T06:35:44.882 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4... 2023-10-26T06:35:44.895 INFO:teuthology.orchestra.run.smithi135.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4... 2023-10-26T06:35:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4... 2023-10-26T06:36:20.932 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:36:20.932 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T06:36:20.932 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "3c558842a6a58df17819a90ea0aee78a222bf7a961aede7c9e924f0377629632", 2023-10-26T06:36:20.933 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2023-10-26T06:36:20.933 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:157f71b6318e4fe6f68f85c327ecc42c0e9055d75a7cde3c8e6625892ed6b040" 2023-10-26T06:36:20.933 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:36:20.933 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:36:33.772 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-10-26T06:36:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T06:36:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "3c558842a6a58df17819a90ea0aee78a222bf7a961aede7c9e924f0377629632", 2023-10-26T06:36:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2023-10-26T06:36:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:157f71b6318e4fe6f68f85c327ecc42c0e9055d75a7cde3c8e6625892ed6b040" 2023-10-26T06:36:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2023-10-26T06:36:33.773 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-10-26T06:36:35.500 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2023-10-26T06:36:35.501 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T06:36:35.501 INFO:teuthology.orchestra.run.smithi135.stdout: "image_id": "3c558842a6a58df17819a90ea0aee78a222bf7a961aede7c9e924f0377629632", 2023-10-26T06:36:35.501 INFO:teuthology.orchestra.run.smithi135.stdout: "repo_digests": [ 2023-10-26T06:36:35.501 INFO:teuthology.orchestra.run.smithi135.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:157f71b6318e4fe6f68f85c327ecc42c0e9055d75a7cde3c8e6625892ed6b040" 2023-10-26T06:36:35.502 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2023-10-26T06:36:35.502 INFO:teuthology.orchestra.run.smithi135.stdout:} 2023-10-26T06:36:35.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2023-10-26T06:36:35.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph 2023-10-26T06:36:35.628 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2023-10-26T06:36:35.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2023-10-26T06:36:35.736 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /etc/ceph 2023-10-26T06:36:35.763 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2023-10-26T06:36:35.795 INFO:tasks.cephadm:Writing seed config... 2023-10-26T06:36:35.796 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-26T06:36:35.797 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-26T06:36:35.797 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-26T06:36:35.797 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-26T06:36:35.797 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-26T06:36:35.798 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-26T06:36:35.798 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-26T06:36:35.798 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-10-26T06:36:35.799 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-10-26T06:36:35.799 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-10-26T06:36:35.799 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-26T06:36:35.800 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-26T06:36:35.800 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-10-26T06:36:35.800 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-10-26T06:36:35.800 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-26T06:36:35.801 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-26T06:36:35.801 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-10-26T06:36:35.801 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2023-10-26T06:36:35.802 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-26T06:36:35.802 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-26T06:36:35.802 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-26T06:36:35.802 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-26T06:36:35.803 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2023-10-26T06:36:35.803 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-26T06:36:35.803 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-26T06:36:35.804 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-10-26T06:36:35.804 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:36:35.805 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-26T06:36:35.849 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e22b2d1a-73c9-11ee-8db9-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2023-10-26T06:36:35.850 DEBUG:teuthology.orchestra.run.smithi089:mon.a> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a.service 2023-10-26T06:36:35.894 DEBUG:teuthology.orchestra.run.smithi089:mgr.x> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.x.service 2023-10-26T06:36:35.939 INFO:tasks.cephadm:Bootstrapping... 2023-10-26T06:36:35.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 -v bootstrap --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.89 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-26T06:36:35.960 INFO:journalctl@ceph.mon.a.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:36:36.007 INFO:journalctl@ceph.mgr.x.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:36:36.168 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-10-26T06:36:36.169 INFO:teuthology.orchestra.run.smithi089.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', '-v', 'bootstrap', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.89', '--skip-admin-label'] 2023-10-26T06:36:36.343 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 4.1.1 2023-10-26T06:36:36.344 INFO:teuthology.orchestra.run.smithi089.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-26T06:36:36.344 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying podman|docker is present... 2023-10-26T06:36:36.515 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 4.1.1 2023-10-26T06:36:36.516 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying lvm2 is present... 2023-10-26T06:36:36.516 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying time synchronization is in place... 2023-10-26T06:36:36.530 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-26T06:36:36.530 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-26T06:36:36.542 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-26T06:36:36.543 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2023-10-26T06:36:36.556 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout enabled 2023-10-26T06:36:36.569 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout active 2023-10-26T06:36:36.569 INFO:teuthology.orchestra.run.smithi089.stdout:Unit chronyd.service is enabled and running 2023-10-26T06:36:36.569 INFO:teuthology.orchestra.run.smithi089.stdout:Repeating the final host check... 2023-10-26T06:36:36.739 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 4.1.1 2023-10-26T06:36:36.740 INFO:teuthology.orchestra.run.smithi089.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-26T06:36:36.740 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl is present 2023-10-26T06:36:36.740 INFO:teuthology.orchestra.run.smithi089.stdout:lvcreate is present 2023-10-26T06:36:36.752 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-26T06:36:36.753 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-26T06:36:36.765 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-26T06:36:36.765 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2023-10-26T06:36:36.779 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout enabled 2023-10-26T06:36:36.792 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout active 2023-10-26T06:36:36.792 INFO:teuthology.orchestra.run.smithi089.stdout:Unit chronyd.service is enabled and running 2023-10-26T06:36:36.792 INFO:teuthology.orchestra.run.smithi089.stdout:Host looks OK 2023-10-26T06:36:36.793 INFO:teuthology.orchestra.run.smithi089.stdout:Cluster fsid: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:36.793 INFO:teuthology.orchestra.run.smithi089.stdout:Acquiring lock 140115939043816 on /run/cephadm/e22b2d1a-73c9-11ee-8db9-212e2dc638e7.lock 2023-10-26T06:36:36.793 INFO:teuthology.orchestra.run.smithi089.stdout:Lock 140115939043816 acquired on /run/cephadm/e22b2d1a-73c9-11ee-8db9-212e2dc638e7.lock 2023-10-26T06:36:36.794 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying IP 172.21.15.89 port 3300 ... 2023-10-26T06:36:36.794 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying IP 172.21.15.89 port 6789 ... 2023-10-26T06:36:36.794 INFO:teuthology.orchestra.run.smithi089.stdout:Base mon IP(s) is [172.21.15.89:3300, 172.21.15.89:6789], mon addrv is [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2023-10-26T06:36:36.801 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-10-26T06:36:36.801 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.89 metric 100 2023-10-26T06:36:36.808 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-26T06:36:36.808 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-10-26T06:36:36.808 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-10-26T06:36:36.814 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-26T06:36:36.815 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-26T06:36:36.815 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-26T06:36:36.815 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-10-26T06:36:36.815 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout inet6 fe80::e999:b82c:300:16e6/64 scope link noprefixroute 2023-10-26T06:36:36.816 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-26T06:36:36.816 INFO:teuthology.orchestra.run.smithi089.stdout:Mon IP `172.21.15.89` is in CIDR network `172.21.0.0/20` 2023-10-26T06:36:36.817 INFO:teuthology.orchestra.run.smithi089.stdout:Mon IP `172.21.15.89` is in CIDR network `172.21.0.0/20` 2023-10-26T06:36:36.817 INFO:teuthology.orchestra.run.smithi089.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-26T06:36:36.817 INFO:teuthology.orchestra.run.smithi089.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-26T06:36:36.817 INFO:teuthology.orchestra.run.smithi089.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4... 2023-10-26T06:36:37.292 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 3c558842a6a58df17819a90ea0aee78a222bf7a961aede7c9e924f0377629632 2023-10-26T06:36:37.292 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4... 2023-10-26T06:36:37.292 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Getting image source signatures 2023-10-26T06:36:37.293 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying blob sha256:e0288be9f59cf54e9f695ad8ed1728839ca25cdcdf0f94b536445ee2ca502075 2023-10-26T06:36:37.293 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying blob sha256:bbe8e0105070d322009febb1dbcee79a3945eab3305036f79cc6f757ea8642c0 2023-10-26T06:36:37.293 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying config sha256:3c558842a6a58df17819a90ea0aee78a222bf7a961aede7c9e924f0377629632 2023-10-26T06:36:37.293 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-26T06:36:37.293 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Storing signatures 2023-10-26T06:36:38.272 INFO:teuthology.orchestra.run.smithi089.stdout:ceph: stdout ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev) 2023-10-26T06:36:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:Ceph version: ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev) 2023-10-26T06:36:38.273 INFO:teuthology.orchestra.run.smithi089.stdout:Extracting ceph user uid/gid from container image... 2023-10-26T06:36:39.149 INFO:teuthology.orchestra.run.smithi089.stdout:stat: stdout 167 167 2023-10-26T06:36:39.149 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial keys... 2023-10-26T06:36:40.025 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQD3CDplWLVqIBAA28ta7nCPL2NMlEnRWef3gw== 2023-10-26T06:36:41.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQD4CDplzch0GRAAV3YH2MEFWF8V+oxbmwWgZA== 2023-10-26T06:36:42.086 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQD5CDpl8YPaJhAAdJ4+sAR7/S7yeUoM1ju2Jw== 2023-10-26T06:36:42.087 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial monmap... 2023-10-26T06:36:42.968 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-26T06:36:42.968 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-26T06:36:42.968 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:42.969 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-26T06:36:42.969 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool for a [v2:172.21.15.89:3300,v1:172.21.15.89:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-26T06:36:42.969 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = pacific 2023-10-26T06:36:42.969 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: set fsid to e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:42.969 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-26T06:36:42.970 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:36:42.970 INFO:teuthology.orchestra.run.smithi089.stdout:Creating mon... 2023-10-26T06:36:43.843 INFO:teuthology.orchestra.run.smithi089.stdout:create mon.a on 2023-10-26T06:36:44.084 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-26T06:36:44.286 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-26T06:36:44.534 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7.target → /etc/systemd/system/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7.target. 2023-10-26T06:36:44.535 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7.target → /etc/systemd/system/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7.target. 2023-10-26T06:36:44.901 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a 2023-10-26T06:36:44.901 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a.service: Unit ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a.service not loaded. 2023-10-26T06:36:45.099 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7.target.wants/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@.service. 2023-10-26T06:36:45.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:45 smithi089 systemd[1]: Starting Ceph mon.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:36:45.891 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:45 smithi089 podman[94482]: 2023-10-26T06:36:45.891 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:45 smithi089 bash[94482]: bd9bc8dbb2567f4b7d5afe6cfd24736b7aba34e0e4526916432d49a66da83ba0 2023-10-26T06:36:45.915 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-26T06:36:45.916 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2023-10-26T06:36:45.932 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-26T06:36:45.932 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2023-10-26T06:36:45.932 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2023-10-26T06:36:45.932 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to enable service . firewalld.service is not available 2023-10-26T06:36:45.933 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mon to start... 2023-10-26T06:36:45.933 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mon... 2023-10-26T06:36:46.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:45 smithi089 systemd[1]: Started Ceph mon.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:36:46.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:45 smithi089 ceph-mon[94524]: mkfs e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:46.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:45 smithi089 ceph-mon[94524]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-26T06:36:47.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:46 smithi089 ceph-mon[94524]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-26T06:36:47.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:46 smithi089 ceph-mon[94524]: monmap e1: 1 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:36:47.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:46 smithi089 ceph-mon[94524]: fsmap 2023-10-26T06:36:47.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:46 smithi089 ceph-mon[94524]: osdmap e1: 0 total, 0 up, 0 in 2023-10-26T06:36:47.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:46 smithi089 ceph-mon[94524]: mgrmap e1: no daemons active 2023-10-26T06:36:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:53 smithi089 ceph-mon[94524]: from='client.? 172.21.15.89:0/3159250493' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-26T06:36:54.740 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout cluster: 2023-10-26T06:36:54.740 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout id: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:54.740 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-26T06:36:54.741 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:36:54.741 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout services: 2023-10-26T06:36:54.741 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 7s) 2023-10-26T06:36:54.741 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-26T06:36:54.741 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-26T06:36:54.742 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:36:54.742 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout data: 2023-10-26T06:36:54.742 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-26T06:36:54.742 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-26T06:36:54.742 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-26T06:36:54.743 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout pgs: 2023-10-26T06:36:54.743 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:36:54.743 INFO:teuthology.orchestra.run.smithi089.stdout:mon is available 2023-10-26T06:36:54.743 INFO:teuthology.orchestra.run.smithi089.stdout:Assimilating anything we can from ceph.conf... 2023-10-26T06:36:55.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:55 smithi089 ceph-mon[94524]: from='client.? 172.21.15.89:0/1800639500' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-26T06:36:55.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:55 smithi089 ceph-mon[94524]: from='client.? 172.21.15.89:0/1800639500' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-26T06:36:55.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:36:55.919 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [global] 2023-10-26T06:36:55.919 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout fsid = e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:55.919 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2023-10-26T06:36:55.919 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-26T06:36:55.920 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-26T06:36:55.920 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-26T06:36:55.920 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-26T06:36:55.920 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:36:55.920 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-26T06:36:55.921 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-26T06:36:55.921 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:36:55.921 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [osd] 2023-10-26T06:36:55.921 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2023-10-26T06:36:55.921 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-26T06:36:55.921 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-26T06:36:55.922 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-26T06:36:55.922 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-26T06:36:55.922 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-26T06:36:55.922 INFO:teuthology.orchestra.run.smithi089.stdout:Generating new minimal ceph.conf... 2023-10-26T06:36:56.994 INFO:teuthology.orchestra.run.smithi089.stdout:Restarting the monitor... 2023-10-26T06:36:57.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:56 smithi089 ceph-mon[94524]: from='client.? 172.21.15.89:0/442155843' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:36:57.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:57 smithi089 systemd[1]: Stopping Ceph mon.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:36:57.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-a[94500]: 2023-10-26T06:36:57.465+0000 7f8ad6851700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:36:57.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-a[94500]: 2023-10-26T06:36:57.465+0000 7f8ad6851700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-26T06:36:58.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:57 smithi089 bash[94860]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-a 2023-10-26T06:36:58.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a.service: Succeeded. 2023-10-26T06:36:58.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 systemd[1]: Stopped Ceph mon.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:36:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 systemd[1]: Starting Ceph mon.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:36:58.964 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 podman[94991]: 2023-10-26T06:36:58.964 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 bash[94991]: 72ab460de1dd16aeac158aaefc72edbc43f83334b745a9cc078943bd98c04aea 2023-10-26T06:36:58.965 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: set uid:gid to 167:167 (ceph:ceph) 2023-10-26T06:36:58.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev), process ceph-mon, pid 7 2023-10-26T06:36:58.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: pidfile_write: ignore empty --pid-file 2023-10-26T06:36:58.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: load: jerasure load: lrc 2023-10-26T06:36:58.982 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: RocksDB version: 7.9.2 2023-10-26T06:36:58.982 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Git sha 0 2023-10-26T06:36:58.982 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Compile date 2023-10-24 16:36:40 2023-10-26T06:36:58.982 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: DB SUMMARY 2023-10-26T06:36:58.982 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: DB Session ID: BVLP3UGMGGDBLYE0QZT9 2023-10-26T06:36:58.983 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: CURRENT file: CURRENT 2023-10-26T06:36:58.983 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: IDENTITY file: IDENTITY 2023-10-26T06:36:58.983 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-26T06:36:58.983 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-26T06:36:58.983 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 91309 ; 2023-10-26T06:36:58.983 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.error_if_exists: 0 2023-10-26T06:36:58.984 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.create_if_missing: 0 2023-10-26T06:36:58.984 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.paranoid_checks: 1 2023-10-26T06:36:58.984 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-26T06:36:58.984 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-26T06:36:58.984 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-26T06:36:58.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.env: 0x5585958d2da0 2023-10-26T06:36:58.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.fs: PosixFileSystem 2023-10-26T06:36:58.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.info_log: 0x5585974416a0 2023-10-26T06:36:58.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_file_opening_threads: 16 2023-10-26T06:36:58.986 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.statistics: (nil) 2023-10-26T06:36:58.986 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.use_fsync: 0 2023-10-26T06:36:58.986 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_log_file_size: 0 2023-10-26T06:36:58.986 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-26T06:36:58.986 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-26T06:36:58.987 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.keep_log_file_num: 1000 2023-10-26T06:36:58.987 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.recycle_log_file_num: 0 2023-10-26T06:36:58.987 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.allow_fallocate: 1 2023-10-26T06:36:58.987 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.allow_mmap_reads: 0 2023-10-26T06:36:58.988 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.allow_mmap_writes: 0 2023-10-26T06:36:58.988 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.use_direct_reads: 0 2023-10-26T06:36:58.988 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-26T06:36:58.988 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.create_missing_column_families: 0 2023-10-26T06:36:58.988 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.db_log_dir: 2023-10-26T06:36:58.989 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.wal_dir: 2023-10-26T06:36:58.989 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-26T06:36:58.989 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-26T06:36:58.990 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-26T06:36:58.990 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-26T06:36:58.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-26T06:36:58.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-26T06:36:58.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.advise_random_on_open: 1 2023-10-26T06:36:58.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.db_write_buffer_size: 0 2023-10-26T06:36:58.992 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.write_buffer_manager: 0x5585973f9360 2023-10-26T06:36:58.992 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-26T06:36:58.992 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-26T06:36:58.992 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-26T06:36:58.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.rate_limiter: (nil) 2023-10-26T06:36:58.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-26T06:36:58.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.wal_recovery_mode: 2 2023-10-26T06:36:58.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.enable_thread_tracking: 0 2023-10-26T06:36:58.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.enable_pipelined_write: 0 2023-10-26T06:36:58.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.unordered_write: 0 2023-10-26T06:36:58.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-26T06:36:58.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-26T06:36:58.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-26T06:36:58.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-26T06:36:58.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.row_cache: None 2023-10-26T06:36:58.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.wal_filter: None 2023-10-26T06:36:58.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-26T06:36:58.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.allow_ingest_behind: 0 2023-10-26T06:36:58.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.two_write_queues: 0 2023-10-26T06:36:58.996 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.manual_wal_flush: 0 2023-10-26T06:36:58.996 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.wal_compression: 0 2023-10-26T06:36:58.996 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.atomic_flush: 0 2023-10-26T06:36:58.996 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-26T06:36:58.996 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-26T06:36:58.997 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-26T06:36:58.997 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.log_readahead_size: 0 2023-10-26T06:36:58.997 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-26T06:36:58.997 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.best_efforts_recovery: 0 2023-10-26T06:36:58.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-26T06:36:58.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-26T06:36:58.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.allow_data_in_errors: 0 2023-10-26T06:36:58.999 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.db_host_id: __hostname__ 2023-10-26T06:36:58.999 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-26T06:36:58.999 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_background_jobs: 2 2023-10-26T06:36:58.999 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_background_compactions: -1 2023-10-26T06:36:59.000 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_subcompactions: 1 2023-10-26T06:36:59.000 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-26T06:36:59.000 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-26T06:36:59.000 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-26T06:36:59.000 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_total_wal_size: 0 2023-10-26T06:36:59.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-26T06:36:59.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-26T06:36:59.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-26T06:36:59.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-26T06:36:59.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_open_files: -1 2023-10-26T06:36:59.002 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bytes_per_sync: 0 2023-10-26T06:36:59.002 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-26T06:36:59.002 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-26T06:36:59.002 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_readahead_size: 0 2023-10-26T06:36:59.002 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_background_flushes: -1 2023-10-26T06:36:59.003 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Compression algorithms supported: 2023-10-26T06:36:59.003 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-26T06:36:59.003 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kZSTD supported: 0 2023-10-26T06:36:59.003 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kXpressCompression supported: 0 2023-10-26T06:36:59.003 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kLZ4HCCompression supported: 1 2023-10-26T06:36:59.003 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kZlibCompression supported: 1 2023-10-26T06:36:59.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kSnappyCompression supported: 1 2023-10-26T06:36:59.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kLZ4Compression supported: 1 2023-10-26T06:36:59.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: kBZip2Compression supported: 0 2023-10-26T06:36:59.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-26T06:36:59.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-26T06:36:59.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-26T06:36:59.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-26T06:36:59.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-26T06:36:59.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.merge_operator: 2023-10-26T06:36:59.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_filter: None 2023-10-26T06:36:59.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_filter_factory: None 2023-10-26T06:36:59.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.sst_partitioner_factory: None 2023-10-26T06:36:59.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-26T06:36:59.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-26T06:36:59.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5585974417a0) 2023-10-26T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi089.stdout: cache_index_and_filter_blocks: 1 2023-10-26T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi089.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-26T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi089.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-26T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi089.stdout: pin_top_level_index_and_filter: 1 2023-10-26T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_type: 0 2023-10-26T06:36:59.008 INFO:journalctl@ceph.mon.a.smithi089.stdout: data_block_index_type: 0 2023-10-26T06:36:59.008 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_shortening: 1 2023-10-26T06:36:59.008 INFO:journalctl@ceph.mon.a.smithi089.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-26T06:36:59.008 INFO:journalctl@ceph.mon.a.smithi089.stdout: checksum: 4 2023-10-26T06:36:59.008 INFO:journalctl@ceph.mon.a.smithi089.stdout: no_block_cache: 0 2023-10-26T06:36:59.009 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache: 0x5585974971f0 2023-10-26T06:36:59.009 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_name: BinnedLRUCache 2023-10-26T06:36:59.009 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_options: 2023-10-26T06:36:59.009 INFO:journalctl@ceph.mon.a.smithi089.stdout: capacity : 536870912 2023-10-26T06:36:59.009 INFO:journalctl@ceph.mon.a.smithi089.stdout: num_shard_bits : 4 2023-10-26T06:36:59.010 INFO:journalctl@ceph.mon.a.smithi089.stdout: strict_capacity_limit : 0 2023-10-26T06:36:59.010 INFO:journalctl@ceph.mon.a.smithi089.stdout: high_pri_pool_ratio: 0.000 2023-10-26T06:36:59.010 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_compressed: (nil) 2023-10-26T06:36:59.010 INFO:journalctl@ceph.mon.a.smithi089.stdout: persistent_cache: (nil) 2023-10-26T06:36:59.010 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_size: 4096 2023-10-26T06:36:59.010 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_size_deviation: 10 2023-10-26T06:36:59.011 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_restart_interval: 16 2023-10-26T06:36:59.011 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_block_restart_interval: 1 2023-10-26T06:36:59.011 INFO:journalctl@ceph.mon.a.smithi089.stdout: metadata_block_size: 4096 2023-10-26T06:36:59.011 INFO:journalctl@ceph.mon.a.smithi089.stdout: partition_filters: 0 2023-10-26T06:36:59.011 INFO:journalctl@ceph.mon.a.smithi089.stdout: use_delta_encoding: 1 2023-10-26T06:36:59.012 INFO:journalctl@ceph.mon.a.smithi089.stdout: filter_policy: bloomfilter 2023-10-26T06:36:59.012 INFO:journalctl@ceph.mon.a.smithi089.stdout: whole_key_filtering: 1 2023-10-26T06:36:59.012 INFO:journalctl@ceph.mon.a.smithi089.stdout: verify_compression: 0 2023-10-26T06:36:59.012 INFO:journalctl@ceph.mon.a.smithi089.stdout: read_amp_bytes_per_bit: 0 2023-10-26T06:36:59.012 INFO:journalctl@ceph.mon.a.smithi089.stdout: format_version: 5 2023-10-26T06:36:59.012 INFO:journalctl@ceph.mon.a.smithi089.stdout: enable_index_compression: 1 2023-10-26T06:36:59.013 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_align: 0 2023-10-26T06:36:59.013 INFO:journalctl@ceph.mon.a.smithi089.stdout: max_auto_readahead_size: 262144 2023-10-26T06:36:59.013 INFO:journalctl@ceph.mon.a.smithi089.stdout: prepopulate_block_cache: 0 2023-10-26T06:36:59.013 INFO:journalctl@ceph.mon.a.smithi089.stdout: initial_auto_readahead_size: 8192 2023-10-26T06:36:59.013 INFO:journalctl@ceph.mon.a.smithi089.stdout: num_file_reads_for_auto_readahead: 2 2023-10-26T06:36:59.014 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.write_buffer_size: 33554432 2023-10-26T06:36:59.014 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_write_buffer_number: 2 2023-10-26T06:36:59.014 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression: NoCompression 2023-10-26T06:36:59.014 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression: Disabled 2023-10-26T06:36:59.014 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.prefix_extractor: nullptr 2023-10-26T06:36:59.015 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-26T06:36:59.015 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.num_levels: 7 2023-10-26T06:36:59.015 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-26T06:36:59.015 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-26T06:36:59.015 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-26T06:36:59.016 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-26T06:36:59.016 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-26T06:36:59.016 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-26T06:36:59.016 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-26T06:36:59.016 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-26T06:36:59.017 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-26T06:36:59.017 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-26T06:36:59.017 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-26T06:36:59.017 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-26T06:36:59.018 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-26T06:36:59.018 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.level: 32767 2023-10-26T06:36:59.018 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.strategy: 0 2023-10-26T06:36:59.018 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-26T06:36:59.018 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-26T06:36:59.019 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-26T06:36:59.019 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-26T06:36:59.019 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.enabled: false 2023-10-26T06:36:59.019 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-26T06:36:59.020 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-26T06:36:59.020 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-26T06:36:59.020 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-26T06:36:59.020 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.target_file_size_base: 67108864 2023-10-26T06:36:59.021 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-26T06:36:59.021 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-26T06:36:59.021 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-26T06:36:59.021 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-26T06:36:59.021 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-26T06:36:59.022 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-26T06:36:59.022 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-26T06:36:59.022 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-26T06:36:59.022 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-26T06:36:59.023 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-26T06:36:59.023 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-26T06:36:59.023 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-26T06:36:59.023 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-26T06:36:59.023 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-26T06:36:59.024 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.arena_block_size: 1048576 2023-10-26T06:36:59.024 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-26T06:36:59.024 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-26T06:36:59.024 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.disable_auto_compactions: 0 2023-10-26T06:36:59.025 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-26T06:36:59.025 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-26T06:36:59.025 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-26T06:36:59.026 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-26T06:36:59.026 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-26T06:36:59.026 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-26T06:36:59.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-26T06:36:59.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-26T06:36:59.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-26T06:36:59.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-26T06:36:59.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.table_properties_collectors: 2023-10-26T06:36:59.028 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.inplace_update_support: 0 2023-10-26T06:36:59.028 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-26T06:36:59.028 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-26T06:36:59.028 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-26T06:36:59.028 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-26T06:36:59.029 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.bloom_locality: 0 2023-10-26T06:36:59.029 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.max_successive_merges: 0 2023-10-26T06:36:59.029 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-26T06:36:59.029 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.paranoid_file_checks: 0 2023-10-26T06:36:59.029 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.force_consistency_checks: 1 2023-10-26T06:36:59.030 INFO:teuthology.orchestra.run.smithi089.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-26T06:36:59.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.report_bg_io_stats: 0 2023-10-26T06:36:59.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.ttl: 2592000 2023-10-26T06:36:59.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-26T06:36:59.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-26T06:36:59.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-26T06:36:59.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.enable_blob_files: false 2023-10-26T06:36:59.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.min_blob_size: 0 2023-10-26T06:36:59.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.blob_file_size: 268435456 2023-10-26T06:36:59.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-26T06:36:59.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-26T06:36:59.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-26T06:36:59.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-26T06:36:59.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-26T06:36:59.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.blob_file_starting_level: 0 2023-10-26T06:36:59.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-26T06:36:59.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-26T06:36:59.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-26T06:36:59.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: df68c639-24e8-4c68-846d-4c67fcb06732 2023-10-26T06:36:59.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302218973018, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-26T06:36:59.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-26T06:36:59.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302218974517, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 86845, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 300, "table_properties": {"data_size": 84925, "index_size": 266, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11495, "raw_average_key_size": 48, "raw_value_size": 78456, "raw_average_value_size": 328, "num_data_blocks": 11, "num_entries": 239, "num_filter_entries": 239, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1698302218, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "df68c639-24e8-4c68-846d-4c67fcb06732", "db_session_id": "BVLP3UGMGGDBLYE0QZT9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-26T06:36:59.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302218974633, "job": 1, "event": "recovery_finished"} 2023-10-26T06:36:59.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-26T06:36:59.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-26T06:36:59.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558597534000 2023-10-26T06:36:59.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: DB pointer 0x558597520000 2023-10-26T06:36:59.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-26T06:36:59.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-26T06:36:59.296 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** DB Stats ** 2023-10-26T06:36:59.296 INFO:journalctl@ceph.mon.a.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-26T06:36:59.296 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-26T06:36:59.296 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-26T06:36:59.296 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-26T06:36:59.297 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-26T06:36:59.297 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-26T06:36:59.297 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-26T06:36:59.297 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2023-10-26T06:36:59.297 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** Compaction Stats [default] ** 2023-10-26T06:36:59.298 INFO:journalctl@ceph.mon.a.smithi089.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-26T06:36:59.298 INFO:journalctl@ceph.mon.a.smithi089.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-26T06:36:59.298 INFO:journalctl@ceph.mon.a.smithi089.stdout: L0 2/0 86.64 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:36:59.298 INFO:journalctl@ceph.mon.a.smithi089.stdout: Sum 2/0 86.64 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:36:59.298 INFO:journalctl@ceph.mon.a.smithi089.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:36:59.299 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2023-10-26T06:36:59.299 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** Compaction Stats [default] ** 2023-10-26T06:36:59.299 INFO:journalctl@ceph.mon.a.smithi089.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-26T06:36:59.299 INFO:journalctl@ceph.mon.a.smithi089.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-26T06:36:59.299 INFO:journalctl@ceph.mon.a.smithi089.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:36:59.300 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2023-10-26T06:36:59.300 INFO:journalctl@ceph.mon.a.smithi089.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-26T06:36:59.300 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2023-10-26T06:36:59.300 INFO:journalctl@ceph.mon.a.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-26T06:36:59.300 INFO:journalctl@ceph.mon.a.smithi089.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-26T06:36:59.301 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-26T06:36:59.301 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-26T06:36:59.301 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-26T06:36:59.301 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-26T06:36:59.301 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative compaction: 0.00 GB write, 13.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-26T06:36:59.302 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval compaction: 0.00 GB write, 13.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-26T06:36:59.302 INFO:journalctl@ceph.mon.a.smithi089.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-26T06:36:59.302 INFO:journalctl@ceph.mon.a.smithi089.stdout: Block cache BinnedLRUCache@0x5585974971f0#7 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-10-26T06:36:59.302 INFO:journalctl@ceph.mon.a.smithi089.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-10-26T06:36:59.302 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2023-10-26T06:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-26T06:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: starting mon.a rank 0 at public addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] at bind addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???) e1 preinit fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???).mds e1 new map 2023-10-26T06:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???).mds e1 print_map 2023-10-26T06:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout: e1 2023-10-26T06:36:59.304 INFO:journalctl@ceph.mon.a.smithi089.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-26T06:36:59.304 INFO:journalctl@ceph.mon.a.smithi089.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-26T06:36:59.304 INFO:journalctl@ceph.mon.a.smithi089.stdout: legacy client fscid: -1 2023-10-26T06:36:59.304 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2023-10-26T06:36:59.305 INFO:journalctl@ceph.mon.a.smithi089.stdout: No filesystems configured 2023-10-26T06:36:59.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-26T06:36:59.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:36:59.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:36:59.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:36:59.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:58 smithi089 ceph-mon[95031]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-26T06:36:59.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:59 smithi089 systemd[1]: Started Ceph mon.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:36:59.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:59 smithi089 ceph-mon[95031]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-26T06:36:59.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:59 smithi089 ceph-mon[95031]: monmap e1: 1 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:36:59.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:59 smithi089 ceph-mon[95031]: fsmap 2023-10-26T06:36:59.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:59 smithi089 ceph-mon[95031]: osdmap e1: 0 total, 0 up, 0 in 2023-10-26T06:36:59.307 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:36:59 smithi089 ceph-mon[95031]: mgrmap e1: no daemons active 2023-10-26T06:37:00.112 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-26T06:37:00.143 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-26T06:37:00.143 INFO:teuthology.orchestra.run.smithi089.stdout:Creating mgr... 2023-10-26T06:37:00.144 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-26T06:37:00.144 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-26T06:37:00.511 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.x 2023-10-26T06:37:00.511 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.x.service: Unit ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.x.service not loaded. 2023-10-26T06:37:00.716 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7.target.wants/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@.service. 2023-10-26T06:37:00.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:00 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3343227792' entity='client.admin' 2023-10-26T06:37:00.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:00 smithi089 systemd[1]: Starting Ceph mgr.x for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:37:01.518 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:01 smithi089 podman[95263]: 2023-10-26T06:37:01.518 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:01 smithi089 bash[95263]: 1fe63502cfdedbb9a8d8cf0b7a428f81509a14ab6ddf632abdbfb1473c9c71b2 2023-10-26T06:37:01.541 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-26T06:37:01.542 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2023-10-26T06:37:01.557 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-26T06:37:01.558 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2023-10-26T06:37:01.558 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2023-10-26T06:37:01.558 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to enable service . firewalld.service is not available 2023-10-26T06:37:01.575 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-26T06:37:01.575 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2023-10-26T06:37:01.591 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-26T06:37:01.591 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2023-10-26T06:37:01.592 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2023-10-26T06:37:01.592 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-26T06:37:01.592 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr to start... 2023-10-26T06:37:01.592 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr... 2023-10-26T06:37:01.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:01 smithi089 systemd[1]: Started Ceph mgr.x for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:37:01.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:01 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:01.650+0000 7f2b6d4e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T06:37:01.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:01 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:01.762+0000 7f2b6d4e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T06:37:02.282 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:01 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:01.888+0000 7f2b6d4e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T06:37:02.283 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:02 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:02.044+0000 7f2b6d4e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T06:37:02.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:02 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2400257857' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-26T06:37:02.540 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:02 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:02.280+0000 7f2b6d4e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T06:37:02.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:37:02.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:02.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "e22b2d1a-73c9-11ee-8db9-212e2dc638e7", 2023-10-26T06:37:02.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2023-10-26T06:37:02.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-26T06:37:02.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-26T06:37:02.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-26T06:37:02.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:02.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-26T06:37:02.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-26T06:37:02.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2023-10-26T06:37:02.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:02.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-26T06:37:02.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2023-10-26T06:37:02.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:02.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-26T06:37:02.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-26T06:37:02.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:02.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-26T06:37:02.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-26T06:37:02.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:02.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-26T06:37:02.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:02.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-26T06:37:02.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-26T06:37:02.681 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-26T06:37:02.681 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-26T06:37:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-26T06:37:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-26T06:37:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-26T06:37:02.682 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-26T06:37:02.683 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-26T06:37:02.683 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-26T06:37:02.683 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-26T06:37:02.683 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-26T06:37:02.683 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-26T06:37:02.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-26T06:37:02.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-26T06:37:02.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:02.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-26T06:37:02.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:02.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-26T06:37:02.685 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-26T06:37:02.685 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:02.685 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-26T06:37:02.685 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-26T06:37:02.685 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-26T06:37:02.686 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-26T06:37:02.686 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-26T06:37:02.686 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-26T06:37:02.686 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2023-10-26T06:37:02.686 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:02.686 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:02.687 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:02.687 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-26T06:37:02.687 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:02.687 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2023-10-26T06:36:45.902252+0000", 2023-10-26T06:37:02.687 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:02.688 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:02.688 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-26T06:37:02.688 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:02.688 INFO:teuthology.orchestra.run.smithi089.stdout:mgr not available, waiting (1/15)... 2023-10-26T06:37:03.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:02 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:02.765+0000 7f2b6d4e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T06:37:03.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:02 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:02.929+0000 7f2b6d4e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T06:37:03.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:03 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:03.045+0000 7f2b6d4e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T06:37:03.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:03 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:03.167+0000 7f2b6d4e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T06:37:04.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:03 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:03.705+0000 7f2b6d4e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T06:37:04.982 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:04 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:04.585+0000 7f2b6d4e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T06:37:04.983 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:04 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:04.691+0000 7f2b6d4e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T06:37:05.702 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/284177699' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-26T06:37:05.754 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:37:05.761 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:05.761 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "e22b2d1a-73c9-11ee-8db9-212e2dc638e7", 2023-10-26T06:37:05.761 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2023-10-26T06:37:05.762 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-26T06:37:05.762 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-26T06:37:05.762 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-26T06:37:05.763 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:05.763 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-26T06:37:05.763 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-26T06:37:05.763 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2023-10-26T06:37:05.764 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:05.764 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-26T06:37:05.764 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2023-10-26T06:37:05.764 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:05.765 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-26T06:37:05.765 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-26T06:37:05.765 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:05.765 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-26T06:37:05.765 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-26T06:37:05.765 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:05.766 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-26T06:37:05.766 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:05.766 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-26T06:37:05.766 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-26T06:37:05.766 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-26T06:37:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-26T06:37:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-26T06:37:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-26T06:37:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:05.767 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-26T06:37:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-26T06:37:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-26T06:37:05.768 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-26T06:37:05.769 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-26T06:37:05.769 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-26T06:37:05.769 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-26T06:37:05.769 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-26T06:37:05.769 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-26T06:37:05.770 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:05.770 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-26T06:37:05.770 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:05.770 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-26T06:37:05.771 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-26T06:37:05.771 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:05.771 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-26T06:37:05.771 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-26T06:37:05.771 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-26T06:37:05.772 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-26T06:37:05.772 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-26T06:37:05.772 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-26T06:37:05.772 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2023-10-26T06:37:05.773 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:05.773 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:05.773 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:05.773 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-26T06:37:05.773 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:05.774 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2023-10-26T06:36:45.902252+0000", 2023-10-26T06:37:05.774 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:05.774 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:05.774 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-26T06:37:05.774 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:05.775 INFO:teuthology.orchestra.run.smithi089.stdout:mgr not available, waiting (2/15)... 2023-10-26T06:37:06.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:05 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:05.700+0000 7f2b6d4e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:37:06.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:05 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:05.823+0000 7f2b6d4e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T06:37:06.416 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:06 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:06.051+0000 7f2b6d4e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:37:06.695 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:06 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:06.414+0000 7f2b6d4e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T06:37:07.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:06 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:06.692+0000 7f2b6d4e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T06:37:07.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:06 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:06.884+0000 7f2b6d4e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T06:37:07.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:07 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:07.230+0000 7f2b6d4e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T06:37:08.372 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:08 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:08.186+0000 7f2b6d4e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T06:37:08.636 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:08 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3683720713' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-26T06:37:08.636 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:08 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:08.369+0000 7f2b6d4e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T06:37:08.844 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:37:08.845 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:08.845 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "e22b2d1a-73c9-11ee-8db9-212e2dc638e7", 2023-10-26T06:37:08.845 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2023-10-26T06:37:08.845 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-26T06:37:08.846 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-26T06:37:08.846 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-26T06:37:08.846 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:08.846 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-26T06:37:08.846 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-26T06:37:08.847 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2023-10-26T06:37:08.847 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:08.847 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-26T06:37:08.847 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2023-10-26T06:37:08.847 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:08.848 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-10-26T06:37:08.848 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-26T06:37:08.848 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:08.848 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-26T06:37:08.848 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-26T06:37:08.849 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:08.849 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-26T06:37:08.849 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:08.849 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-26T06:37:08.849 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-26T06:37:08.853 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-26T06:37:08.853 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-26T06:37:08.853 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-26T06:37:08.854 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-26T06:37:08.854 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:08.854 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-26T06:37:08.854 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-26T06:37:08.854 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-26T06:37:08.855 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-26T06:37:08.855 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-26T06:37:08.855 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-26T06:37:08.855 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-26T06:37:08.856 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-26T06:37:08.856 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-26T06:37:08.856 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:08.856 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-26T06:37:08.856 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:08.857 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-26T06:37:08.857 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-26T06:37:08.857 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:08.857 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-26T06:37:08.858 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-26T06:37:08.858 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-26T06:37:08.858 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-26T06:37:08.858 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-26T06:37:08.858 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-26T06:37:08.859 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2023-10-26T06:37:08.859 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:08.859 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:08.859 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:08.859 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-26T06:37:08.860 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:08.860 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2023-10-26T06:36:45.902252+0000", 2023-10-26T06:37:08.860 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:08.860 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:08.860 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-26T06:37:08.861 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:08.861 INFO:teuthology.orchestra.run.smithi089.stdout:mgr not available, waiting (3/15)... 2023-10-26T06:37:09.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:08 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:08.634+0000 7f2b6d4e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T06:37:09.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:08 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:08.784+0000 7f2b6d4e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T06:37:09.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:09 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:09.066+0000 7f2b6d4e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T06:37:10.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:10.127+0000 7f2b6d4e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T06:37:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: Activating manager daemon x 2023-10-26T06:37:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: mgrmap e2: x(active, starting, since 0.00464217s) 2023-10-26T06:37:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-26T06:37:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-26T06:37:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-26T06:37:10.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:37:10.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-26T06:37:10.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: Manager daemon x is now available 2023-10-26T06:37:10.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-26T06:37:10.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-26T06:37:10.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' 2023-10-26T06:37:10.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' 2023-10-26T06:37:10.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:10 smithi089 ceph-mon[95031]: from='mgr.14100 172.21.15.89:0/468750495' entity='mgr.x' 2023-10-26T06:37:12.130 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:37:12.131 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:12.131 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "e22b2d1a-73c9-11ee-8db9-212e2dc638e7", 2023-10-26T06:37:12.131 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2023-10-26T06:37:12.131 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-26T06:37:12.132 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-26T06:37:12.132 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-26T06:37:12.132 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:12.132 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-26T06:37:12.133 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-26T06:37:12.133 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2023-10-26T06:37:12.133 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:12.133 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-26T06:37:12.133 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2023-10-26T06:37:12.133 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:12.134 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-10-26T06:37:12.134 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-26T06:37:12.134 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:12.134 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-26T06:37:12.135 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-26T06:37:12.135 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:12.135 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-26T06:37:12.135 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:12.135 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-26T06:37:12.135 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-26T06:37:12.136 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-26T06:37:12.140 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-26T06:37:12.140 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-26T06:37:12.140 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-26T06:37:12.141 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:12.141 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-26T06:37:12.141 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-26T06:37:12.141 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-26T06:37:12.141 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-26T06:37:12.142 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-26T06:37:12.142 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-26T06:37:12.142 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-26T06:37:12.142 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-26T06:37:12.142 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-26T06:37:12.143 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:12.143 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-26T06:37:12.143 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:12.143 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-26T06:37:12.143 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-26T06:37:12.144 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:12.144 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-26T06:37:12.144 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-26T06:37:12.144 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-26T06:37:12.144 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-26T06:37:12.144 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-26T06:37:12.145 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-26T06:37:12.145 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2023-10-26T06:37:12.145 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2023-10-26T06:37:12.145 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:12.145 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:12.146 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-26T06:37:12.146 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-26T06:37:12.146 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2023-10-26T06:36:45.902252+0000", 2023-10-26T06:37:12.146 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-26T06:37:12.146 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2023-10-26T06:37:12.147 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-26T06:37:12.147 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:12.147 INFO:teuthology.orchestra.run.smithi089.stdout:mgr is available 2023-10-26T06:37:12.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:12 smithi089 ceph-mon[95031]: mgrmap e3: x(active, since 1.00978s) 2023-10-26T06:37:12.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:12 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1535975852' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-26T06:37:13.313 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:37:13.313 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [global] 2023-10-26T06:37:13.313 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout fsid = e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:37:13.314 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-26T06:37:13.314 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-26T06:37:13.314 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-26T06:37:13.314 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-26T06:37:13.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:37:13.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-26T06:37:13.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-26T06:37:13.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2023-10-26T06:37:13.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [osd] 2023-10-26T06:37:13.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2023-10-26T06:37:13.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-26T06:37:13.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-26T06:37:13.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-26T06:37:13.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-26T06:37:13.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-26T06:37:13.317 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling cephadm module... 2023-10-26T06:37:13.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:13 smithi089 ceph-mon[95031]: mgrmap e4: x(active, since 2s) 2023-10-26T06:37:13.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:13 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/841937321' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-26T06:37:15.268 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:14 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/453148125' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-26T06:37:15.269 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: ignoring --setuser ceph since I am not root 2023-10-26T06:37:15.269 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: ignoring --setgroup ceph since I am not root 2023-10-26T06:37:15.269 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:15.267+0000 7f0dca0d1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T06:37:15.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:15.372+0000 7f0dca0d1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T06:37:15.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:15.507+0000 7f0dca0d1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T06:37:15.992 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:15.634+0000 7f0dca0d1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T06:37:15.992 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:15.873+0000 7f0dca0d1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T06:37:16.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/453148125' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-26T06:37:16.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:15 smithi089 ceph-mon[95031]: mgrmap e5: x(active, since 4s) 2023-10-26T06:37:16.626 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:16.340+0000 7f0dca0d1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T06:37:16.626 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:16.507+0000 7f0dca0d1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T06:37:16.627 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:16.624+0000 7f0dca0d1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T06:37:16.683 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:16.683 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-26T06:37:16.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-26T06:37:16.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-26T06:37:16.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-26T06:37:16.684 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:16.684 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for the mgr to restart... 2023-10-26T06:37:16.685 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr epoch 5... 2023-10-26T06:37:16.992 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:16.769+0000 7f0dca0d1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T06:37:17.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:16 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3228169496' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-26T06:37:17.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:17 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:17.331+0000 7f0dca0d1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T06:37:18.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:18 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:18.165+0000 7f0dca0d1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T06:37:18.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:18 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:18.268+0000 7f0dca0d1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T06:37:19.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:19 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:19.216+0000 7f0dca0d1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:37:19.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:19 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:19.335+0000 7f0dca0d1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T06:37:19.925 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:19 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:19.559+0000 7f0dca0d1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:37:20.203 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:19 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:19.924+0000 7f0dca0d1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T06:37:20.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:20 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:20.202+0000 7f0dca0d1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T06:37:20.540 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:20 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:20.392+0000 7f0dca0d1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T06:37:21.041 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:20 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:20.734+0000 7f0dca0d1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T06:37:22.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:21 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:21.648+0000 7f0dca0d1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T06:37:22.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:21 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:21.818+0000 7f0dca0d1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T06:37:22.485 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:22 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:22.066+0000 7f0dca0d1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T06:37:22.486 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:22 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:22.211+0000 7f0dca0d1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T06:37:22.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:22 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:22.484+0000 7f0dca0d1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T06:37:23.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:23.535+0000 7f0dca0d1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T06:37:23.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: Active manager daemon x restarted 2023-10-26T06:37:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: Activating manager daemon x 2023-10-26T06:37:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: osdmap e2: 0 total, 0 up, 0 in 2023-10-26T06:37:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: mgrmap e6: x(active, starting, since 0.00402071s) 2023-10-26T06:37:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:37:23.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-26T06:37:23.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-26T06:37:23.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-26T06:37:23.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-26T06:37:23.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: Manager daemon x is now available 2023-10-26T06:37:23.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:23.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:23.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:23.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:23 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:24.986 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:24.986 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-26T06:37:24.987 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-26T06:37:24.987 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:24.987 INFO:teuthology.orchestra.run.smithi089.stdout:mgr epoch 5 is available 2023-10-26T06:37:24.987 INFO:teuthology.orchestra.run.smithi089.stdout:Setting orchestrator backend to cephadm... 2023-10-26T06:37:25.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:24 smithi089 ceph-mon[95031]: Found migration_current of "None". Setting to last migration. 2023-10-26T06:37:25.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:24 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:25.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:24 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-26T06:37:25.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:24 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-26T06:37:25.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:24 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:25.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:24 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:25.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:24 smithi089 ceph-mon[95031]: mgrmap e7: x(active, since 1.00649s) 2023-10-26T06:37:25.901 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:25 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:24] ENGINE Bus STARTING 2023-10-26T06:37:25.902 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:25 smithi089 ceph-mon[95031]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-26T06:37:25.902 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:25 smithi089 ceph-mon[95031]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-26T06:37:25.902 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:25 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:24] ENGINE Serving on https://172.21.15.89:7150 2023-10-26T06:37:25.903 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:25 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:24] ENGINE Serving on http://172.21.15.89:8765 2023-10-26T06:37:25.903 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:25 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:24] ENGINE Bus STARTED 2023-10-26T06:37:25.903 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:25 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:27.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:26 smithi089 ceph-mon[95031]: mgrmap e8: x(active, since 2s) 2023-10-26T06:37:27.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:26 smithi089 ceph-mon[95031]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:27.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:26 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:27.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:26 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:27.542 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-26T06:37:27.542 INFO:teuthology.orchestra.run.smithi089.stdout:Generating ssh key... 2023-10-26T06:37:27.846 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:27 smithi089 ceph-mon[95031]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:28.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: Generating public/private rsa key pair. 2023-10-26T06:37:28.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: Your identification has been saved in /tmp/tmp1cb6za9r/key. 2023-10-26T06:37:28.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: Your public key has been saved in /tmp/tmp1cb6za9r/key.pub. 2023-10-26T06:37:28.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: The key fingerprint is: 2023-10-26T06:37:28.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: SHA256:Np6JlKRyvLZtqN6BduMw0HU1zBgbLGdq406+C2kws5A ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:37:28.790 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: The key's randomart image is: 2023-10-26T06:37:28.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: +---[RSA 3072]----+ 2023-10-26T06:37:28.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: | .o=o | 2023-10-26T06:37:28.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: | . =+o. | 2023-10-26T06:37:28.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: | .=+ | 2023-10-26T06:37:28.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: | o o++ . | 2023-10-26T06:37:28.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: |E oo+.o S | 2023-10-26T06:37:28.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: |.* =oo + + | 2023-10-26T06:37:28.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: |. O+*.. + | 2023-10-26T06:37:28.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: | o O+=. | 2023-10-26T06:37:28.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: | .o.*+. | 2023-10-26T06:37:28.793 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: +----[SHA256]-----+ 2023-10-26T06:37:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:29 smithi089 ceph-mon[95031]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:29 smithi089 ceph-mon[95031]: Generating ssh key... 2023-10-26T06:37:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:29 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:29 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:30.096 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfaB2voh0JWV3iPLZeSPR0qI2+/hO9D7fOaq66rzrhnB6qdMBW7Y32gHPfhnQtKX4eqQPpcB9XiIkncEmgn0fN0cSdKAU15t0bims8bURtNariQmVFnI424UFQaescFvqJrmk9Q4rkQlnvLbT05KPRyRv5X8shlmGHLR6Av7S9arg/GabeE88uIfPz9MfJJVzPXphfViFbqS8nBLBbFnHYio+gsXR3fERiXsHiS7/SAfYRgpfhEbZocqvX5swuOmtZjg9IHwZaHt8QgedA8br9W3qXH5mvBBHudMZ0rylO85ZTj8ugwZ3CwLAkT0F5t404wpUix36booUcc6iducnevE6ddVK6epWTT8VK7dKslqGpji62B3FgtZGjlCdE9FfEtLaN/fpI/RgoKiSVOpZUSuS53OyJTP7d6fX2nYW2DfQaJr7e1X60Zb8+2mZzj6jn07hXZhUdAC/VIiOj4y4kTowmkLQrIcxsFPxBGyZM9pCXBuZriqViXkJ74aZdv8= ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:37:30.097 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-26T06:37:30.097 INFO:teuthology.orchestra.run.smithi089.stdout:Adding key to root@localhost authorized_keys... 2023-10-26T06:37:30.099 INFO:teuthology.orchestra.run.smithi089.stdout:Adding host smithi089... 2023-10-26T06:37:30.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:30 smithi089 ceph-mon[95031]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:31.591 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:31 smithi089 ceph-mon[95031]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "addr": "172.21.15.89", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:32.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:32 smithi089 ceph-mon[95031]: Deploying cephadm binary to smithi089 2023-10-26T06:37:34.583 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Added host 'smithi089' with addr '172.21.15.89' 2023-10-26T06:37:34.583 INFO:teuthology.orchestra.run.smithi089.stdout:Deploying unmanaged mon service... 2023-10-26T06:37:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:35 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:35 smithi089 ceph-mon[95031]: Added host smithi089 2023-10-26T06:37:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:35 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:36.000 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-26T06:37:36.000 INFO:teuthology.orchestra.run.smithi089.stdout:Deploying unmanaged mgr service... 2023-10-26T06:37:36.681 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:36 smithi089 ceph-mon[95031]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:36.681 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:36 smithi089 ceph-mon[95031]: Saving service mon spec with placement count:5 2023-10-26T06:37:36.681 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:36 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:37.378 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-26T06:37:38.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:37 smithi089 ceph-mon[95031]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:38.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:37 smithi089 ceph-mon[95031]: Saving service mgr spec with placement count:2 2023-10-26T06:37:38.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:37 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:38.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:37 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:38.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:37 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:39.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:39 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2493871718' entity='client.admin' 2023-10-26T06:37:40.040 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling the dashboard module... 2023-10-26T06:37:40.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:40 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3361369556' entity='client.admin' 2023-10-26T06:37:40.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:40 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:41.691 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:41 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1248314529' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-26T06:37:41.691 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:41 smithi089 ceph-mon[95031]: from='mgr.14120 172.21.15.89:0/722593452' entity='mgr.x' 2023-10-26T06:37:41.692 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:41 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: ignoring --setuser ceph since I am not root 2023-10-26T06:37:41.692 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:41 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: ignoring --setgroup ceph since I am not root 2023-10-26T06:37:41.692 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:41 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:41.689+0000 7f7e4121c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T06:37:42.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:41 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:41.795+0000 7f7e4121c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T06:37:42.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:41 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:41.937+0000 7f7e4121c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T06:37:42.387 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:42 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:42.064+0000 7f7e4121c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T06:37:42.387 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:42 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:42.304+0000 7f7e4121c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T06:37:42.692 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:42 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1248314529' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-26T06:37:42.692 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:42 smithi089 ceph-mon[95031]: mgrmap e9: x(active, since 17s) 2023-10-26T06:37:42.954 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:42 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:42.782+0000 7f7e4121c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T06:37:42.954 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:42 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:42.951+0000 7f7e4121c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T06:37:43.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:43.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-26T06:37:43.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-26T06:37:43.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-26T06:37:43.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-26T06:37:43.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:43.124 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for the mgr to restart... 2023-10-26T06:37:43.124 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr epoch 9... 2023-10-26T06:37:43.210 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:43 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:43.069+0000 7f7e4121c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T06:37:43.211 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:43 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:43.208+0000 7f7e4121c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T06:37:43.463 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:43 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3857998388' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-26T06:37:44.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:43 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:43.776+0000 7f7e4121c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T06:37:45.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:44 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:44.619+0000 7f7e4121c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T06:37:45.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:44 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:44.725+0000 7f7e4121c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T06:37:46.024 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:45 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:45.674+0000 7f7e4121c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:37:46.024 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:45 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:45.795+0000 7f7e4121c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T06:37:46.667 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:46 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:46.021+0000 7f7e4121c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:37:46.667 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:46 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:46.386+0000 7f7e4121c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T06:37:46.667 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:46 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:46.664+0000 7f7e4121c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T06:37:47.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:46 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:46.857+0000 7f7e4121c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T06:37:47.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:47 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:47.202+0000 7f7e4121c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T06:37:48.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:48 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:48.123+0000 7f7e4121c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T06:37:48.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:48 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:48.297+0000 7f7e4121c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T06:37:48.975 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:48 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:48.548+0000 7f7e4121c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T06:37:48.976 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:48 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:48.696+0000 7f7e4121c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T06:37:49.289 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:48 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:48.973+0000 7f7e4121c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T06:37:50.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:37:50.040+0000 7f7e4121c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T06:37:50.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: Active manager daemon x restarted 2023-10-26T06:37:50.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: Activating manager daemon x 2023-10-26T06:37:50.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: osdmap e3: 0 total, 0 up, 0 in 2023-10-26T06:37:50.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: mgrmap e10: x(active, starting, since 0.00597466s) 2023-10-26T06:37:50.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:37:50.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-26T06:37:50.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-26T06:37:50.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-26T06:37:50.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-26T06:37:50.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: Manager daemon x is now available 2023-10-26T06:37:50.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:50.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:50 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:37:51.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2023-10-26T06:37:51.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-26T06:37:51.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-26T06:37:51.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2023-10-26T06:37:51.518 INFO:teuthology.orchestra.run.smithi089.stdout:mgr epoch 9 is available 2023-10-26T06:37:51.518 INFO:teuthology.orchestra.run.smithi089.stdout:Generating a dashboard self-signed certificate... 2023-10-26T06:37:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-26T06:37:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-26T06:37:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:51 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:50] ENGINE Bus STARTING 2023-10-26T06:37:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:51 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:50] ENGINE Serving on https://172.21.15.89:7150 2023-10-26T06:37:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:51 smithi089 ceph-mon[95031]: mgrmap e11: x(active, since 1.00747s) 2023-10-26T06:37:52.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:52 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:50] ENGINE Serving on http://172.21.15.89:8765 2023-10-26T06:37:52.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:52 smithi089 ceph-mon[95031]: [26/Oct/2023:06:37:50] ENGINE Bus STARTED 2023-10-26T06:37:52.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:52 smithi089 ceph-mon[95031]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-26T06:37:52.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:52 smithi089 ceph-mon[95031]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-26T06:37:53.095 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-26T06:37:53.095 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial admin user... 2023-10-26T06:37:53.358 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:53 smithi089 ceph-mon[95031]: mgrmap e12: x(active, since 2s) 2023-10-26T06:37:53.359 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:53 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:37:53.359 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:53 smithi089 ceph-mon[95031]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:53.359 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:53 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:37:53.359 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:53 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:37:54.573 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$yDFMN1YJHRKGb17d1piIE.ZURGBXqkZ566Rf2/ehnE.U.aKZ0SzAK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1698302274, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-26T06:37:54.574 INFO:teuthology.orchestra.run.smithi089.stdout:Fetching dashboard port number... 2023-10-26T06:37:54.910 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:54 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:37:54.911 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:54 smithi089 ceph-mon[95031]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:37:54.911 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:54 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:37:56.118 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:55 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1842801926' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-26T06:37:56.152 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 8443 2023-10-26T06:37:56.171 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-26T06:37:56.172 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2023-10-26T06:37:56.187 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-26T06:37:56.187 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2023-10-26T06:37:56.187 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2023-10-26T06:37:56.187 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-26T06:37:56.189 INFO:teuthology.orchestra.run.smithi089.stdout:Ceph Dashboard is now available at: 2023-10-26T06:37:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: URL: https://smithi089.front.sepia.ceph.com:8443/ 2023-10-26T06:37:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: User: admin 2023-10-26T06:37:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: Password: fn1w9aq8va 2023-10-26T06:37:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:56.191 INFO:teuthology.orchestra.run.smithi089.stdout:Saving cluster configuration to /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config directory 2023-10-26T06:37:56.191 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling autotune for osd_memory_target 2023-10-26T06:37:58.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:57 smithi089 ceph-mon[95031]: mgrmap e13: x(active, since 6s) 2023-10-26T06:37:58.846 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-26T06:37:58.847 INFO:teuthology.orchestra.run.smithi089.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-26T06:37:58.847 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.847 INFO:teuthology.orchestra.run.smithi089.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-26T06:37:58.847 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.848 INFO:teuthology.orchestra.run.smithi089.stdout:Or, if you are only running a single cluster on this host: 2023-10-26T06:37:58.848 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.848 INFO:teuthology.orchestra.run.smithi089.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-26T06:37:58.848 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.848 INFO:teuthology.orchestra.run.smithi089.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-26T06:37:58.848 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.849 INFO:teuthology.orchestra.run.smithi089.stdout: ceph telemetry on 2023-10-26T06:37:58.849 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.849 INFO:teuthology.orchestra.run.smithi089.stdout:For more information see: 2023-10-26T06:37:58.849 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.849 INFO:teuthology.orchestra.run.smithi089.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-26T06:37:58.850 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:37:58.850 INFO:teuthology.orchestra.run.smithi089.stdout:Bootstrap complete. 2023-10-26T06:37:58.907 INFO:tasks.cephadm:Fetching config... 2023-10-26T06:37:58.907 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:37:58.908 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-26T06:37:58.951 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-26T06:37:58.951 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:37:58.951 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-26T06:37:59.034 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-26T06:37:59.035 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:37:59.035 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-26T06:37:59.146 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-26T06:37:59.146 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:37:59.146 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-26T06:37:59.229 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-26T06:37:59.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfaB2voh0JWV3iPLZeSPR0qI2+/hO9D7fOaq66rzrhnB6qdMBW7Y32gHPfhnQtKX4eqQPpcB9XiIkncEmgn0fN0cSdKAU15t0bims8bURtNariQmVFnI424UFQaescFvqJrmk9Q4rkQlnvLbT05KPRyRv5X8shlmGHLR6Av7S9arg/GabeE88uIfPz9MfJJVzPXphfViFbqS8nBLBbFnHYio+gsXR3fERiXsHiS7/SAfYRgpfhEbZocqvX5swuOmtZjg9IHwZaHt8QgedA8br9W3qXH5mvBBHudMZ0rylO85ZTj8ugwZ3CwLAkT0F5t404wpUix36booUcc6iducnevE6ddVK6epWTT8VK7dKslqGpji62B3FgtZGjlCdE9FfEtLaN/fpI/RgoKiSVOpZUSuS53OyJTP7d6fX2nYW2DfQaJr7e1X60Zb8+2mZzj6jn07hXZhUdAC/VIiOj4y4kTowmkLQrIcxsFPxBGyZM9pCXBuZriqViXkJ74aZdv8= ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-26T06:37:59.387 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:37:59 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2576808894' entity='client.admin' 2023-10-26T06:37:59.396 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfaB2voh0JWV3iPLZeSPR0qI2+/hO9D7fOaq66rzrhnB6qdMBW7Y32gHPfhnQtKX4eqQPpcB9XiIkncEmgn0fN0cSdKAU15t0bims8bURtNariQmVFnI424UFQaescFvqJrmk9Q4rkQlnvLbT05KPRyRv5X8shlmGHLR6Av7S9arg/GabeE88uIfPz9MfJJVzPXphfViFbqS8nBLBbFnHYio+gsXR3fERiXsHiS7/SAfYRgpfhEbZocqvX5swuOmtZjg9IHwZaHt8QgedA8br9W3qXH5mvBBHudMZ0rylO85ZTj8ugwZ3CwLAkT0F5t404wpUix36booUcc6iducnevE6ddVK6epWTT8VK7dKslqGpji62B3FgtZGjlCdE9FfEtLaN/fpI/RgoKiSVOpZUSuS53OyJTP7d6fX2nYW2DfQaJr7e1X60Zb8+2mZzj6jn07hXZhUdAC/VIiOj4y4kTowmkLQrIcxsFPxBGyZM9pCXBuZriqViXkJ74aZdv8= ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:37:59.430 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfaB2voh0JWV3iPLZeSPR0qI2+/hO9D7fOaq66rzrhnB6qdMBW7Y32gHPfhnQtKX4eqQPpcB9XiIkncEmgn0fN0cSdKAU15t0bims8bURtNariQmVFnI424UFQaescFvqJrmk9Q4rkQlnvLbT05KPRyRv5X8shlmGHLR6Av7S9arg/GabeE88uIfPz9MfJJVzPXphfViFbqS8nBLBbFnHYio+gsXR3fERiXsHiS7/SAfYRgpfhEbZocqvX5swuOmtZjg9IHwZaHt8QgedA8br9W3qXH5mvBBHudMZ0rylO85ZTj8ugwZ3CwLAkT0F5t404wpUix36booUcc6iducnevE6ddVK6epWTT8VK7dKslqGpji62B3FgtZGjlCdE9FfEtLaN/fpI/RgoKiSVOpZUSuS53OyJTP7d6fX2nYW2DfQaJr7e1X60Zb8+2mZzj6jn07hXZhUdAC/VIiOj4y4kTowmkLQrIcxsFPxBGyZM9pCXBuZriqViXkJ74aZdv8= ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-26T06:37:59.475 INFO:teuthology.orchestra.run.smithi135.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfaB2voh0JWV3iPLZeSPR0qI2+/hO9D7fOaq66rzrhnB6qdMBW7Y32gHPfhnQtKX4eqQPpcB9XiIkncEmgn0fN0cSdKAU15t0bims8bURtNariQmVFnI424UFQaescFvqJrmk9Q4rkQlnvLbT05KPRyRv5X8shlmGHLR6Av7S9arg/GabeE88uIfPz9MfJJVzPXphfViFbqS8nBLBbFnHYio+gsXR3fERiXsHiS7/SAfYRgpfhEbZocqvX5swuOmtZjg9IHwZaHt8QgedA8br9W3qXH5mvBBHudMZ0rylO85ZTj8ugwZ3CwLAkT0F5t404wpUix36booUcc6iducnevE6ddVK6epWTT8VK7dKslqGpji62B3FgtZGjlCdE9FfEtLaN/fpI/RgoKiSVOpZUSuS53OyJTP7d6fX2nYW2DfQaJr7e1X60Zb8+2mZzj6jn07hXZhUdAC/VIiOj4y4kTowmkLQrIcxsFPxBGyZM9pCXBuZriqViXkJ74aZdv8= ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:37:59.487 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfaB2voh0JWV3iPLZeSPR0qI2+/hO9D7fOaq66rzrhnB6qdMBW7Y32gHPfhnQtKX4eqQPpcB9XiIkncEmgn0fN0cSdKAU15t0bims8bURtNariQmVFnI424UFQaescFvqJrmk9Q4rkQlnvLbT05KPRyRv5X8shlmGHLR6Av7S9arg/GabeE88uIfPz9MfJJVzPXphfViFbqS8nBLBbFnHYio+gsXR3fERiXsHiS7/SAfYRgpfhEbZocqvX5swuOmtZjg9IHwZaHt8QgedA8br9W3qXH5mvBBHudMZ0rylO85ZTj8ugwZ3CwLAkT0F5t404wpUix36booUcc6iducnevE6ddVK6epWTT8VK7dKslqGpji62B3FgtZGjlCdE9FfEtLaN/fpI/RgoKiSVOpZUSuS53OyJTP7d6fX2nYW2DfQaJr7e1X60Zb8+2mZzj6jn07hXZhUdAC/VIiOj4y4kTowmkLQrIcxsFPxBGyZM9pCXBuZriqViXkJ74aZdv8= ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-26T06:37:59.532 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnfaB2voh0JWV3iPLZeSPR0qI2+/hO9D7fOaq66rzrhnB6qdMBW7Y32gHPfhnQtKX4eqQPpcB9XiIkncEmgn0fN0cSdKAU15t0bims8bURtNariQmVFnI424UFQaescFvqJrmk9Q4rkQlnvLbT05KPRyRv5X8shlmGHLR6Av7S9arg/GabeE88uIfPz9MfJJVzPXphfViFbqS8nBLBbFnHYio+gsXR3fERiXsHiS7/SAfYRgpfhEbZocqvX5swuOmtZjg9IHwZaHt8QgedA8br9W3qXH5mvBBHudMZ0rylO85ZTj8ugwZ3CwLAkT0F5t404wpUix36booUcc6iducnevE6ddVK6epWTT8VK7dKslqGpji62B3FgtZGjlCdE9FfEtLaN/fpI/RgoKiSVOpZUSuS53OyJTP7d6fX2nYW2DfQaJr7e1X60Zb8+2mZzj6jn07hXZhUdAC/VIiOj4y4kTowmkLQrIcxsFPxBGyZM9pCXBuZriqViXkJ74aZdv8= ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:37:59.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-26T06:38:00.293 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:02.775 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-26T06:38:02.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-26T06:38:03.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:03.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:03.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:38:03.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:03.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:03.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:03.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1134026810' entity='client.admin' 2023-10-26T06:38:03.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:03.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:02 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:03.511 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:05.792 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi135 2023-10-26T06:38:05.792 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:38:05.793 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.conf 2023-10-26T06:38:05.814 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:38:05.814 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:05.873 INFO:tasks.cephadm:Adding host smithi135 to orchestrator... 2023-10-26T06:38:05.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch host add smithi135 2023-10-26T06:38:06.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:06 smithi089 ceph-mon[95031]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:06 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:06 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:06 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:06 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:06.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:06 smithi089 ceph-mon[95031]: Updating smithi089:/etc/ceph/ceph.conf 2023-10-26T06:38:06.637 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:07.508 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:07 smithi089 ceph-mon[95031]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:07.508 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:07 smithi089 ceph-mon[95031]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:08.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:08 smithi089 ceph-mon[95031]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:38:08.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:08 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:08.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:08 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:08.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:08 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:09.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:09 smithi089 ceph-mon[95031]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:10.781 INFO:teuthology.orchestra.run.smithi089.stdout:Added host 'smithi135' with addr '172.21.15.135' 2023-10-26T06:38:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:10 smithi089 ceph-mon[95031]: Deploying cephadm binary to smithi135 2023-10-26T06:38:11.327 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-26T06:38:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:11 smithi089 ceph-mon[95031]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:11 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:11 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:11 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:12.061 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:12.670 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:12 smithi089 ceph-mon[95031]: Added host smithi135 2023-10-26T06:38:12.670 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:12 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:13 smithi089 ceph-mon[95031]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:13 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:38:13.887 INFO:teuthology.orchestra.run.smithi089.stdout:[{"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}] 2023-10-26T06:38:14.338 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2023-10-26T06:38:14.339 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:38:14.339 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2023-10-26T06:38:14.360 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:38:14.360 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:14.419 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2023-10-26T06:38:14.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch host add smithi192 2023-10-26T06:38:15.153 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:15.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:15 smithi089 ceph-mon[95031]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:38:15.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:15 smithi089 ceph-mon[95031]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:17.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:17.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:17.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:17.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:17.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:17.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:38:17.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:17.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:17.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:38:17.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:17.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:17.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:17.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:17 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:18.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:18 smithi089 ceph-mon[95031]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:18.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:18 smithi089 ceph-mon[95031]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:38:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:18 smithi089 ceph-mon[95031]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:19 smithi089 ceph-mon[95031]: Deploying cephadm binary to smithi192 2023-10-26T06:38:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:19 smithi089 ceph-mon[95031]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:19.592 INFO:teuthology.orchestra.run.smithi089.stdout:Added host 'smithi192' with addr '172.21.15.192' 2023-10-26T06:38:20.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-26T06:38:20.773 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:21.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:20 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:21.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:20 smithi089 ceph-mon[95031]: Added host smithi192 2023-10-26T06:38:21.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:20 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:21.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:20 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:21.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:20 smithi089 ceph-mon[95031]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:22.003 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:21 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:22.462 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:38:22.462 INFO:teuthology.orchestra.run.smithi089.stdout:[{"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2023-10-26T06:38:22.955 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-26T06:38:22.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd crush tunables default 2023-10-26T06:38:23.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:22 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:23.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:22 smithi089 ceph-mon[95031]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:23.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:22 smithi089 ceph-mon[95031]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:38:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:25.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:25 smithi089 ceph-mon[95031]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:26.099 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2023-10-26T06:38:26.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:26.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:26.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:26.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:26.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:38:26.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:26.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:26.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:38:26.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:26.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/918140397' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-26T06:38:26.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:26.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:26.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:26 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:26.665 INFO:tasks.cephadm:Adding mon.a on smithi089 2023-10-26T06:38:26.666 INFO:tasks.cephadm:Adding mon.b on smithi135 2023-10-26T06:38:26.666 INFO:tasks.cephadm:Adding mon.c on smithi192 2023-10-26T06:38:26.666 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch apply mon '3;smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c' 2023-10-26T06:38:27.248 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:27.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:27 smithi089 ceph-mon[95031]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:27.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:27 smithi089 ceph-mon[95031]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:38:27.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:27 smithi089 ceph-mon[95031]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:27.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:27 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/918140397' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-26T06:38:27.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:27 smithi089 ceph-mon[95031]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:28.695 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2023-10-26T06:38:29.110 DEBUG:teuthology.orchestra.run.smithi135:mon.b> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.b.service 2023-10-26T06:38:29.113 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.c.service 2023-10-26T06:38:29.115 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-26T06:38:29.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json 2023-10-26T06:38:29.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:38:29.148 INFO:journalctl@ceph.mon.b.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:38:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:29.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:29.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:29 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:30.006 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:30.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:30 smithi089 ceph-mon[95031]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:30.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:30 smithi089 ceph-mon[95031]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c;count:3 2023-10-26T06:38:30.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:30 smithi089 ceph-mon[95031]: Deploying daemon mon.c on smithi192 2023-10-26T06:38:31.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:31 smithi089 ceph-mon[95031]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:31.985 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:38:31.985 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","modified":"2023-10-26T06:36:42.499062Z","created":"2023-10-26T06:36:42.499062Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-26T06:38:31.987 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2023-10-26T06:38:32.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 podman[131596]: 2023-10-26T06:38:32.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:32 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/1784001494' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: set uid:gid to 167:167 (ceph:ceph) 2023-10-26T06:38:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev), process ceph-mon, pid 7 2023-10-26T06:38:32.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pidfile_write: ignore empty --pid-file 2023-10-26T06:38:32.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: load: jerasure load: lrc 2023-10-26T06:38:32.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: RocksDB version: 7.9.2 2023-10-26T06:38:32.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Git sha 0 2023-10-26T06:38:32.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Compile date 2023-10-24 16:36:40 2023-10-26T06:38:32.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: DB SUMMARY 2023-10-26T06:38:32.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: DB Session ID: WRXUO8QBMWC3QTZLZVYV 2023-10-26T06:38:32.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: CURRENT file: CURRENT 2023-10-26T06:38:32.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: IDENTITY file: IDENTITY 2023-10-26T06:38:32.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-26T06:38:32.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-10-26T06:38:32.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-10-26T06:38:32.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.error_if_exists: 0 2023-10-26T06:38:32.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.create_if_missing: 0 2023-10-26T06:38:32.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.paranoid_checks: 1 2023-10-26T06:38:32.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-26T06:38:32.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-26T06:38:32.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-26T06:38:32.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.env: 0x55edac2cdda0 2023-10-26T06:38:32.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.fs: PosixFileSystem 2023-10-26T06:38:32.901 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.info_log: 0x55edacefd660 2023-10-26T06:38:32.901 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_file_opening_threads: 16 2023-10-26T06:38:32.901 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.statistics: (nil) 2023-10-26T06:38:32.901 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.use_fsync: 0 2023-10-26T06:38:32.901 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_log_file_size: 0 2023-10-26T06:38:32.902 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-26T06:38:32.902 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-26T06:38:32.902 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.keep_log_file_num: 1000 2023-10-26T06:38:32.902 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.recycle_log_file_num: 0 2023-10-26T06:38:32.902 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.allow_fallocate: 1 2023-10-26T06:38:32.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.allow_mmap_reads: 0 2023-10-26T06:38:32.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.allow_mmap_writes: 0 2023-10-26T06:38:32.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.use_direct_reads: 0 2023-10-26T06:38:32.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-26T06:38:32.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.create_missing_column_families: 0 2023-10-26T06:38:32.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.db_log_dir: 2023-10-26T06:38:32.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.wal_dir: 2023-10-26T06:38:32.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-26T06:38:32.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-26T06:38:32.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-26T06:38:32.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-26T06:38:32.905 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-26T06:38:32.905 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-26T06:38:32.906 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.advise_random_on_open: 1 2023-10-26T06:38:32.906 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.db_write_buffer_size: 0 2023-10-26T06:38:32.906 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.write_buffer_manager: 0x55edaceb5360 2023-10-26T06:38:32.907 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-26T06:38:32.907 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-26T06:38:32.907 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-26T06:38:32.908 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.rate_limiter: (nil) 2023-10-26T06:38:32.908 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-26T06:38:32.908 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.wal_recovery_mode: 2 2023-10-26T06:38:32.909 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.enable_thread_tracking: 0 2023-10-26T06:38:32.909 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.enable_pipelined_write: 0 2023-10-26T06:38:32.909 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.unordered_write: 0 2023-10-26T06:38:32.909 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-26T06:38:32.909 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-26T06:38:32.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-26T06:38:32.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-26T06:38:32.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.row_cache: None 2023-10-26T06:38:32.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.wal_filter: None 2023-10-26T06:38:32.910 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-26T06:38:32.911 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.allow_ingest_behind: 0 2023-10-26T06:38:32.911 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.two_write_queues: 0 2023-10-26T06:38:32.911 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.manual_wal_flush: 0 2023-10-26T06:38:32.911 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.wal_compression: 0 2023-10-26T06:38:32.911 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.atomic_flush: 0 2023-10-26T06:38:32.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-26T06:38:32.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-26T06:38:32.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-26T06:38:32.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.log_readahead_size: 0 2023-10-26T06:38:32.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-26T06:38:32.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.best_efforts_recovery: 0 2023-10-26T06:38:32.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 bash[131596]: b12c0617d7321be4e77a9534cc48c5ca9c677d872a021a84e4790ef32778c32b 2023-10-26T06:38:32.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-26T06:38:32.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-26T06:38:32.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.allow_data_in_errors: 0 2023-10-26T06:38:32.915 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.db_host_id: __hostname__ 2023-10-26T06:38:32.915 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-26T06:38:32.916 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_background_jobs: 2 2023-10-26T06:38:32.916 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_background_compactions: -1 2023-10-26T06:38:32.916 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_subcompactions: 1 2023-10-26T06:38:32.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-26T06:38:32.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-26T06:38:32.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-26T06:38:32.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_total_wal_size: 0 2023-10-26T06:38:32.917 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-26T06:38:32.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-26T06:38:32.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-26T06:38:32.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-26T06:38:32.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_open_files: -1 2023-10-26T06:38:32.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bytes_per_sync: 0 2023-10-26T06:38:32.919 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-26T06:38:32.919 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-26T06:38:32.919 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_readahead_size: 0 2023-10-26T06:38:32.919 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_background_flushes: -1 2023-10-26T06:38:32.920 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Compression algorithms supported: 2023-10-26T06:38:32.920 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-26T06:38:32.920 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kZSTD supported: 0 2023-10-26T06:38:32.920 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kXpressCompression supported: 0 2023-10-26T06:38:32.920 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kLZ4HCCompression supported: 1 2023-10-26T06:38:32.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kZlibCompression supported: 1 2023-10-26T06:38:32.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kSnappyCompression supported: 1 2023-10-26T06:38:32.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kLZ4Compression supported: 1 2023-10-26T06:38:32.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: kBZip2Compression supported: 0 2023-10-26T06:38:32.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-26T06:38:32.922 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-26T06:38:32.922 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-10-26T06:38:32.922 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-26T06:38:32.922 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-26T06:38:32.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.merge_operator: 2023-10-26T06:38:32.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_filter: None 2023-10-26T06:38:32.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_filter_factory: None 2023-10-26T06:38:32.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.sst_partitioner_factory: None 2023-10-26T06:38:32.924 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-26T06:38:32.924 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-26T06:38:32.924 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55edacefd760) 2023-10-26T06:38:32.924 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks: 1 2023-10-26T06:38:32.924 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-26T06:38:32.925 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-26T06:38:32.925 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_top_level_index_and_filter: 1 2023-10-26T06:38:32.925 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_type: 0 2023-10-26T06:38:32.925 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_index_type: 0 2023-10-26T06:38:32.925 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_shortening: 1 2023-10-26T06:38:32.926 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-26T06:38:32.926 INFO:journalctl@ceph.mon.c.smithi192.stdout: checksum: 4 2023-10-26T06:38:32.926 INFO:journalctl@ceph.mon.c.smithi192.stdout: no_block_cache: 0 2023-10-26T06:38:32.926 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache: 0x55edacf531f0 2023-10-26T06:38:32.926 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_name: BinnedLRUCache 2023-10-26T06:38:32.927 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_options: 2023-10-26T06:38:32.927 INFO:journalctl@ceph.mon.c.smithi192.stdout: capacity : 536870912 2023-10-26T06:38:32.927 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_shard_bits : 4 2023-10-26T06:38:32.927 INFO:journalctl@ceph.mon.c.smithi192.stdout: strict_capacity_limit : 0 2023-10-26T06:38:32.927 INFO:journalctl@ceph.mon.c.smithi192.stdout: high_pri_pool_ratio: 0.000 2023-10-26T06:38:32.928 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_compressed: (nil) 2023-10-26T06:38:32.928 INFO:journalctl@ceph.mon.c.smithi192.stdout: persistent_cache: (nil) 2023-10-26T06:38:32.928 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size: 4096 2023-10-26T06:38:32.928 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size_deviation: 10 2023-10-26T06:38:32.928 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_restart_interval: 16 2023-10-26T06:38:32.928 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_block_restart_interval: 1 2023-10-26T06:38:32.929 INFO:journalctl@ceph.mon.c.smithi192.stdout: metadata_block_size: 4096 2023-10-26T06:38:32.929 INFO:journalctl@ceph.mon.c.smithi192.stdout: partition_filters: 0 2023-10-26T06:38:32.929 INFO:journalctl@ceph.mon.c.smithi192.stdout: use_delta_encoding: 1 2023-10-26T06:38:32.929 INFO:journalctl@ceph.mon.c.smithi192.stdout: filter_policy: bloomfilter 2023-10-26T06:38:32.929 INFO:journalctl@ceph.mon.c.smithi192.stdout: whole_key_filtering: 1 2023-10-26T06:38:32.930 INFO:journalctl@ceph.mon.c.smithi192.stdout: verify_compression: 0 2023-10-26T06:38:32.930 INFO:journalctl@ceph.mon.c.smithi192.stdout: read_amp_bytes_per_bit: 0 2023-10-26T06:38:32.930 INFO:journalctl@ceph.mon.c.smithi192.stdout: format_version: 5 2023-10-26T06:38:32.930 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_index_compression: 1 2023-10-26T06:38:32.930 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_align: 0 2023-10-26T06:38:32.931 INFO:journalctl@ceph.mon.c.smithi192.stdout: max_auto_readahead_size: 262144 2023-10-26T06:38:32.931 INFO:journalctl@ceph.mon.c.smithi192.stdout: prepopulate_block_cache: 0 2023-10-26T06:38:32.931 INFO:journalctl@ceph.mon.c.smithi192.stdout: initial_auto_readahead_size: 8192 2023-10-26T06:38:32.931 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2023-10-26T06:38:32.931 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.write_buffer_size: 33554432 2023-10-26T06:38:32.931 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_write_buffer_number: 2 2023-10-26T06:38:32.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression: NoCompression 2023-10-26T06:38:32.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression: Disabled 2023-10-26T06:38:32.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.prefix_extractor: nullptr 2023-10-26T06:38:32.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-26T06:38:32.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.num_levels: 7 2023-10-26T06:38:32.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-26T06:38:32.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-26T06:38:32.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-26T06:38:32.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-26T06:38:32.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-26T06:38:32.934 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-26T06:38:32.934 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-26T06:38:32.934 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-26T06:38:32.934 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-26T06:38:32.934 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-26T06:38:32.935 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-26T06:38:32.935 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-26T06:38:32.935 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-26T06:38:32.935 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.level: 32767 2023-10-26T06:38:32.935 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.strategy: 0 2023-10-26T06:38:32.936 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-26T06:38:32.936 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-26T06:38:32.936 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-26T06:38:32.936 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-26T06:38:32.936 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.enabled: false 2023-10-26T06:38:32.937 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-26T06:38:32.937 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-26T06:38:32.937 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-26T06:38:32.937 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-26T06:38:32.937 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.target_file_size_base: 67108864 2023-10-26T06:38:32.937 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-26T06:38:32.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-26T06:38:32.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-26T06:38:32.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-26T06:38:32.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-26T06:38:32.939 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-26T06:38:32.939 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-26T06:38:32.939 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-26T06:38:32.939 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-26T06:38:32.939 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-26T06:38:32.940 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-26T06:38:32.940 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-26T06:38:32.940 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-26T06:38:32.940 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-26T06:38:32.940 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.arena_block_size: 1048576 2023-10-26T06:38:32.940 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-26T06:38:32.941 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-26T06:38:32.941 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.disable_auto_compactions: 0 2023-10-26T06:38:32.941 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-26T06:38:32.941 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-26T06:38:32.941 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-26T06:38:32.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-26T06:38:32.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-26T06:38:32.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-26T06:38:32.942 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-26T06:38:32.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-26T06:38:32.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-26T06:38:32.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-26T06:38:32.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.table_properties_collectors: 2023-10-26T06:38:32.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.inplace_update_support: 0 2023-10-26T06:38:32.943 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-26T06:38:32.944 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-26T06:38:32.944 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-26T06:38:32.944 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-26T06:38:32.944 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.bloom_locality: 0 2023-10-26T06:38:32.944 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.max_successive_merges: 0 2023-10-26T06:38:32.945 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-26T06:38:32.945 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.paranoid_file_checks: 0 2023-10-26T06:38:32.945 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.force_consistency_checks: 1 2023-10-26T06:38:32.945 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.report_bg_io_stats: 0 2023-10-26T06:38:32.945 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.ttl: 2592000 2023-10-26T06:38:32.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-26T06:38:32.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-26T06:38:32.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-26T06:38:32.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.enable_blob_files: false 2023-10-26T06:38:32.946 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.min_blob_size: 0 2023-10-26T06:38:32.947 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.blob_file_size: 268435456 2023-10-26T06:38:32.947 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-26T06:38:32.947 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-26T06:38:32.947 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-26T06:38:32.947 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-26T06:38:32.948 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-26T06:38:32.948 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.blob_file_starting_level: 0 2023-10-26T06:38:32.948 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-26T06:38:32.948 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-26T06:38:32.948 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-26T06:38:32.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: acf7883a-af5c-4438-9946-4da1b10bf080 2023-10-26T06:38:32.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302312576262, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-26T06:38:32.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-26T06:38:32.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302312576664, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1698302312, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "acf7883a-af5c-4438-9946-4da1b10bf080", "db_session_id": "WRXUO8QBMWC3QTZLZVYV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-26T06:38:32.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302312576729, "job": 1, "event": "recovery_finished"} 2023-10-26T06:38:32.950 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-26T06:38:32.950 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-26T06:38:32.950 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55edacff0000 2023-10-26T06:38:32.950 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: DB pointer 0x55edacfdc000 2023-10-26T06:38:32.950 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-26T06:38:32.951 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-26T06:38:32.951 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** DB Stats ** 2023-10-26T06:38:32.951 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-26T06:38:32.951 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-26T06:38:32.951 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-26T06:38:32.952 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-26T06:38:32.952 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-26T06:38:32.952 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-26T06:38:32.952 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-26T06:38:32.952 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-10-26T06:38:32.953 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2023-10-26T06:38:32.953 INFO:journalctl@ceph.mon.c.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-26T06:38:32.953 INFO:journalctl@ceph.mon.c.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-26T06:38:32.953 INFO:journalctl@ceph.mon.c.smithi192.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-26T06:38:32.954 INFO:journalctl@ceph.mon.c.smithi192.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-26T06:38:32.954 INFO:journalctl@ceph.mon.c.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-26T06:38:32.954 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-10-26T06:38:32.954 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2023-10-26T06:38:32.954 INFO:journalctl@ceph.mon.c.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-26T06:38:32.955 INFO:journalctl@ceph.mon.c.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-26T06:38:32.955 INFO:journalctl@ceph.mon.c.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-26T06:38:32.955 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-10-26T06:38:32.955 INFO:journalctl@ceph.mon.c.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-26T06:38:32.956 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-10-26T06:38:32.956 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-26T06:38:32.956 INFO:journalctl@ceph.mon.c.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-26T06:38:32.956 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-26T06:38:32.956 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-26T06:38:32.957 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-26T06:38:32.957 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-26T06:38:32.957 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-26T06:38:32.957 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-26T06:38:32.957 INFO:journalctl@ceph.mon.c.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-26T06:38:32.958 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache BinnedLRUCache@0x55edacf531f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-26T06:38:32.958 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-26T06:38:32.958 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-10-26T06:38:32.958 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-26T06:38:32.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-10-26T06:38:32.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: using public_addr v2:172.21.15.192:0/0 -> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2023-10-26T06:38:32.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: starting mon.c rank -1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:38:32.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(???) e0 preinit fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:38:32.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 systemd[1]: Started Ceph mon.c for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:38:32.960 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).mds e1 new map 2023-10-26T06:38:32.960 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).mds e1 print_map 2023-10-26T06:38:32.960 INFO:journalctl@ceph.mon.c.smithi192.stdout: e1 2023-10-26T06:38:32.960 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-26T06:38:32.961 INFO:journalctl@ceph.mon.c.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-26T06:38:32.961 INFO:journalctl@ceph.mon.c.smithi192.stdout: legacy client fscid: -1 2023-10-26T06:38:32.961 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2023-10-26T06:38:32.961 INFO:journalctl@ceph.mon.c.smithi192.stdout: No filesystems configured 2023-10-26T06:38:32.961 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-26T06:38:32.962 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-26T06:38:32.962 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-26T06:38:32.962 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-26T06:38:32.963 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-26T06:38:32.963 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-26T06:38:32.963 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-26T06:38:32.963 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:38:32.963 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:38:32.964 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:38:32.964 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:38:32.964 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.964 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.964 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.965 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.965 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.965 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.965 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:38:32.965 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:32.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:32.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:38:32.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:32.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:32.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:38:32.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:32.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Deploying cephadm binary to smithi192 2023-10-26T06:38:32.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Added host smithi192 2023-10-26T06:38:32.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:32.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:38:32.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:38:32.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:32.971 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:32.972 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:38:32.972 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:32.972 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/918140397' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-26T06:38:32.973 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.973 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:32.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:38:32.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/918140397' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-26T06:38:32.975 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:32.975 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.975 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.975 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:32.975 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:32.976 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:32.976 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:32.976 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:32.976 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:32.977 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:32.977 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c;count:3 2023-10-26T06:38:32.977 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: Deploying daemon mon.c on smithi192 2023-10-26T06:38:32.977 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:32.977 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/1784001494' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:32.977 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:32 smithi192 ceph-mon[131661]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-26T06:38:33.517 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-26T06:38:33.519 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json 2023-10-26T06:38:34.092 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:38:38.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:38:38.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: mon.a calling monitor election 2023-10-26T06:38:38.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:38.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: mon.c calling monitor election 2023-10-26T06:38:38.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:38.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-26T06:38:38.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: monmap e2: 2 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:38.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: fsmap 2023-10-26T06:38:38.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:38.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: mgrmap e13: x(active, since 47s) 2023-10-26T06:38:38.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: overall HEALTH_OK 2023-10-26T06:38:38.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:38.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:37 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:38.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:38:38.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: mon.a calling monitor election 2023-10-26T06:38:38.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:38.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: mon.c calling monitor election 2023-10-26T06:38:38.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:38.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:38.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-26T06:38:38.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: monmap e2: 2 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:38.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: fsmap 2023-10-26T06:38:38.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:38.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: mgrmap e13: x(active, since 47s) 2023-10-26T06:38:38.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: overall HEALTH_OK 2023-10-26T06:38:38.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:38.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:37 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:38:38.565 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","modified":"2023-10-26T06:38:32.605168Z","created":"2023-10-26T06:36:42.499062Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-26T06:38:38.566 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2023-10-26T06:38:38.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:38 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:38.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:38 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:38.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:38 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:38.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:38 smithi192 ceph-mon[131661]: Deploying daemon mon.b on smithi135 2023-10-26T06:38:38.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:38 smithi192 ceph-mon[131661]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:38.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:38 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/3527504952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:38.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:38 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:39.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:38 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:39.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:38 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:39.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:38 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:39.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:38 smithi089 ceph-mon[95031]: Deploying daemon mon.b on smithi135 2023-10-26T06:38:39.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:38 smithi089 ceph-mon[95031]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:39.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:38 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/3527504952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:39.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:38 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:39.943 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-26T06:38:39.944 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json 2023-10-26T06:38:40.513 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:38:40.625 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 systemd[1]: Starting Ceph mon.b for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:38:40.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 podman[131364]: 2023-10-26T06:38:40.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: set uid:gid to 167:167 (ceph:ceph) 2023-10-26T06:38:40.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev), process ceph-mon, pid 7 2023-10-26T06:38:40.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pidfile_write: ignore empty --pid-file 2023-10-26T06:38:40.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 bash[131364]: 185c8937ba0cffec6c87f1cb5e8bcf06ea7a21e10dcc68ca38b6ef50932fa2d7 2023-10-26T06:38:40.973 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: load: jerasure load: lrc 2023-10-26T06:38:40.973 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: RocksDB version: 7.9.2 2023-10-26T06:38:40.973 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Git sha 0 2023-10-26T06:38:40.973 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Compile date 2023-10-24 16:36:40 2023-10-26T06:38:40.973 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: DB SUMMARY 2023-10-26T06:38:40.974 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: DB Session ID: 4APBUGQVBAGP8CX8EBKV 2023-10-26T06:38:40.974 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: CURRENT file: CURRENT 2023-10-26T06:38:40.974 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: IDENTITY file: IDENTITY 2023-10-26T06:38:40.974 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-26T06:38:40.974 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-26T06:38:40.975 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-26T06:38:40.975 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.error_if_exists: 0 2023-10-26T06:38:40.975 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.create_if_missing: 0 2023-10-26T06:38:40.975 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.paranoid_checks: 1 2023-10-26T06:38:40.975 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-26T06:38:40.976 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-26T06:38:40.976 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-26T06:38:40.976 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.env: 0x55d16be50da0 2023-10-26T06:38:40.976 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.fs: PosixFileSystem 2023-10-26T06:38:40.976 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.info_log: 0x55d16ddcf660 2023-10-26T06:38:40.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_file_opening_threads: 16 2023-10-26T06:38:40.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.statistics: (nil) 2023-10-26T06:38:40.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.use_fsync: 0 2023-10-26T06:38:40.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_log_file_size: 0 2023-10-26T06:38:40.977 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-26T06:38:40.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-26T06:38:40.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.keep_log_file_num: 1000 2023-10-26T06:38:40.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.recycle_log_file_num: 0 2023-10-26T06:38:40.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.allow_fallocate: 1 2023-10-26T06:38:40.978 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.allow_mmap_reads: 0 2023-10-26T06:38:40.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.allow_mmap_writes: 0 2023-10-26T06:38:40.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.use_direct_reads: 0 2023-10-26T06:38:40.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-26T06:38:40.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.create_missing_column_families: 0 2023-10-26T06:38:40.979 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.db_log_dir: 2023-10-26T06:38:40.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.wal_dir: 2023-10-26T06:38:40.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-26T06:38:40.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-26T06:38:40.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-26T06:38:40.980 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-26T06:38:40.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-26T06:38:40.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-26T06:38:40.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.advise_random_on_open: 1 2023-10-26T06:38:40.981 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.db_write_buffer_size: 0 2023-10-26T06:38:40.982 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.write_buffer_manager: 0x55d16dd87360 2023-10-26T06:38:40.982 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-26T06:38:40.982 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-26T06:38:40.982 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-26T06:38:40.982 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.rate_limiter: (nil) 2023-10-26T06:38:40.983 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-26T06:38:40.983 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.wal_recovery_mode: 2 2023-10-26T06:38:40.983 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.enable_thread_tracking: 0 2023-10-26T06:38:40.983 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.enable_pipelined_write: 0 2023-10-26T06:38:40.984 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.unordered_write: 0 2023-10-26T06:38:40.984 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-26T06:38:40.984 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-26T06:38:40.984 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-26T06:38:40.984 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-26T06:38:40.985 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.row_cache: None 2023-10-26T06:38:40.985 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.wal_filter: None 2023-10-26T06:38:40.985 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-26T06:38:40.985 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.allow_ingest_behind: 0 2023-10-26T06:38:40.985 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.two_write_queues: 0 2023-10-26T06:38:40.986 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.manual_wal_flush: 0 2023-10-26T06:38:40.986 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.wal_compression: 0 2023-10-26T06:38:40.986 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.atomic_flush: 0 2023-10-26T06:38:40.986 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-26T06:38:40.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-26T06:38:40.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-26T06:38:40.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.log_readahead_size: 0 2023-10-26T06:38:40.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-26T06:38:40.987 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.best_efforts_recovery: 0 2023-10-26T06:38:40.988 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-26T06:38:40.988 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-26T06:38:40.988 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.allow_data_in_errors: 0 2023-10-26T06:38:40.988 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.db_host_id: __hostname__ 2023-10-26T06:38:40.989 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-26T06:38:40.989 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_background_jobs: 2 2023-10-26T06:38:40.989 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_background_compactions: -1 2023-10-26T06:38:40.989 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_subcompactions: 1 2023-10-26T06:38:40.989 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-26T06:38:40.990 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-26T06:38:40.990 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-26T06:38:40.990 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_total_wal_size: 0 2023-10-26T06:38:40.990 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-26T06:38:40.991 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-26T06:38:40.991 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-26T06:38:40.991 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-26T06:38:40.991 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_open_files: -1 2023-10-26T06:38:40.991 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bytes_per_sync: 0 2023-10-26T06:38:40.991 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-26T06:38:40.992 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-26T06:38:40.992 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_readahead_size: 0 2023-10-26T06:38:40.992 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_background_flushes: -1 2023-10-26T06:38:40.992 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Compression algorithms supported: 2023-10-26T06:38:40.992 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-26T06:38:40.993 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kZSTD supported: 0 2023-10-26T06:38:40.993 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kXpressCompression supported: 0 2023-10-26T06:38:40.993 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kLZ4HCCompression supported: 1 2023-10-26T06:38:40.993 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kZlibCompression supported: 1 2023-10-26T06:38:40.994 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kSnappyCompression supported: 1 2023-10-26T06:38:40.994 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kLZ4Compression supported: 1 2023-10-26T06:38:40.994 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: kBZip2Compression supported: 0 2023-10-26T06:38:40.994 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-26T06:38:40.995 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-26T06:38:40.995 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-26T06:38:40.995 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-26T06:38:40.995 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-26T06:38:40.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.merge_operator: 2023-10-26T06:38:40.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_filter: None 2023-10-26T06:38:40.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_filter_factory: None 2023-10-26T06:38:40.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.sst_partitioner_factory: None 2023-10-26T06:38:40.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-26T06:38:40.997 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-26T06:38:40.997 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d16ddcf760) 2023-10-26T06:38:40.997 INFO:journalctl@ceph.mon.b.smithi135.stdout: cache_index_and_filter_blocks: 1 2023-10-26T06:38:40.997 INFO:journalctl@ceph.mon.b.smithi135.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-26T06:38:40.998 INFO:journalctl@ceph.mon.b.smithi135.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-26T06:38:40.998 INFO:journalctl@ceph.mon.b.smithi135.stdout: pin_top_level_index_and_filter: 1 2023-10-26T06:38:40.998 INFO:journalctl@ceph.mon.b.smithi135.stdout: index_type: 0 2023-10-26T06:38:40.998 INFO:journalctl@ceph.mon.b.smithi135.stdout: data_block_index_type: 0 2023-10-26T06:38:40.998 INFO:journalctl@ceph.mon.b.smithi135.stdout: index_shortening: 1 2023-10-26T06:38:40.999 INFO:journalctl@ceph.mon.b.smithi135.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-26T06:38:40.999 INFO:journalctl@ceph.mon.b.smithi135.stdout: checksum: 4 2023-10-26T06:38:40.999 INFO:journalctl@ceph.mon.b.smithi135.stdout: no_block_cache: 0 2023-10-26T06:38:40.999 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache: 0x55d16de251f0 2023-10-26T06:38:40.999 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache_name: BinnedLRUCache 2023-10-26T06:38:41.000 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache_options: 2023-10-26T06:38:41.000 INFO:journalctl@ceph.mon.b.smithi135.stdout: capacity : 536870912 2023-10-26T06:38:41.000 INFO:journalctl@ceph.mon.b.smithi135.stdout: num_shard_bits : 4 2023-10-26T06:38:41.000 INFO:journalctl@ceph.mon.b.smithi135.stdout: strict_capacity_limit : 0 2023-10-26T06:38:41.000 INFO:journalctl@ceph.mon.b.smithi135.stdout: high_pri_pool_ratio: 0.000 2023-10-26T06:38:41.000 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_cache_compressed: (nil) 2023-10-26T06:38:41.001 INFO:journalctl@ceph.mon.b.smithi135.stdout: persistent_cache: (nil) 2023-10-26T06:38:41.001 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_size: 4096 2023-10-26T06:38:41.001 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_size_deviation: 10 2023-10-26T06:38:41.001 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_restart_interval: 16 2023-10-26T06:38:41.001 INFO:journalctl@ceph.mon.b.smithi135.stdout: index_block_restart_interval: 1 2023-10-26T06:38:41.002 INFO:journalctl@ceph.mon.b.smithi135.stdout: metadata_block_size: 4096 2023-10-26T06:38:41.002 INFO:journalctl@ceph.mon.b.smithi135.stdout: partition_filters: 0 2023-10-26T06:38:41.002 INFO:journalctl@ceph.mon.b.smithi135.stdout: use_delta_encoding: 1 2023-10-26T06:38:41.002 INFO:journalctl@ceph.mon.b.smithi135.stdout: filter_policy: bloomfilter 2023-10-26T06:38:41.002 INFO:journalctl@ceph.mon.b.smithi135.stdout: whole_key_filtering: 1 2023-10-26T06:38:41.003 INFO:journalctl@ceph.mon.b.smithi135.stdout: verify_compression: 0 2023-10-26T06:38:41.003 INFO:journalctl@ceph.mon.b.smithi135.stdout: read_amp_bytes_per_bit: 0 2023-10-26T06:38:41.003 INFO:journalctl@ceph.mon.b.smithi135.stdout: format_version: 5 2023-10-26T06:38:41.003 INFO:journalctl@ceph.mon.b.smithi135.stdout: enable_index_compression: 1 2023-10-26T06:38:41.003 INFO:journalctl@ceph.mon.b.smithi135.stdout: block_align: 0 2023-10-26T06:38:41.003 INFO:journalctl@ceph.mon.b.smithi135.stdout: max_auto_readahead_size: 262144 2023-10-26T06:38:41.004 INFO:journalctl@ceph.mon.b.smithi135.stdout: prepopulate_block_cache: 0 2023-10-26T06:38:41.004 INFO:journalctl@ceph.mon.b.smithi135.stdout: initial_auto_readahead_size: 8192 2023-10-26T06:38:41.004 INFO:journalctl@ceph.mon.b.smithi135.stdout: num_file_reads_for_auto_readahead: 2 2023-10-26T06:38:41.004 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.write_buffer_size: 33554432 2023-10-26T06:38:41.004 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_write_buffer_number: 2 2023-10-26T06:38:41.005 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression: NoCompression 2023-10-26T06:38:41.005 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression: Disabled 2023-10-26T06:38:41.005 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.prefix_extractor: nullptr 2023-10-26T06:38:41.005 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-26T06:38:41.005 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.num_levels: 7 2023-10-26T06:38:41.006 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-26T06:38:41.006 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-26T06:38:41.006 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-26T06:38:41.006 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-26T06:38:41.006 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-26T06:38:41.007 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-26T06:38:41.007 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-26T06:38:41.007 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-26T06:38:41.007 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-26T06:38:41.007 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-26T06:38:41.007 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-26T06:38:41.008 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-26T06:38:41.008 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-26T06:38:41.008 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.level: 32767 2023-10-26T06:38:41.008 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.strategy: 0 2023-10-26T06:38:41.008 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-26T06:38:41.009 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-26T06:38:41.009 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-26T06:38:41.009 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-26T06:38:41.009 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.enabled: false 2023-10-26T06:38:41.009 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-26T06:38:41.010 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-26T06:38:41.010 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-26T06:38:41.010 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-26T06:38:41.010 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.target_file_size_base: 67108864 2023-10-26T06:38:41.010 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-26T06:38:41.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-26T06:38:41.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-26T06:38:41.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-26T06:38:41.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-26T06:38:41.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-26T06:38:41.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-26T06:38:41.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-26T06:38:41.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-26T06:38:41.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-26T06:38:41.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-26T06:38:41.013 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-26T06:38:41.013 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-26T06:38:41.013 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-26T06:38:41.013 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.arena_block_size: 1048576 2023-10-26T06:38:41.013 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-26T06:38:41.013 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-26T06:38:41.014 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.disable_auto_compactions: 0 2023-10-26T06:38:41.014 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-26T06:38:41.014 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-26T06:38:41.014 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-26T06:38:41.015 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-26T06:38:41.015 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-26T06:38:41.015 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-26T06:38:41.015 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-26T06:38:41.015 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-26T06:38:41.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-26T06:38:41.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-26T06:38:41.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.table_properties_collectors: 2023-10-26T06:38:41.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.inplace_update_support: 0 2023-10-26T06:38:41.016 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-26T06:38:41.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-26T06:38:41.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-26T06:38:41.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-26T06:38:41.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.bloom_locality: 0 2023-10-26T06:38:41.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.max_successive_merges: 0 2023-10-26T06:38:41.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-26T06:38:41.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.paranoid_file_checks: 0 2023-10-26T06:38:41.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.force_consistency_checks: 1 2023-10-26T06:38:41.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.report_bg_io_stats: 0 2023-10-26T06:38:41.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.ttl: 2592000 2023-10-26T06:38:41.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-26T06:38:41.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-26T06:38:41.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-26T06:38:41.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.enable_blob_files: false 2023-10-26T06:38:41.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.min_blob_size: 0 2023-10-26T06:38:41.019 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.blob_file_size: 268435456 2023-10-26T06:38:41.020 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-26T06:38:41.020 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-26T06:38:41.020 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-26T06:38:41.020 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-26T06:38:41.020 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-26T06:38:41.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.blob_file_starting_level: 0 2023-10-26T06:38:41.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-26T06:38:41.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-26T06:38:41.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-26T06:38:41.021 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e82778e6-9031-4483-bdd1-b113ea37169c 2023-10-26T06:38:41.022 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302320825184, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-26T06:38:41.022 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-26T06:38:41.022 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302320825783, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1698302320, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e82778e6-9031-4483-bdd1-b113ea37169c", "db_session_id": "4APBUGQVBAGP8CX8EBKV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-26T06:38:41.022 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: EVENT_LOG_v1 {"time_micros": 1698302320825872, "job": 1, "event": "recovery_finished"} 2023-10-26T06:38:41.022 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-26T06:38:41.023 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-26T06:38:41.023 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d16dec2000 2023-10-26T06:38:41.023 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: DB pointer 0x55d16deae000 2023-10-26T06:38:41.023 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-26T06:38:41.023 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-26T06:38:41.023 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** DB Stats ** 2023-10-26T06:38:41.024 INFO:journalctl@ceph.mon.b.smithi135.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-26T06:38:41.024 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-26T06:38:41.024 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-26T06:38:41.024 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-26T06:38:41.024 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-26T06:38:41.025 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-26T06:38:41.025 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-26T06:38:41.025 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2023-10-26T06:38:41.025 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** Compaction Stats [default] ** 2023-10-26T06:38:41.025 INFO:journalctl@ceph.mon.b.smithi135.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-26T06:38:41.026 INFO:journalctl@ceph.mon.b.smithi135.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-26T06:38:41.026 INFO:journalctl@ceph.mon.b.smithi135.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:38:41.026 INFO:journalctl@ceph.mon.b.smithi135.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:38:41.026 INFO:journalctl@ceph.mon.b.smithi135.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:38:41.026 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2023-10-26T06:38:41.027 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** Compaction Stats [default] ** 2023-10-26T06:38:41.027 INFO:journalctl@ceph.mon.b.smithi135.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-26T06:38:41.027 INFO:journalctl@ceph.mon.b.smithi135.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-26T06:38:41.027 INFO:journalctl@ceph.mon.b.smithi135.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-26T06:38:41.027 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2023-10-26T06:38:41.028 INFO:journalctl@ceph.mon.b.smithi135.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-26T06:38:41.028 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2023-10-26T06:38:41.028 INFO:journalctl@ceph.mon.b.smithi135.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-26T06:38:41.028 INFO:journalctl@ceph.mon.b.smithi135.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-26T06:38:41.028 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-26T06:38:41.029 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-26T06:38:41.029 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-26T06:38:41.029 INFO:journalctl@ceph.mon.b.smithi135.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-26T06:38:41.029 INFO:journalctl@ceph.mon.b.smithi135.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-26T06:38:41.029 INFO:journalctl@ceph.mon.b.smithi135.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-26T06:38:41.029 INFO:journalctl@ceph.mon.b.smithi135.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-26T06:38:41.030 INFO:journalctl@ceph.mon.b.smithi135.stdout: Block cache BinnedLRUCache@0x55d16de251f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-26T06:38:41.030 INFO:journalctl@ceph.mon.b.smithi135.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-26T06:38:41.030 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2023-10-26T06:38:41.030 INFO:journalctl@ceph.mon.b.smithi135.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-26T06:38:41.030 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-26T06:38:41.031 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: using public_addr v2:172.21.15.135:0/0 -> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] 2023-10-26T06:38:41.031 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: starting mon.b rank -1 at public addrs [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] at bind addrs [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:38:41.031 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(???) e0 preinit fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:38:41.031 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 systemd[1]: Started Ceph mon.b for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:38:41.031 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).mds e1 new map 2023-10-26T06:38:41.032 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-26T06:38:41.032 INFO:journalctl@ceph.mon.b.smithi135.stdout: e1 2023-10-26T06:38:41.032 INFO:journalctl@ceph.mon.b.smithi135.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-26T06:38:41.032 INFO:journalctl@ceph.mon.b.smithi135.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-26T06:38:41.032 INFO:journalctl@ceph.mon.b.smithi135.stdout: legacy client fscid: -1 2023-10-26T06:38:41.033 INFO:journalctl@ceph.mon.b.smithi135.stdout: 2023-10-26T06:38:41.033 INFO:journalctl@ceph.mon.b.smithi135.stdout: No filesystems configured 2023-10-26T06:38:41.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-26T06:38:41.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-26T06:38:41.033 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-26T06:38:41.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-26T06:38:41.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-26T06:38:41.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-26T06:38:41.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-26T06:38:41.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:38:41.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:38:41.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-26T06:38:41.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:38:41.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:38:41.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:41.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:41.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:38:41.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:41.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:41.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:38:41.038 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:41.039 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Deploying cephadm binary to smithi192 2023-10-26T06:38:41.039 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.039 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.039 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Added host smithi192 2023-10-26T06:38:41.039 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:41.040 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.040 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.040 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.040 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.040 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.041 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:38:41.041 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.041 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.041 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.041 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:38:41.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:41.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:41.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:38:41.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:41.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/918140397' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-26T06:38:41.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.044 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:41.044 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:38:41.044 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.044 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/918140397' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-26T06:38:41.044 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:41.045 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.045 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.045 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:41.045 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:41.045 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:38:41.046 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.046 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:41.046 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:41.046 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:38:41.046 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c;count:3 2023-10-26T06:38:41.047 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Deploying daemon mon.c on smithi192 2023-10-26T06:38:41.047 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.047 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/1784001494' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:41.047 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:38:41.047 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:41.048 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.a calling monitor election 2023-10-26T06:38:41.048 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:41.048 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.048 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:41.048 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.c calling monitor election 2023-10-26T06:38:41.049 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:41.049 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.049 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:41.049 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:41.049 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-26T06:38:41.050 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: monmap e2: 2 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:41.050 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: fsmap 2023-10-26T06:38:41.050 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:41.050 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mgrmap e13: x(active, since 47s) 2023-10-26T06:38:41.050 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: overall HEALTH_OK 2023-10-26T06:38:41.050 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.051 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.051 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:41.051 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:41.051 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:41.051 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: Deploying daemon mon.b on smithi135 2023-10-26T06:38:41.052 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:41.052 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/3527504952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:41.052 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:41.052 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:40 smithi135 ceph-mon[131403]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-26T06:38:46.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:38:46.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: mon.c calling monitor election 2023-10-26T06:38:46.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: mon.a calling monitor election 2023-10-26T06:38:46.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:46.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:46.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:46.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-26T06:38:46.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: monmap e3: 3 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:46.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: fsmap 2023-10-26T06:38:46.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:46.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: mgrmap e13: x(active, since 55s) 2023-10-26T06:38:46.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-26T06:38:46.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-26T06:38:46.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-26T06:38:46.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: mon.b (rank 2) addr [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] is down (out of quorum) 2023-10-26T06:38:46.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:45 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:38:46.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: mon.c calling monitor election 2023-10-26T06:38:46.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: mon.a calling monitor election 2023-10-26T06:38:46.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:46.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:46.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:46.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:46.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-26T06:38:46.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: monmap e3: 3 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:46.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: fsmap 2023-10-26T06:38:46.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:46.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: mgrmap e13: x(active, since 55s) 2023-10-26T06:38:46.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-26T06:38:46.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-26T06:38:46.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-26T06:38:46.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: mon.b (rank 2) addr [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] is down (out of quorum) 2023-10-26T06:38:46.400 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.400 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.400 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.400 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:45 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-10-26T06:38:46.401 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","modified":"2023-10-26T06:38:40.856643Z","created":"2023-10-26T06:36:42.499062Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-26T06:38:46.404 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2023-10-26T06:38:46.765 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-26T06:38:46.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-26T06:38:47.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:46 smithi192 ceph-mon[131661]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:47.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:46 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:47.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:46 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/142022048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:47.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:46 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:47.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:46 smithi089 ceph-mon[95031]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:47.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:46 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:47.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:46 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/142022048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:47.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:46 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:47.511 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:48.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:38:48.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: mon.c calling monitor election 2023-10-26T06:38:48.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: mon.a calling monitor election 2023-10-26T06:38:48.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:48.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:38:48.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:48.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:48.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:48.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:48.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:48.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:48.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:48.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-26T06:38:48.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: monmap e3: 3 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:48.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: fsmap 2023-10-26T06:38:48.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:48.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: mgrmap e13: x(active, since 55s) 2023-10-26T06:38:48.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-26T06:38:48.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-26T06:38:48.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-26T06:38:48.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: mon.b (rank 2) addr [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] is down (out of quorum) 2023-10-26T06:38:48.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:48.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:48.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:48.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:48.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:48.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:38:48.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/142022048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-26T06:38:48.223 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:47 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:49.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: mon.b calling monitor election 2023-10-26T06:38:49.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Updating smithi089:/etc/ceph/ceph.conf 2023-10-26T06:38:49.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:38:49.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:38:49.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: mon.b calling monitor election 2023-10-26T06:38:49.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: mon.c calling monitor election 2023-10-26T06:38:49.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: mon.a calling monitor election 2023-10-26T06:38:49.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-26T06:38:49.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: monmap e3: 3 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:49.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: fsmap 2023-10-26T06:38:49.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:49.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: mgrmap e13: x(active, since 57s) 2023-10-26T06:38:49.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-26T06:38:49.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Cluster is now healthy 2023-10-26T06:38:49.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: overall HEALTH_OK 2023-10-26T06:38:49.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:49.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.150 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.150 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.150 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.150 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.150 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.151 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:48 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:49.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: mon.b calling monitor election 2023-10-26T06:38:49.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Updating smithi089:/etc/ceph/ceph.conf 2023-10-26T06:38:49.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:38:49.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:38:49.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: mon.b calling monitor election 2023-10-26T06:38:49.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: mon.c calling monitor election 2023-10-26T06:38:49.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: mon.a calling monitor election 2023-10-26T06:38:49.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-26T06:38:49.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: monmap e3: 3 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:49.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: fsmap 2023-10-26T06:38:49.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:49.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: mgrmap e13: x(active, since 57s) 2023-10-26T06:38:49.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-26T06:38:49.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Cluster is now healthy 2023-10-26T06:38:49.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: overall HEALTH_OK 2023-10-26T06:38:49.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:49.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.222 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:48 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:49.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: mon.b calling monitor election 2023-10-26T06:38:49.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Updating smithi089:/etc/ceph/ceph.conf 2023-10-26T06:38:49.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:38:49.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:38:49.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: mon.b calling monitor election 2023-10-26T06:38:49.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: mon.c calling monitor election 2023-10-26T06:38:49.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: mon.a calling monitor election 2023-10-26T06:38:49.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-26T06:38:49.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: monmap e3: 3 mons at {a=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],b=[v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0],c=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-26T06:38:49.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: fsmap 2023-10-26T06:38:49.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: osdmap e4: 0 total, 0 up, 0 in 2023-10-26T06:38:49.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: mgrmap e13: x(active, since 57s) 2023-10-26T06:38:49.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-26T06:38:49.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Cluster is now healthy 2023-10-26T06:38:49.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: overall HEALTH_OK 2023-10-26T06:38:49.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:49.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:49.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:38:49.295 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:48 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:38:49.499 INFO:teuthology.orchestra.run.smithi089.stdout:# minimal ceph.conf for e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:38:49.500 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2023-10-26T06:38:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:38:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: mon_host = [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2023-10-26T06:38:50.057 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-26T06:38:50.057 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:38:50.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2023-10-26T06:38:50.113 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:49.857+0000 7f7dfa8a4700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-26T06:38:50.114 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.114 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.114 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.115 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.115 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.115 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.115 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.115 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: Reconfiguring mon.a (unknown last config time)... 2023-10-26T06:38:50.116 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:50.116 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:50.116 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:50.116 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: Reconfiguring daemon mon.a on smithi089 2023-10-26T06:38:50.116 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:49 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4121008268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:50.136 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:38:50.136 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:50.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: Reconfiguring mon.a (unknown last config time)... 2023-10-26T06:38:50.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:50.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:50.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:50.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: Reconfiguring daemon mon.a on smithi089 2023-10-26T06:38:50.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:49 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4121008268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:50.245 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:38:50.246 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.conf 2023-10-26T06:38:50.282 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:38:50.282 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:50.352 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:38:50.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2023-10-26T06:38:50.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:50.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: Reconfiguring mon.a (unknown last config time)... 2023-10-26T06:38:50.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:50.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:50.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:50.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: Reconfiguring daemon mon.a on smithi089 2023-10-26T06:38:50.389 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:49 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4121008268' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:50.390 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:38:50.391 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:38:50.462 INFO:tasks.cephadm:enabling module snap_schedule 2023-10-26T06:38:50.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2023-10-26T06:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:38:51.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:51 smithi192 ceph-mon[131661]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:51.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:51 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:51 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:51 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:51 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:51.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:51 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:51.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:51 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:51.442 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:51 smithi135 ceph-mon[131403]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:51.442 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:51 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.443 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:51 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.443 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:51 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.443 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:51 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:51.443 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:51 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:51.443 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:51 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:51.476 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:51 smithi089 ceph-mon[95031]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:51.477 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.477 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.477 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:51.477 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:51.477 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:51.477 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:51 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:52.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:52 smithi192 ceph-mon[131661]: Reconfiguring mon.b (monmap changed)... 2023-10-26T06:38:52.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:52 smithi192 ceph-mon[131661]: Reconfiguring daemon mon.b on smithi135 2023-10-26T06:38:52.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:52 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:52.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:52 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:52.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:52 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:52.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:52 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:52.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:52 smithi192 ceph-mon[131661]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:52.446 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:52 smithi089 ceph-mon[95031]: Reconfiguring mon.b (monmap changed)... 2023-10-26T06:38:52.446 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:52 smithi089 ceph-mon[95031]: Reconfiguring daemon mon.b on smithi135 2023-10-26T06:38:52.446 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:52 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:52.447 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:52 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:52.447 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:52 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:52.447 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:52 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:52.447 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:52 smithi089 ceph-mon[95031]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:52.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:52 smithi135 ceph-mon[131403]: Reconfiguring mon.b (monmap changed)... 2023-10-26T06:38:52.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:52 smithi135 ceph-mon[131403]: Reconfiguring daemon mon.b on smithi135 2023-10-26T06:38:52.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:52 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:52.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:52 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' 2023-10-26T06:38:52.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:52 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:38:52.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:52 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:38:52.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:52 smithi135 ceph-mon[131403]: from='mgr.14152 172.21.15.89:0/1592020361' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:38:53.206 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:53 smithi192 ceph-mon[131661]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:53.206 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:53 smithi192 ceph-mon[131661]: Reconfiguring mon.c (monmap changed)... 2023-10-26T06:38:53.207 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:53 smithi192 ceph-mon[131661]: Reconfiguring daemon mon.c on smithi192 2023-10-26T06:38:53.207 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:53 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2466120647' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-26T06:38:53.207 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:53 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-26T06:38:53.391 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-mon[95031]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:53.391 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-mon[95031]: Reconfiguring mon.c (monmap changed)... 2023-10-26T06:38:53.392 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-mon[95031]: Reconfiguring daemon mon.c on smithi192 2023-10-26T06:38:53.392 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2466120647' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-26T06:38:53.392 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-26T06:38:53.392 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: ignoring --setuser ceph since I am not root 2023-10-26T06:38:53.393 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: ignoring --setgroup ceph since I am not root 2023-10-26T06:38:53.393 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:53.390+0000 7f5828b68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T06:38:53.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:53 smithi135 ceph-mon[131403]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:38:53.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:53 smithi135 ceph-mon[131403]: Reconfiguring mon.c (monmap changed)... 2023-10-26T06:38:53.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:53 smithi135 ceph-mon[131403]: Reconfiguring daemon mon.c on smithi192 2023-10-26T06:38:53.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:53 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2466120647' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-26T06:38:53.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:53 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-26T06:38:53.592 INFO:tasks.cephadm:Adding mgr.x on smithi089 2023-10-26T06:38:53.593 INFO:tasks.cephadm:Adding mgr.y on smithi135 2023-10-26T06:38:53.593 INFO:tasks.cephadm:Adding mgr.z on smithi192 2023-10-26T06:38:53.594 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch apply mgr '3;smithi089=x;smithi135=y;smithi192=z' 2023-10-26T06:38:53.734 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:53.493+0000 7f5828b68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T06:38:53.735 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:53.611+0000 7f5828b68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T06:38:54.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:53.733+0000 7f5828b68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T06:38:54.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:53.955+0000 7f5828b68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T06:38:54.151 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:38:54.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:54 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-26T06:38:54.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:38:54 smithi192 ceph-mon[131661]: mgrmap e14: x(active, since 63s) 2023-10-26T06:38:54.401 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:54 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-26T06:38:54.401 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:38:54 smithi089 ceph-mon[95031]: mgrmap e14: x(active, since 63s) 2023-10-26T06:38:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:54 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-26T06:38:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:38:54 smithi135 ceph-mon[131403]: mgrmap e14: x(active, since 63s) 2023-10-26T06:38:54.675 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:54.400+0000 7f5828b68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T06:38:54.675 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:54.560+0000 7f5828b68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T06:38:55.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:54.674+0000 7f5828b68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T06:38:55.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:54.796+0000 7f5828b68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T06:38:55.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:55.330+0000 7f5828b68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T06:38:56.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:56.174+0000 7f5828b68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T06:38:56.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:56.281+0000 7f5828b68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T06:38:57.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:57.233+0000 7f5828b68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:38:57.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:57.353+0000 7f5828b68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T06:38:57.942 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:57.577+0000 7f5828b68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:38:58.242 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:57.941+0000 7f5828b68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T06:38:58.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:58 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:58.240+0000 7f5828b68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T06:38:58.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:58 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:58.432+0000 7f5828b68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T06:38:59.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:58 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:58.783+0000 7f5828b68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T06:39:00.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:59 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:59.709+0000 7f5828b68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T06:39:00.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:38:59 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:38:59.884+0000 7f5828b68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T06:39:00.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:39:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:39:00.139+0000 7f5828b68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T06:39:00.539 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:39:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:39:00.288+0000 7f5828b68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T06:39:01.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:39:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:39:00.569+0000 7f5828b68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T06:39:01.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: Active manager daemon x restarted 2023-10-26T06:39:01.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: Activating manager daemon x 2023-10-26T06:39:01.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: osdmap e5: 0 total, 0 up, 0 in 2023-10-26T06:39:01.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: mgrmap e15: x(active, starting, since 0.0047004s) 2023-10-26T06:39:01.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:39:01.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:39:01.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:39:01.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-26T06:39:01.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-26T06:39:01.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-26T06:39:01.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-26T06:39:01.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: Manager daemon x is now available 2023-10-26T06:39:01.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:01.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:01.994 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: Active manager daemon x restarted 2023-10-26T06:39:01.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: Activating manager daemon x 2023-10-26T06:39:01.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: osdmap e5: 0 total, 0 up, 0 in 2023-10-26T06:39:01.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: mgrmap e15: x(active, starting, since 0.0047004s) 2023-10-26T06:39:01.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:39:01.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:39:01.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:39:01.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-26T06:39:01.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-26T06:39:01.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-26T06:39:01.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-26T06:39:01.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: Manager daemon x is now available 2023-10-26T06:39:01.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:01.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:02.009 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:39:01.637+0000 7f5828b68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T06:39:02.010 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: Active manager daemon x restarted 2023-10-26T06:39:02.010 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: Activating manager daemon x 2023-10-26T06:39:02.010 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: osdmap e5: 0 total, 0 up, 0 in 2023-10-26T06:39:02.010 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: mgrmap e15: x(active, starting, since 0.0047004s) 2023-10-26T06:39:02.011 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:39:02.011 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:39:02.011 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:39:02.011 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-26T06:39:02.012 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-26T06:39:02.012 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-26T06:39:02.012 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-26T06:39:02.012 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: Manager daemon x is now available 2023-10-26T06:39:02.012 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:02.013 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:02.657 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2023-10-26T06:39:02.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-26T06:39:02.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-26T06:39:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: [26/Oct/2023:06:39:02] ENGINE Bus STARTING 2023-10-26T06:39:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: [26/Oct/2023:06:39:02] ENGINE Serving on https://172.21.15.89:7150 2023-10-26T06:39:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: [26/Oct/2023:06:39:02] ENGINE Serving on http://172.21.15.89:8765 2023-10-26T06:39:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: [26/Oct/2023:06:39:02] ENGINE Bus STARTED 2023-10-26T06:39:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: mgrmap e16: x(active, since 1.00912s) 2023-10-26T06:39:02.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:03.025 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-26T06:39:03.025 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-26T06:39:03.025 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: [26/Oct/2023:06:39:02] ENGINE Bus STARTING 2023-10-26T06:39:03.026 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: [26/Oct/2023:06:39:02] ENGINE Serving on https://172.21.15.89:7150 2023-10-26T06:39:03.026 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: [26/Oct/2023:06:39:02] ENGINE Serving on http://172.21.15.89:8765 2023-10-26T06:39:03.026 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: [26/Oct/2023:06:39:02] ENGINE Bus STARTED 2023-10-26T06:39:03.026 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: mgrmap e16: x(active, since 1.00912s) 2023-10-26T06:39:03.026 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:03.028 DEBUG:teuthology.orchestra.run.smithi135:mgr.y> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.y.service 2023-10-26T06:39:03.031 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.z.service 2023-10-26T06:39:03.036 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2023-10-26T06:39:03.037 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set client client_mount_timeout 600 2023-10-26T06:39:03.041 INFO:tasks.cephadm:setting config [client] debug_client = 20 2023-10-26T06:39:03.042 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set client debug_client 20 2023-10-26T06:39:03.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-26T06:39:03.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-26T06:39:03.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: [26/Oct/2023:06:39:02] ENGINE Bus STARTING 2023-10-26T06:39:03.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: [26/Oct/2023:06:39:02] ENGINE Serving on https://172.21.15.89:7150 2023-10-26T06:39:03.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: [26/Oct/2023:06:39:02] ENGINE Serving on http://172.21.15.89:8765 2023-10-26T06:39:03.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: [26/Oct/2023:06:39:02] ENGINE Bus STARTED 2023-10-26T06:39:03.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: mgrmap e16: x(active, since 1.00912s) 2023-10-26T06:39:03.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:03.056 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2023-10-26T06:39:03.056 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set client debug_ms 1 2023-10-26T06:39:03.059 INFO:journalctl@ceph.mgr.y.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:39:03.070 INFO:journalctl@ceph.mgr.z.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:39:03.106 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2023-10-26T06:39:03.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set client rados_mon_op_timeout 900 2023-10-26T06:39:03.155 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2023-10-26T06:39:03.156 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set client rados_osd_op_timeout 900 2023-10-26T06:39:03.206 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2023-10-26T06:39:03.207 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds debug_mds 20 2023-10-26T06:39:03.255 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2023-10-26T06:39:03.255 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds debug_mds_balancer 20 2023-10-26T06:39:03.309 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2023-10-26T06:39:03.309 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds debug_ms 1 2023-10-26T06:39:03.360 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-26T06:39:03.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_bal_fragment_size_max 10000 2023-10-26T06:39:03.413 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2023-10-26T06:39:03.414 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_bal_merge_size 5 2023-10-26T06:39:03.468 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2023-10-26T06:39:03.468 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_bal_split_bits 3 2023-10-26T06:39:03.524 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2023-10-26T06:39:03.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_bal_split_size 100 2023-10-26T06:39:03.579 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2023-10-26T06:39:03.581 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_debug_frag True 2023-10-26T06:39:03.641 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2023-10-26T06:39:03.641 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_debug_scatterstat True 2023-10-26T06:39:03.698 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2023-10-26T06:39:03.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_op_complaint_time 180 2023-10-26T06:39:03.756 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2023-10-26T06:39:03.757 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds mds_verify_scatter True 2023-10-26T06:39:03.824 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2023-10-26T06:39:03.824 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds osd_op_complaint_time 180 2023-10-26T06:39:03.879 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2023-10-26T06:39:03.879 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds rados_mon_op_timeout 900 2023-10-26T06:39:03.935 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2023-10-26T06:39:03.935 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mds rados_osd_op_timeout 900 2023-10-26T06:39:03.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:03 smithi135 ceph-mon[131403]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi135=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:39:03.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:03 smithi135 ceph-mon[131403]: Saving service mgr spec with placement smithi089=x;smithi135=y;smithi192=z;count:3 2023-10-26T06:39:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.001 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2023-10-26T06:39:04.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mgr debug_cephsqlite 20 2023-10-26T06:39:04.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:03 smithi089 ceph-mon[95031]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi135=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:39:04.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:03 smithi089 ceph-mon[95031]: Saving service mgr spec with placement smithi089=x;smithi135=y;smithi192=z;count:3 2023-10-26T06:39:04.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.004 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.005 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.070 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2023-10-26T06:39:04.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-26T06:39:04.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:03 smithi192 ceph-mon[131661]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi135=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:39:04.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:03 smithi192 ceph-mon[131661]: Saving service mgr spec with placement smithi089=x;smithi135=y;smithi192=z;count:3 2023-10-26T06:39:04.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:04.154 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2023-10-26T06:39:04.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-26T06:39:04.229 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2023-10-26T06:39:04.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set mon mon_op_complaint_time 120 2023-10-26T06:39:04.318 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2023-10-26T06:39:04.318 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config set osd osd_op_complaint_time 180 2023-10-26T06:39:04.384 DEBUG:tasks.cephadm:set 24 configs 2023-10-26T06:39:04.385 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi089') 2023-10-26T06:39:04.397 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:04.881 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:05.149 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:05.249 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:05.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:05.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:05.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:05 smithi089 ceph-mon[95031]: mgrmap e17: x(active, since 2s) 2023-10-26T06:39:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:05 smithi192 ceph-mon[131661]: mgrmap e17: x(active, since 2s) 2023-10-26T06:39:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:05.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:05 smithi135 ceph-mon[131403]: mgrmap e17: x(active, since 2s) 2023-10-26T06:39:05.830 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:06.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:06.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:06.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:06.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:06.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:06.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:09.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:09.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi089:/etc/ceph/ceph.conf 2023-10-26T06:39:09.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:39:09.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:39:09.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:09.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:09.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:09.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:09 smithi192 ceph-mon[131661]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:09.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:09.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:09.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:09.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:09.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:09.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi089:/etc/ceph/ceph.conf 2023-10-26T06:39:09.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:39:09.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:39:09.673 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.674 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.674 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:09.674 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:09.674 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.674 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:09.675 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:09 smithi089 ceph-mon[95031]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:09.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:39:09.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:09.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:09.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi089:/etc/ceph/ceph.conf 2023-10-26T06:39:09.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi135:/etc/ceph/ceph.conf 2023-10-26T06:39:09.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi192:/etc/ceph/ceph.conf 2023-10-26T06:39:09.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:09.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:09.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.conf 2023-10-26T06:39:09.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:09.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:09 smithi135 ceph-mon[131403]: Updating smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:10.173 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:10.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:10 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:10 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:10 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:10 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:10 smithi192 ceph-mon[131661]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:10.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:10 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3212295319' entity='client.admin' 2023-10-26T06:39:10.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:10 smithi192 ceph-mon[131661]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:10.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:10 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:10 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:10 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:10 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:10 smithi135 ceph-mon[131403]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:10.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:10 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3212295319' entity='client.admin' 2023-10-26T06:39:10.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:10 smithi135 ceph-mon[131403]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:10.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:10 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:10 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:10 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:10 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:10 smithi089 ceph-mon[95031]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2023-10-26T06:39:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:10 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3212295319' entity='client.admin' 2023-10-26T06:39:10.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:10 smithi089 ceph-mon[95031]: Updating smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-26T06:39:11.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 -- 172.21.15.89:0/2108333173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f59e0038b40 msgr2=0x7f59e003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 --2- 172.21.15.89:0/2108333173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f59e0038b40 0x7f59e003b000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f59e4006010 tx=0x7f59e400b260 comp rx=0 tx=0).stop 2023-10-26T06:39:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 -- 172.21.15.89:0/2108333173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f0101260 msgr2=0x7f59f0196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 --2- 172.21.15.89:0/2108333173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f0101260 0x7f59f0196660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00e9c0 tx=0x7f59dc00ecd0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 -- 172.21.15.89:0/2108333173 shutdown_connections 2023-10-26T06:39:11.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 --2- 172.21.15.89:0/2108333173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f59e0038b40 0x7f59e003b000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 --2- 172.21.15.89:0/2108333173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59f01057c0 0x7f59f019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 --2- 172.21.15.89:0/2108333173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59f01008a0 0x7f59f0196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 --2- 172.21.15.89:0/2108333173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f0101260 0x7f59f0196660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 -- 172.21.15.89:0/2108333173 >> 172.21.15.89:0/2108333173 conn(0x7f59f00779e0 msgr2=0x7f59f00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:11.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 -- 172.21.15.89:0/2108333173 shutdown_connections 2023-10-26T06:39:11.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.043+0000 7f59f6d78700 1 -- 172.21.15.89:0/2108333173 wait complete. 2023-10-26T06:39:11.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 -- 172.21.15.89:0/2018046656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 msgr2=0x7f46f010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:11.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 --2- 172.21.15.89:0/2018046656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f010cdc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009990 tx=0x7f46e4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 -- 172.21.15.89:0/2018046656 shutdown_connections 2023-10-26T06:39:11.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 --2- 172.21.15.89:0/2018046656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f010d300 0x7f46f010f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 --2- 172.21.15.89:0/2018046656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46f0068190 0x7f46f0068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 --2- 172.21.15.89:0/2018046656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f010cdc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 -- 172.21.15.89:0/2018046656 >> 172.21.15.89:0/2018046656 conn(0x7f46f0075790 msgr2=0x7f46f0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:11.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 -- 172.21.15.89:0/2018046656 shutdown_connections 2023-10-26T06:39:11.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.156+0000 7f46f58eb700 1 -- 172.21.15.89:0/2018046656 wait complete. 2023-10-26T06:39:11.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.159+0000 7f46f58eb700 1 Processor -- start 2023-10-26T06:39:11.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.159+0000 7f46f58eb700 1 -- start start 2023-10-26T06:39:11.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46f58eb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f0068190 0x7f46f010cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:11.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46f58eb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f0109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:11.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46f58eb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46f010d300 0x7f46f010a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:11.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46f58eb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f010a980 con 0x7f46f0068ab0 2023-10-26T06:39:11.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46f58eb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f010aaf0 con 0x7f46f0068190 2023-10-26T06:39:11.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46f58eb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f01a8cd0 con 0x7f46f010d300 2023-10-26T06:39:11.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46ee7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f0109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:11.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46ef7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46f010d300 0x7f46f010a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:11.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46ee7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f0109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51206/0 (socket says 172.21.15.89:51206) 2023-10-26T06:39:11.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46ee7fc700 1 -- 172.21.15.89:0/333796618 learned_addr learned my addr 172.21.15.89:0/333796618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:11.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46ef7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46f010d300 0x7f46f010a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:49942/0 (socket says 172.21.15.89:49942) 2023-10-26T06:39:11.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.160+0000 7f46eeffd700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f0068190 0x7f46f010cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:11.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46ee7fc700 1 -- 172.21.15.89:0/333796618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46f010d300 msgr2=0x7f46f010a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:11.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46ee7fc700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46f010d300 0x7f46f010a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46ee7fc700 1 -- 172.21.15.89:0/333796618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f0068190 msgr2=0x7f46f010cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:11.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46ee7fc700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f0068190 0x7f46f010cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46ee7fc700 1 -- 172.21.15.89:0/333796618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8009c40 con 0x7f46f0068ab0 2023-10-26T06:39:11.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46eeffd700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f0068190 0x7f46f010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:11.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46ee7fc700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f0109cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b650 tx=0x7f46e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:11.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46f48e9700 1 -- 172.21.15.89:0/333796618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e401d070 con 0x7f46f0068ab0 2023-10-26T06:39:11.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46f48e9700 1 -- 172.21.15.89:0/333796618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(23 keys) v1 ==== 1012+0+0 (secure 0 0 0) 0x7f46e4022470 con 0x7f46f0068ab0 2023-10-26T06:39:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e40095f0 con 0x7f46f0068ab0 2023-10-26T06:39:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.161+0000 7f46f48e9700 1 -- 172.21.15.89:0/333796618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4017650 con 0x7f46f0068ab0 2023-10-26T06:39:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.162+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f01a91d0 con 0x7f46f0068ab0 2023-10-26T06:39:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.163+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f0104110 con 0x7f46f0068ab0 2023-10-26T06:39:11.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.163+0000 7f46f48e9700 1 -- 172.21.15.89:0/333796618 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f46e4022850 con 0x7f46f0068ab0 2023-10-26T06:39:11.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.163+0000 7f46f48e9700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46d8038b40 0x7f46d803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:11.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.163+0000 7f46f48e9700 1 -- 172.21.15.89:0/333796618 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f46e404cab0 con 0x7f46f0068ab0 2023-10-26T06:39:11.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.166+0000 7f46eeffd700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46d8038b40 0x7f46d803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:11.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.167+0000 7f46eeffd700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46d8038b40 0x7f46d803b000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f46e0011490 tx=0x7f46e0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:11.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.168+0000 7f46f48e9700 1 -- 172.21.15.89:0/333796618 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f46e402a770 con 0x7f46f0068ab0 2023-10-26T06:39:11.276 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:11.278 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:11.312 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.495+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7f46f010b190 con 0x7f46f0068ab0 2023-10-26T06:39:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.503+0000 7f46f48e9700 1 -- 172.21.15.89:0/333796618 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v12)=0 v12) v1 ==== 108+0+0 (secure 0 0 0) 0x7f46e4027020 con 0x7f46f0068ab0 2023-10-26T06:39:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.506+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46d8038b40 msgr2=0x7f46d803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.506+0000 7f46f58eb700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46d8038b40 0x7f46d803b000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f46e0011490 tx=0x7f46e0009260 comp rx=0 tx=0).stop 2023-10-26T06:39:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.506+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 msgr2=0x7f46f0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.506+0000 7f46f58eb700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f0109cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b650 tx=0x7f46e4005740 comp rx=0 tx=0).stop 2023-10-26T06:39:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.510+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 shutdown_connections 2023-10-26T06:39:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.510+0000 7f46f58eb700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46d8038b40 0x7f46d803b000 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.510+0000 7f46f58eb700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46f010d300 0x7f46f010a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.510+0000 7f46f58eb700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f0068190 0x7f46f010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.510+0000 7f46f58eb700 1 --2- 172.21.15.89:0/333796618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46f0068ab0 0x7f46f0109cb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:11.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.510+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 >> 172.21.15.89:0/333796618 conn(0x7f46f0075790 msgr2=0x7f46f010ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:11.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.511+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 shutdown_connections 2023-10-26T06:39:11.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:11.511+0000 7f46f58eb700 1 -- 172.21.15.89:0/333796618 wait complete. 2023-10-26T06:39:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:11.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-26T06:39:11.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:11.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:11.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: Deploying daemon mgr.z on smithi192 2023-10-26T06:39:11.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2108333173' entity='client.admin' 2023-10-26T06:39:11.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:11 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/333796618' entity='client.admin' 2023-10-26T06:39:12.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:12.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:12.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:12.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:12.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-26T06:39:12.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:12.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:12.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: Deploying daemon mgr.z on smithi192 2023-10-26T06:39:12.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2108333173' entity='client.admin' 2023-10-26T06:39:12.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:11 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/333796618' entity='client.admin' 2023-10-26T06:39:12.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:12.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:12.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:12.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:12.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-26T06:39:12.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:12.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:12.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: Deploying daemon mgr.z on smithi192 2023-10-26T06:39:12.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2108333173' entity='client.admin' 2023-10-26T06:39:12.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:11 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/333796618' entity='client.admin' 2023-10-26T06:39:12.396 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:12 smithi192 systemd[1]: Starting Ceph mgr.z for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:39:12.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 -- 172.21.15.89:0/2381049020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324072a20 msgr2=0x7f5324072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 --2- 172.21.15.89:0/2381049020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324072a20 0x7f5324072e00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5314009990 tx=0x7f5314009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 -- 172.21.15.89:0/2381049020 shutdown_connections 2023-10-26T06:39:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 --2- 172.21.15.89:0/2381049020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324072a20 0x7f5324072e00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 --2- 172.21.15.89:0/2381049020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 0x7f532410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 --2- 172.21.15.89:0/2381049020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5324073340 0x7f532410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 -- 172.21.15.89:0/2381049020 >> 172.21.15.89:0/2381049020 conn(0x7f53240779e0 msgr2=0x7f5324077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:12.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 -- 172.21.15.89:0/2381049020 shutdown_connections 2023-10-26T06:39:12.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.532+0000 7f5329794700 1 -- 172.21.15.89:0/2381049020 wait complete. 2023-10-26T06:39:12.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 Processor -- start 2023-10-26T06:39:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 -- start start 2023-10-26T06:39:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5324072a20 0x7f53241984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324073340 0x7f5324198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 0x7f532419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:12.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419d3d0 con 0x7f5324073340 2023-10-26T06:39:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419d540 con 0x7f5324072a20 2023-10-26T06:39:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.533+0000 7f5329794700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419d700 con 0x7f532410d250 2023-10-26T06:39:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 0x7f532419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53227fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324073340 0x7f5324198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f5322ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5324072a20 0x7f53241984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:12.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 0x7f532419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:49966/0 (socket says 172.21.15.89:49966) 2023-10-26T06:39:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 -- 172.21.15.89:0/3863284864 learned_addr learned my addr 172.21.15.89:0/3863284864 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 -- 172.21.15.89:0/3863284864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5324072a20 msgr2=0x7f53241984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5324072a20 0x7f53241984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 -- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324073340 msgr2=0x7f5324198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:12.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324073340 0x7f5324198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 -- 172.21.15.89:0/3863284864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53140095f0 con 0x7f532410d250 2023-10-26T06:39:12.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53227fc700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324073340 0x7f5324198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:12.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.534+0000 7f53237fe700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 0x7f532419cdc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f531c007ee0 tx=0x7f531c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:12.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.535+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c004db0 con 0x7f532410d250 2023-10-26T06:39:12.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.535+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(23 keys) v1 ==== 1012+0+0 (secure 0 0 0) 0x7f531c00f040 con 0x7f532410d250 2023-10-26T06:39:12.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.535+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532419d9e0 con 0x7f532410d250 2023-10-26T06:39:12.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.535+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c005630 con 0x7f532410d250 2023-10-26T06:39:12.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.535+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53241a5240 con 0x7f532410d250 2023-10-26T06:39:12.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.536+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f531c005790 con 0x7f532410d250 2023-10-26T06:39:12.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.536+0000 7f530bfff700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f530c038b40 0x7f530c03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.536+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5324108460 con 0x7f532410d250 2023-10-26T06:39:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.536+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f531c04ae90 con 0x7f532410d250 2023-10-26T06:39:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.536+0000 7f5322ffd700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f530c038b40 0x7f530c03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.537+0000 7f5322ffd700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f530c038b40 0x7f530c03b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f531400b2e0 tx=0x7f53140058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:12.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.542+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f531c014360 con 0x7f532410d250 2023-10-26T06:39:12.713 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:12 smithi192 podman[134654]: 2023-10-26T06:39:12.730 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.777+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f532404e4f0 con 0x7f532410d250 2023-10-26T06:39:12.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.782+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 7 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7f531c013eb0 con 0x7f532410d250 2023-10-26T06:39:12.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.783+0000 7f530bfff700 1 -- 172.21.15.89:0/3863284864 <== mon.1 v2:172.21.15.192:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) v1 ==== 123+0+0 (secure 0 0 0) 0x7f531c021120 con 0x7f532410d250 2023-10-26T06:39:12.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f530c038b40 msgr2=0x7f530c03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:12.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f530c038b40 0x7f530c03b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f531400b2e0 tx=0x7f53140058e0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 msgr2=0x7f532419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:12.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 0x7f532419cdc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f531c007ee0 tx=0x7f531c00d3b0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 shutdown_connections 2023-10-26T06:39:12.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f530c038b40 0x7f530c03b000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f532410d250 0x7f532419cdc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5324072a20 0x7f53241984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 --2- 172.21.15.89:0/3863284864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5324073340 0x7f5324198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:12.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.786+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 >> 172.21.15.89:0/3863284864 conn(0x7f53240779e0 msgr2=0x7f53240fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:12.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.787+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 shutdown_connections 2023-10-26T06:39:12.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.787+0000 7f5329794700 1 -- 172.21.15.89:0/3863284864 wait complete. 2023-10-26T06:39:13.007 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:12 smithi192 bash[134654]: bb5a7ff15e5bcd735d3dd3c626a651a5da0eb732d2b177c67a78c6b074f9eb96 2023-10-26T06:39:13.007 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:12 smithi192 ceph-mgr[134693]: -- 172.21.15.192:0/3902955962 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643473a31e0 con 0x564347378800 2023-10-26T06:39:13.007 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:12 smithi192 systemd[1]: Started Ceph mgr.z for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:39:13.007 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:12.834+0000 7fd73554b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T06:39:13.008 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:12.916+0000 7fd73554b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T06:39:13.008 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:13.005+0000 7fd73554b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T06:39:13.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 -- 172.21.15.89:0/363530217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 msgr2=0x7f4314105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 --2- 172.21.15.89:0/363530217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 0x7f4314105ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43100098f0 tx=0x7f4310009c00 comp rx=0 tx=0).stop 2023-10-26T06:39:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 -- 172.21.15.89:0/363530217 shutdown_connections 2023-10-26T06:39:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 --2- 172.21.15.89:0/363530217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 0x7f4314105ba0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 --2- 172.21.15.89:0/363530217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4314101260 0x7f431410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 --2- 172.21.15.89:0/363530217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43141008a0 0x7f4314100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 -- 172.21.15.89:0/363530217 >> 172.21.15.89:0/363530217 conn(0x7f43140779e0 msgr2=0x7f4314077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 -- 172.21.15.89:0/363530217 shutdown_connections 2023-10-26T06:39:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.894+0000 7f431c480700 1 -- 172.21.15.89:0/363530217 wait complete. 2023-10-26T06:39:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.895+0000 7f431c480700 1 Processor -- start 2023-10-26T06:39:13.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.895+0000 7f431c480700 1 -- start start 2023-10-26T06:39:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43141008a0 0x7f4314072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4314101260 0x7f4314073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 0x7f4314073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141a4e90 con 0x7f43141057c0 2023-10-26T06:39:13.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141a5000 con 0x7f4314101260 2023-10-26T06:39:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141a51c0 con 0x7f43141008a0 2023-10-26T06:39:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43141008a0 0x7f4314072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43141008a0 0x7f4314072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:49972/0 (socket says 172.21.15.89:49972) 2023-10-26T06:39:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 -- 172.21.15.89:0/2360651519 learned_addr learned my addr 172.21.15.89:0/2360651519 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:13.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f4319a1b700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4314101260 0x7f4314073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431aa1d700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 0x7f4314073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 -- 172.21.15.89:0/2360651519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4314101260 msgr2=0x7f4314073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4314101260 0x7f4314073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 -- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 msgr2=0x7f4314073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:13.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 0x7f4314073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 -- 172.21.15.89:0/2360651519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43040095f0 con 0x7f43141008a0 2023-10-26T06:39:13.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431a21c700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43141008a0 0x7f4314072af0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4310009230 tx=0x7f4310010ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:13.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431001c070 con 0x7f43141008a0 2023-10-26T06:39:13.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(24 keys) v1 ==== 1043+0+0 (secure 0 0 0) 0x7f431000b9e0 con 0x7f43141008a0 2023-10-26T06:39:13.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4310009520 con 0x7f43141008a0 2023-10-26T06:39:13.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310003680 con 0x7f43141008a0 2023-10-26T06:39:13.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.896+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43141a56c0 con 0x7f43141008a0 2023-10-26T06:39:13.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.898+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4314108d80 con 0x7f43141008a0 2023-10-26T06:39:13.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.898+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f4310020070 con 0x7f43141008a0 2023-10-26T06:39:13.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.899+0000 7f430b7fe700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4300038b40 0x7f430003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.899+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f431004bff0 con 0x7f43141008a0 2023-10-26T06:39:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.899+0000 7f4319a1b700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4300038b40 0x7f430003b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.900+0000 7f4319a1b700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4300038b40 0x7f430003b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4314074220 tx=0x7f4304009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:13.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:12.904+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f431000ebd0 con 0x7f43141008a0 2023-10-26T06:39:13.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.145+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) v1 -- 0x7f4314101f30 con 0x7f43141008a0 2023-10-26T06:39:13.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.151+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 7 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f431000ebd0 con 0x7f43141008a0 2023-10-26T06:39:13.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.151+0000 7f430b7fe700 1 -- 172.21.15.89:0/2360651519 <== mon.1 v2:172.21.15.192:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4310005040 con 0x7f43141008a0 2023-10-26T06:39:13.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4300038b40 msgr2=0x7f430003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:13.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4300038b40 0x7f430003b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4314074220 tx=0x7f4304009310 comp rx=0 tx=0).stop 2023-10-26T06:39:13.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43141008a0 msgr2=0x7f4314072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:13.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43141008a0 0x7f4314072af0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4310009230 tx=0x7f4310010ff0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 shutdown_connections 2023-10-26T06:39:13.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4300038b40 0x7f430003b000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f43141057c0 0x7f4314073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43141008a0 0x7f4314072af0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 --2- 172.21.15.89:0/2360651519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4314101260 0x7f4314073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:13.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 >> 172.21.15.89:0/2360651519 conn(0x7f43140779e0 msgr2=0x7f43140fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:13.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.153+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 shutdown_connections 2023-10-26T06:39:13.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:13.154+0000 7f431c480700 1 -- 172.21.15.89:0/2360651519 wait complete. 2023-10-26T06:39:13.270 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:13.098+0000 7fd73554b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T06:39:13.365 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:13.633 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:13.269+0000 7fd73554b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T06:39:13.634 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:13.632+0000 7fd73554b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T06:39:13.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:13.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:13.895 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:13.767+0000 7fd73554b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T06:39:13.895 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:13.857+0000 7fd73554b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T06:39:13.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:13.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:13.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:13.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:13.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:13.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-26T06:39:13.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:13.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:13.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: Deploying daemon mgr.y on smithi135 2023-10-26T06:39:13.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:14.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:14.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:14.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:14.042 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:14.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:14.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-26T06:39:14.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:14.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:14.043 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: Deploying daemon mgr.y on smithi135 2023-10-26T06:39:14.044 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:13 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:14.100 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:14.100 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:14.100 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:14.100 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-26T06:39:14.101 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:14.101 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:14.101 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: Deploying daemon mgr.y on smithi135 2023-10-26T06:39:14.101 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:13 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:14.377 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:13.952+0000 7fd73554b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T06:39:14.589 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:14 smithi135 systemd[1]: Starting Ceph mgr.y for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:39:14.645 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:14 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:14.376+0000 7fd73554b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T06:39:14.856 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:14 smithi135 podman[133734]: 2023-10-26T06:39:14.856 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:14 smithi135 ceph-mgr[133773]: -- 172.21.15.135:0/2359384285 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55655183b1e0 con 0x556551811000 2023-10-26T06:39:15.124 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:14 smithi135 bash[133734]: a80388b3afbef3b7aa1e9f210e5eac8cb1e191dc92e172db32ca0c83fbfa893e 2023-10-26T06:39:15.124 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:14 smithi135 systemd[1]: Started Ceph mgr.y for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:39:15.124 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:14 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:14.952+0000 7f8369cce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T06:39:15.125 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:15.034+0000 7f8369cce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T06:39:15.125 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:15.122+0000 7f8369cce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T06:39:15.395 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:15.040+0000 7fd73554b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T06:39:15.395 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:15.121+0000 7fd73554b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T06:39:15.404 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:15.222+0000 7f8369cce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T06:39:15.405 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:15.404+0000 7f8369cce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T06:39:15.497 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi089') 2023-10-26T06:39:15.497 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi089') 2023-10-26T06:39:15.498 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi089') 2023-10-26T06:39:15.498 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi089') 2023-10-26T06:39:15.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 -- 172.21.15.89:0/3824192864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 msgr2=0x7fae8c10cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:15.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 --2- 172.21.15.89:0/3824192864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 0x7fae8c10cd10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae800098f0 tx=0x7fae80009c00 comp rx=0 tx=0).stop 2023-10-26T06:39:15.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 -- 172.21.15.89:0/3824192864 shutdown_connections 2023-10-26T06:39:15.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 --2- 172.21.15.89:0/3824192864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10d250 0x7fae8c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 --2- 172.21.15.89:0/3824192864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c072a20 0x7fae8c072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 --2- 172.21.15.89:0/3824192864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 0x7fae8c10cd10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 -- 172.21.15.89:0/3824192864 >> 172.21.15.89:0/3824192864 conn(0x7fae8c0779e0 msgr2=0x7fae8c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:15.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 -- 172.21.15.89:0/3824192864 shutdown_connections 2023-10-26T06:39:15.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.369+0000 7fae91f58700 1 -- 172.21.15.89:0/3824192864 wait complete. 2023-10-26T06:39:15.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 Processor -- start 2023-10-26T06:39:15.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 -- start start 2023-10-26T06:39:15.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c072a20 0x7fae8c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:15.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 0x7fae8c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:15.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10d250 0x7fae8c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:15.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c19dd30 con 0x7fae8c073340 2023-10-26T06:39:15.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c19dea0 con 0x7fae8c072a20 2023-10-26T06:39:15.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.371+0000 7fae91f58700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c19e060 con 0x7fae8c10d250 2023-10-26T06:39:15.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c072a20 0x7fae8c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:15.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10d250 0x7fae8c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:15.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 0x7fae8c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:15.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c072a20 0x7fae8c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:57754/0 (socket says 172.21.15.89:57754) 2023-10-26T06:39:15.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8b7fe700 1 -- 172.21.15.89:0/3494935671 learned_addr learned my addr 172.21.15.89:0/3494935671 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:15.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8bfff700 1 -- 172.21.15.89:0/3494935671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c072a20 msgr2=0x7fae8c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:15.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8bfff700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c072a20 0x7fae8c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8bfff700 1 -- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 msgr2=0x7fae8c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:15.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8bfff700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 0x7fae8c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8bfff700 1 -- 172.21.15.89:0/3494935671 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009520 con 0x7fae8c10d250 2023-10-26T06:39:15.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8affd700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 0x7fae8c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:15.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.372+0000 7fae8bfff700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10d250 0x7fae8c19d720 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae8400ef90 tx=0x7fae8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:15.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.373+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84018070 con 0x7fae8c10d250 2023-10-26T06:39:15.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.373+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae8400f040 con 0x7fae8c10d250 2023-10-26T06:39:15.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.373+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c19e2e0 con 0x7fae8c10d250 2023-10-26T06:39:15.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.373+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400cae0 con 0x7fae8c10d250 2023-10-26T06:39:15.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.373+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c1a5ba0 con 0x7fae8c10d250 2023-10-26T06:39:15.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.374+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7fae8400cd00 con 0x7fae8c10d250 2023-10-26T06:39:15.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.374+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c108460 con 0x7fae8c10d250 2023-10-26T06:39:15.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.375+0000 7fae88ff9700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae74038b40 0x7fae7403b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:15.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.375+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fae8404b120 con 0x7fae8c10d250 2023-10-26T06:39:15.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.375+0000 7fae8b7fe700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae74038b40 0x7fae7403b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:15.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.375+0000 7fae8b7fe700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae74038b40 0x7fae7403b000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae7c005950 tx=0x7fae7c009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:15.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.380+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae8407e100 con 0x7fae8c10d250 2023-10-26T06:39:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.665+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fae8c04e530 con 0x7fae8c10d250 2023-10-26T06:39:15.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.671+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae8401ce20 con 0x7fae8c10d250 2023-10-26T06:39:15.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.671+0000 7fae88ff9700 1 -- 172.21.15.89:0/3494935671 <== mon.1 v2:172.21.15.192:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) v1 ==== 123+0+0 (secure 0 0 0) 0x7fae8c04e530 con 0x7fae8c10d250 2023-10-26T06:39:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae74038b40 msgr2=0x7fae7403b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae74038b40 0x7fae7403b000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae7c005950 tx=0x7fae7c009310 comp rx=0 tx=0).stop 2023-10-26T06:39:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10d250 msgr2=0x7fae8c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10d250 0x7fae8c19d720 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae8400ef90 tx=0x7fae8400c3a0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 shutdown_connections 2023-10-26T06:39:15.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae74038b40 0x7fae7403b000 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10d250 0x7fae8c19d720 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae8c072a20 0x7fae8c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 --2- 172.21.15.89:0/3494935671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae8c073340 0x7fae8c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 >> 172.21.15.89:0/3494935671 conn(0x7fae8c0779e0 msgr2=0x7fae8c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 shutdown_connections 2023-10-26T06:39:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.674+0000 7fae91f58700 1 -- 172.21.15.89:0/3494935671 wait complete. 2023-10-26T06:39:16.122 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:15.782+0000 7f8369cce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T06:39:16.122 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:15.920+0000 7f8369cce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T06:39:16.123 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:16 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:16.017+0000 7f8369cce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T06:39:16.123 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:16 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:16.121+0000 7f8369cce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T06:39:16.123 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.123 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.124 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.124 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.124 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:16.124 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:16.124 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:15 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:16.215 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:15.932+0000 7fd73554b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:39:16.215 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:16.031+0000 7fd73554b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T06:39:16.215 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.216 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.216 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.216 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.216 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:16.217 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:16.217 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:15 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:16.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:15 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:15 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:15 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:15 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:16.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:15 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:16.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:15 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:16.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:15 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:16.510 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:16.213+0000 7fd73554b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:39:16.511 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:16.509+0000 7fd73554b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T06:39:16.895 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:16.747+0000 7fd73554b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T06:39:16.967 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:16 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:16.569+0000 7f8369cce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T06:39:17.161 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:16.894+0000 7fd73554b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T06:39:17.695 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:17.247+0000 7f8369cce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T06:39:17.695 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:17.327+0000 7f8369cce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T06:39:17.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 -- 172.21.15.89:0/2781494918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18141008a0 msgr2=0x7f1814100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 --2- 172.21.15.89:0/2781494918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18141008a0 0x7f1814100d20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1804009990 tx=0x7f1804009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 -- 172.21.15.89:0/2781494918 shutdown_connections 2023-10-26T06:39:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 --2- 172.21.15.89:0/2781494918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1814101260 0x7f181410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 --2- 172.21.15.89:0/2781494918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 0x7f1814105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 --2- 172.21.15.89:0/2781494918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f18141008a0 0x7f1814100d20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 -- 172.21.15.89:0/2781494918 >> 172.21.15.89:0/2781494918 conn(0x7f18140779e0 msgr2=0x7f1814077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:17.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 -- 172.21.15.89:0/2781494918 shutdown_connections 2023-10-26T06:39:17.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.431+0000 7f181c44e700 1 -- 172.21.15.89:0/2781494918 wait complete. 2023-10-26T06:39:17.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.433+0000 7f181c44e700 1 Processor -- start 2023-10-26T06:39:17.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.433+0000 7f181c44e700 1 -- start start 2023-10-26T06:39:17.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181c44e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18141008a0 0x7f181419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:17.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181c44e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1814101260 0x7f181419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:17.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181c44e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 0x7f1814196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:17.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181c44e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1814197160 con 0x7f1814101260 2023-10-26T06:39:17.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181c44e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141972d0 con 0x7f18141008a0 2023-10-26T06:39:17.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181c44e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1814197490 con 0x7f18141057c0 2023-10-26T06:39:17.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 0x7f1814196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:17.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 0x7f1814196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50002/0 (socket says 172.21.15.89:50002) 2023-10-26T06:39:17.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 -- 172.21.15.89:0/2979803180 learned_addr learned my addr 172.21.15.89:0/2979803180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:17.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a1ea700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18141008a0 0x7f181419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:17.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f18199e9700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1814101260 0x7f181419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:17.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 -- 172.21.15.89:0/2979803180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18141008a0 msgr2=0x7f181419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:17.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18141008a0 0x7f181419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 -- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1814101260 msgr2=0x7f181419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1814101260 0x7f181419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a9eb700 1 -- 172.21.15.89:0/2979803180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040095f0 con 0x7f18141057c0 2023-10-26T06:39:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f181a1ea700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18141008a0 0x7f181419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.434+0000 7f18199e9700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1814101260 0x7f181419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:39:17.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.435+0000 7f181a9eb700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 0x7f1814196b00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f180c00efc0 tx=0x7f180c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.435+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c018070 con 0x7f18141057c0 2023-10-26T06:39:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.435+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1814197710 con 0x7f18141057c0 2023-10-26T06:39:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.435+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f180c00f040 con 0x7f18141057c0 2023-10-26T06:39:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.435+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00cbe0 con 0x7f18141057c0 2023-10-26T06:39:17.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.435+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181406a3e0 con 0x7f18141057c0 2023-10-26T06:39:17.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.436+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f180c00cd60 con 0x7f18141057c0 2023-10-26T06:39:17.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.436+0000 7f180b7fe700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1800038b40 0x7f180003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:17.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.436+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f180c04b0e0 con 0x7f18141057c0 2023-10-26T06:39:17.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.436+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1814108d80 con 0x7f18141057c0 2023-10-26T06:39:17.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.437+0000 7f181a1ea700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1800038b40 0x7f180003b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:17.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.437+0000 7f181a1ea700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1800038b40 0x7f180003b000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1814101930 tx=0x7f1810009190 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:17.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.442+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f180c0040f0 con 0x7f18141057c0 2023-10-26T06:39:17.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.671+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180c039e90 con 0x7f18141057c0 2023-10-26T06:39:17.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.710+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f181404e4c0 con 0x7f18141057c0 2023-10-26T06:39:17.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.717+0000 7f180b7fe700 1 -- 172.21.15.89:0/2979803180 <== mon.1 v2:172.21.15.192:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v16)=0 v16) v1 ==== 127+0+0 (secure 0 0 0) 0x7f180c01c3c0 con 0x7f18141057c0 2023-10-26T06:39:17.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1800038b40 msgr2=0x7f180003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:17.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1800038b40 0x7f180003b000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1814101930 tx=0x7f1810009190 comp rx=0 tx=0).stop 2023-10-26T06:39:17.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 msgr2=0x7f1814196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:17.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 0x7f1814196b00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f180c00efc0 tx=0x7f180c00c3a0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 shutdown_connections 2023-10-26T06:39:17.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1800038b40 0x7f180003b000 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1814101930 tx=0x7f1810009190 comp rx=0 tx=0).stop 2023-10-26T06:39:17.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18141057c0 0x7f1814196b00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18141008a0 0x7f181419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 --2- 172.21.15.89:0/2979803180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1814101260 0x7f181419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:17.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 >> 172.21.15.89:0/2979803180 conn(0x7f18140779e0 msgr2=0x7f18140fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:17.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.719+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 shutdown_connections 2023-10-26T06:39:17.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:15.720+0000 7f181c44e700 1 -- 172.21.15.89:0/2979803180 wait complete. 2023-10-26T06:39:17.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: Reconfiguring mgr.x (unknown last config time)... 2023-10-26T06:39:17.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:17.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:17.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:17.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:17 smithi135 ceph-mon[131403]: Reconfiguring daemon mgr.x on smithi089 2023-10-26T06:39:17.993 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:17.160+0000 7fd73554b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T06:39:17.994 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:17.861+0000 7fd73554b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T06:39:17.994 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:17.992+0000 7fd73554b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T06:39:17.994 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.994 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:17.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:17.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:17.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: Reconfiguring mgr.x (unknown last config time)... 2023-10-26T06:39:17.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:17.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:17.996 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:17.997 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:17 smithi192 ceph-mon[131661]: Reconfiguring daemon mgr.x on smithi089 2023-10-26T06:39:18.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:18.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: Reconfiguring mgr.x (unknown last config time)... 2023-10-26T06:39:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-26T06:39:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-26T06:39:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:18.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:17 smithi089 ceph-mon[95031]: Reconfiguring daemon mgr.x on smithi089 2023-10-26T06:39:18.301 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:18 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:18.186+0000 7fd73554b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T06:39:18.363 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:18 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:18.098+0000 7f8369cce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:39:18.363 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:18 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:18.189+0000 7f8369cce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T06:39:18.641 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:18 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:18.361+0000 7f8369cce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T06:39:18.645 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:18 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:18.300+0000 7fd73554b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T06:39:18.645 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:18 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:18.519+0000 7fd73554b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T06:39:18.967 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:18 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:18.639+0000 7f8369cce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T06:39:18.967 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:18 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:18.867+0000 7f8369cce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T06:39:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/3561623009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9e01008a0 msgr2=0x7fd9e0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/3561623009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9e01008a0 0x7fd9e0100d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009990 tx=0x7fd9d4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/3561623009 shutdown_connections 2023-10-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/3561623009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e01057c0 0x7fd9e0105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/3561623009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e0101260 0x7fd9e010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/3561623009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9e01008a0 0x7fd9e0100d20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/3561623009 >> 172.21.15.89:0/3561623009 conn(0x7fd9e00779e0 msgr2=0x7fd9e0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/3561623009 shutdown_connections 2023-10-26T06:39:19.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.226+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/3561623009 wait complete. 2023-10-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.227+0000 7fd9e4a7c700 1 Processor -- start 2023-10-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9e4a7c700 1 -- start start 2023-10-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9e4a7c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e01008a0 0x7fd9e0198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:19.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9e4a7c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9e0101260 0x7fd9e0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9e4a7c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e01057c0 0x7fd9e019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9e4a7c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e019dd30 con 0x7fd9e01057c0 2023-10-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9e4a7c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e019dea0 con 0x7fd9e01008a0 2023-10-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9e4a7c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e019e060 con 0x7fd9e0101260 2023-10-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.228+0000 7fd9de59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e01008a0 0x7fd9e0198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e01008a0 0x7fd9e0198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:57794/0 (socket says 172.21.15.89:57794) 2023-10-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 -- 172.21.15.89:0/2593829866 learned_addr learned my addr 172.21.15.89:0/2593829866 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9ded9d700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e01057c0 0x7fd9e019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 -- 172.21.15.89:0/2593829866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9e0101260 msgr2=0x7fd9e0199370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T06:39:19.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9e0101260 0x7fd9e0199370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 -- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e01057c0 msgr2=0x7fd9e019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e01057c0 0x7fd9e019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 -- 172.21.15.89:0/2593829866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40095f0 con 0x7fd9e01008a0 2023-10-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9ded9d700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e01057c0 0x7fd9e019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.229+0000 7fd9de59c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e01008a0 0x7fd9e0198e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000e970 tx=0x7fd9d000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:19.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.230+0000 7fd9cf7fe700 1 -- 172.21.15.89:0/2593829866 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0009790 con 0x7fd9e01008a0 2023-10-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.230+0000 7fd9cf7fe700 1 -- 172.21.15.89:0/2593829866 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d00044e0 con 0x7fd9e01008a0 2023-10-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.230+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e019e340 con 0x7fd9e01008a0 2023-10-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.230+0000 7fd9cf7fe700 1 -- 172.21.15.89:0/2593829866 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0003e30 con 0x7fd9e01008a0 2023-10-26T06:39:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.230+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e01a5ba0 con 0x7fd9e01008a0 2023-10-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.231+0000 7fd9cf7fe700 1 -- 172.21.15.89:0/2593829866 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) v1 ==== 45652+0+0 (secure 0 0 0) 0x7fd9d0018430 con 0x7fd9e01008a0 2023-10-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.232+0000 7fd9cf7fe700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9c8038b40 0x7fd9c803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.232+0000 7fd9cf7fe700 1 -- 172.21.15.89:0/2593829866 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd9d0014070 con 0x7fd9e01008a0 2023-10-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.232+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e0101f30 con 0x7fd9e01008a0 2023-10-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.234+0000 7fd9ddd9b700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9c8038b40 0x7fd9c803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.235+0000 7fd9ddd9b700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9c8038b40 0x7fd9c803b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098e0 tx=0x7fd9d4000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:19.239 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi089') 2023-10-26T06:39:19.240 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi089') 2023-10-26T06:39:19.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.237+0000 7fd9cf7fe700 1 -- 172.21.15.89:0/2593829866 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9d000fe20 con 0x7fd9e01008a0 2023-10-26T06:39:19.279 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:19 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:19.014+0000 7f8369cce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T06:39:19.388 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.503+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7fd9e004e4c0 con 0x7fd9e01008a0 2023-10-26T06:39:19.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.510+0000 7fd9cf7fe700 1 -- 172.21.15.89:0/2593829866 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v17)=0 v17) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd9d000f3c0 con 0x7fd9e01008a0 2023-10-26T06:39:19.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9c8038b40 msgr2=0x7fd9c803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9c8038b40 0x7fd9c803b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40098e0 tx=0x7fd9d4000bc0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e01008a0 msgr2=0x7fd9e0198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e01008a0 0x7fd9e0198e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000e970 tx=0x7fd9d000ed30 comp rx=0 tx=0).stop 2023-10-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 shutdown_connections 2023-10-26T06:39:19.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9c8038b40 0x7fd9c803b000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9e01057c0 0x7fd9e019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9e01008a0 0x7fd9e0198e30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 --2- 172.21.15.89:0/2593829866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9e0101260 0x7fd9e0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 >> 172.21.15.89:0/2593829866 conn(0x7fd9e00779e0 msgr2=0x7fd9e010ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:19.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 shutdown_connections 2023-10-26T06:39:19.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:19.513+0000 7fd9e4a7c700 1 -- 172.21.15.89:0/2593829866 wait complete. 2023-10-26T06:39:19.717 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:19 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:19.277+0000 7f8369cce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T06:39:19.895 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:39:19 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z[134669]: 2023-10-26T06:39:19.396+0000 7fd73554b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T06:39:20.149 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:20 smithi089 ceph-mon[95031]: Standby manager daemon z started 2023-10-26T06:39:20.150 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:20 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-26T06:39:20.150 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:20 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-26T06:39:20.150 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:20 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-26T06:39:20.150 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:20 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-26T06:39:20.150 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:20 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:20.303 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:19 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:19.978+0000 7f8369cce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T06:39:20.303 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:20.109+0000 7f8369cce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T06:39:20.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-mon[131403]: Standby manager daemon z started 2023-10-26T06:39:20.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-26T06:39:20.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-26T06:39:20.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-26T06:39:20.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-26T06:39:20.305 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:20.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:20 smithi192 ceph-mon[131661]: Standby manager daemon z started 2023-10-26T06:39:20.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:20 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-26T06:39:20.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:20 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-26T06:39:20.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:20 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-26T06:39:20.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:20 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.192:0/194026665' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-26T06:39:20.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:20 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:20.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.602+0000 7fa18bbd9700 1 -- 172.21.15.89:0/2988355903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 msgr2=0x7fa18410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.602+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/2988355903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18410f640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b0a0 tx=0x7fa17c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.603+0000 7fa18bbd9700 1 -- 172.21.15.89:0/2988355903 shutdown_connections 2023-10-26T06:39:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.603+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/2988355903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18410f640 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.603+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/2988355903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa184072a20 0x7fa184072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.603+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/2988355903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa184073340 0x7fa18410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.603+0000 7fa18bbd9700 1 -- 172.21.15.89:0/2988355903 >> 172.21.15.89:0/2988355903 conn(0x7fa1840779e0 msgr2=0x7fa184077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:20.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.603+0000 7fa18bbd9700 1 -- 172.21.15.89:0/2988355903 shutdown_connections 2023-10-26T06:39:20.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.603+0000 7fa18bbd9700 1 -- 172.21.15.89:0/2988355903 wait complete. 2023-10-26T06:39:20.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.604+0000 7fa18bbd9700 1 Processor -- start 2023-10-26T06:39:20.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.604+0000 7fa18bbd9700 1 -- start start 2023-10-26T06:39:20.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.604+0000 7fa18bbd9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa184072a20 0x7fa184199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.604+0000 7fa18bbd9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa184073340 0x7fa1841995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.604+0000 7fa18bbd9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18bbd9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18419df60 con 0x7fa184072a20 2023-10-26T06:39:20.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18bbd9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18419e0d0 con 0x7fa18410d250 2023-10-26T06:39:20.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18bbd9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18419e290 con 0x7fa184073340 2023-10-26T06:39:20.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa189975700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa184072a20 0x7fa184199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa189174700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa184073340 0x7fa1841995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18419d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:57944/0 (socket says 172.21.15.89:57944) 2023-10-26T06:39:20.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa189975700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa184072a20 0x7fa184199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46052/0 (socket says 172.21.15.89:46052) 2023-10-26T06:39:20.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 -- 172.21.15.89:0/877319694 learned_addr learned my addr 172.21.15.89:0/877319694 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:20.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 -- 172.21.15.89:0/877319694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa184073340 msgr2=0x7fa1841995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa184073340 0x7fa1841995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 -- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa184072a20 msgr2=0x7fa184199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa184072a20 0x7fa184199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.605+0000 7fa18a176700 1 -- 172.21.15.89:0/877319694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c009d00 con 0x7fa18410d250 2023-10-26T06:39:20.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.606+0000 7fa18a176700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18419d950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa17c006070 tx=0x7fa17c007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:20.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.606+0000 7fa17affd700 1 -- 172.21.15.89:0/877319694 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00e040 con 0x7fa18410d250 2023-10-26T06:39:20.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.606+0000 7fa17affd700 1 -- 172.21.15.89:0/877319694 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa17c01d470 con 0x7fa18410d250 2023-10-26T06:39:20.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.606+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18419e510 con 0x7fa18410d250 2023-10-26T06:39:20.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.606+0000 7fa17affd700 1 -- 172.21.15.89:0/877319694 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c01b6c0 con 0x7fa18410d250 2023-10-26T06:39:20.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.607+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1841a5dd0 con 0x7fa18410d250 2023-10-26T06:39:20.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.608+0000 7fa17affd700 1 -- 172.21.15.89:0/877319694 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7fa17c003a00 con 0x7fa18410d250 2023-10-26T06:39:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.608+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa184108460 con 0x7fa18410d250 2023-10-26T06:39:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.609+0000 7fa17affd700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa17006cfc0 0x7fa17006f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.609+0000 7fa17affd700 1 -- 172.21.15.89:0/877319694 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa17c0909e0 con 0x7fa18410d250 2023-10-26T06:39:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.609+0000 7fa189975700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa17006cfc0 0x7fa17006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.610+0000 7fa189975700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa17006cfc0 0x7fa17006f480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa180009c60 tx=0x7fa180009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:20.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.614+0000 7fa17affd700 1 -- 172.21.15.89:0/877319694 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa17c056cb0 con 0x7fa18410d250 2023-10-26T06:39:20.634 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:20.301+0000 7f8369cce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T06:39:20.634 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:20.414+0000 7f8369cce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T06:39:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.725+0000 7fa585b66700 1 -- 172.21.15.89:0/5465773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801029c0 msgr2=0x7fa580102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.725+0000 7fa585b66700 1 --2- 172.21.15.89:0/5465773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801029c0 0x7fa580102e40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa574009990 tx=0x7fa574009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.726+0000 7fa585b66700 1 -- 172.21.15.89:0/5465773 shutdown_connections 2023-10-26T06:39:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.726+0000 7fa585b66700 1 --2- 172.21.15.89:0/5465773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa580103380 0x7fa58010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.726+0000 7fa585b66700 1 --2- 172.21.15.89:0/5465773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801089c0 0x7fa580108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.726+0000 7fa585b66700 1 --2- 172.21.15.89:0/5465773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801029c0 0x7fa580102e40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.726+0000 7fa585b66700 1 -- 172.21.15.89:0/5465773 >> 172.21.15.89:0/5465773 conn(0x7fa5800fe4b0 msgr2=0x7fa5801008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.726+0000 7fa585b66700 1 -- 172.21.15.89:0/5465773 shutdown_connections 2023-10-26T06:39:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.726+0000 7fa585b66700 1 -- 172.21.15.89:0/5465773 wait complete. 2023-10-26T06:39:20.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 Processor -- start 2023-10-26T06:39:20.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 -- start start 2023-10-26T06:39:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801029c0 0x7fa580198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa580103380 0x7fa5801993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801089c0 0x7fa58019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58019dd60 con 0x7fa5801089c0 2023-10-26T06:39:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58019ded0 con 0x7fa5801029c0 2023-10-26T06:39:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa585b66700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58019e090 con 0x7fa580103380 2023-10-26T06:39:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa57f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801029c0 0x7fa580198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.727+0000 7fa57f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801029c0 0x7fa580198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:57958/0 (socket says 172.21.15.89:57958) 2023-10-26T06:39:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57f7fe700 1 -- 172.21.15.89:0/838986004 learned_addr learned my addr 172.21.15.89:0/838986004 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57effd700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa580103380 0x7fa5801993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57ffff700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801089c0 0x7fa58019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57f7fe700 1 -- 172.21.15.89:0/838986004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa580103380 msgr2=0x7fa5801993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57f7fe700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa580103380 0x7fa5801993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57f7fe700 1 -- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801089c0 msgr2=0x7fa58019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57f7fe700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801089c0 0x7fa58019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57f7fe700 1 -- 172.21.15.89:0/838986004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740095f0 con 0x7fa5801029c0 2023-10-26T06:39:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57f7fe700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801029c0 0x7fa580198e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa57000e9a0 tx=0x7fa57000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57cff9700 1 -- 172.21.15.89:0/838986004 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570009790 con 0x7fa5801029c0 2023-10-26T06:39:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57cff9700 1 -- 172.21.15.89:0/838986004 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa57000cbd0 con 0x7fa5801029c0 2023-10-26T06:39:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58019e370 con 0x7fa5801029c0 2023-10-26T06:39:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.728+0000 7fa57cff9700 1 -- 172.21.15.89:0/838986004 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570018680 con 0x7fa5801029c0 2023-10-26T06:39:20.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.729+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5801a5bd0 con 0x7fa5801029c0 2023-10-26T06:39:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.730+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58010afe0 con 0x7fa5801029c0 2023-10-26T06:39:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.730+0000 7fa57cff9700 1 -- 172.21.15.89:0/838986004 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7fa570010430 con 0x7fa5801029c0 2023-10-26T06:39:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.731+0000 7fa57cff9700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa56806cfc0 0x7fa56806f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.731+0000 7fa57cff9700 1 -- 172.21.15.89:0/838986004 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa570014070 con 0x7fa5801029c0 2023-10-26T06:39:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.731+0000 7fa57effd700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa56806cfc0 0x7fa56806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:20.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.732+0000 7fa57effd700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa56806cfc0 0x7fa56806f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa57400b2e0 tx=0x7fa574005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:20.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.736+0000 7fa57cff9700 1 -- 172.21.15.89:0/838986004 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa570052100 con 0x7fa5801029c0 2023-10-26T06:39:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.853+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7fa18404e4c0 con 0x7fa18410d250 2023-10-26T06:39:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.859+0000 7fa17affd700 1 -- 172.21.15.89:0/877319694 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v18)=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa17c003cb0 con 0x7fa18410d250 2023-10-26T06:39:20.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa17006cfc0 msgr2=0x7fa17006f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa17006cfc0 0x7fa17006f480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa180009c60 tx=0x7fa180009260 comp rx=0 tx=0).stop 2023-10-26T06:39:20.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 msgr2=0x7fa18419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18419d950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa17c006070 tx=0x7fa17c007860 comp rx=0 tx=0).stop 2023-10-26T06:39:20.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 shutdown_connections 2023-10-26T06:39:20.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa17006cfc0 0x7fa17006f480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa184072a20 0x7fa184199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa18410d250 0x7fa18419d950 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 --2- 172.21.15.89:0/877319694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa184073340 0x7fa1841995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 >> 172.21.15.89:0/877319694 conn(0x7fa1840779e0 msgr2=0x7fa1840fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 shutdown_connections 2023-10-26T06:39:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.862+0000 7fa18bbd9700 1 -- 172.21.15.89:0/877319694 wait complete. 2023-10-26T06:39:20.967 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:20 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:20.632+0000 7f8369cce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T06:39:20.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.971+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fa580104100 con 0x7fa5801029c0 2023-10-26T06:39:20.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.976+0000 7fa57cff9700 1 -- 172.21.15.89:0/838986004 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa570056980 con 0x7fa5801029c0 2023-10-26T06:39:20.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa56806cfc0 msgr2=0x7fa56806f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa56806cfc0 0x7fa56806f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa57400b2e0 tx=0x7fa574005fd0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801029c0 msgr2=0x7fa580198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801029c0 0x7fa580198e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa57000e9a0 tx=0x7fa57000ed60 comp rx=0 tx=0).stop 2023-10-26T06:39:20.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 shutdown_connections 2023-10-26T06:39:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa56806cfc0 0x7fa56806f480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5801089c0 0x7fa58019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801029c0 0x7fa580198e60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 --2- 172.21.15.89:0/838986004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa580103380 0x7fa5801993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:20.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.979+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 >> 172.21.15.89:0/838986004 conn(0x7fa5800fe4b0 msgr2=0x7fa5800fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:20.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.980+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 shutdown_connections 2023-10-26T06:39:20.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:20.980+0000 7fa585b66700 1 -- 172.21.15.89:0/838986004 wait complete. 2023-10-26T06:39:21.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:21 smithi089 ceph-mon[95031]: mgrmap e18: x(active, since 18s), standbys: z 2023-10-26T06:39:21.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:21 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-26T06:39:21.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:21 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:21.143 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:21 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:21 smithi192 ceph-mon[131661]: mgrmap e18: x(active, since 18s), standbys: z 2023-10-26T06:39:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:21 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-26T06:39:21.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:21 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:21.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:21 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:21 smithi135 ceph-mon[131403]: mgrmap e18: x(active, since 18s), standbys: z 2023-10-26T06:39:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:21 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-26T06:39:21.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:21 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:21.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:21 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:21.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 -- 172.21.15.89:0/3123693419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e4068b30 msgr2=0x7f84e410d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:21.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 --2- 172.21.15.89:0/3123693419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e4068b30 0x7f84e410d670 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00b0a0 tx=0x7f84dc00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:39:21.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 -- 172.21.15.89:0/3123693419 shutdown_connections 2023-10-26T06:39:21.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 --2- 172.21.15.89:0/3123693419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e4068b30 0x7f84e410d670 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:21.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 --2- 172.21.15.89:0/3123693419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e41067a0 0x7f84e4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:21.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 --2- 172.21.15.89:0/3123693419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4068190 0x7f84e40685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:21.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 -- 172.21.15.89:0/3123693419 >> 172.21.15.89:0/3123693419 conn(0x7f84e4075790 msgr2=0x7f84e4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:21.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 -- 172.21.15.89:0/3123693419 shutdown_connections 2023-10-26T06:39:21.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.777+0000 7f84eb591700 1 -- 172.21.15.89:0/3123693419 wait complete. 2023-10-26T06:39:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.778+0000 7f84eb591700 1 Processor -- start 2023-10-26T06:39:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.778+0000 7f84eb591700 1 -- start start 2023-10-26T06:39:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84eb591700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e4068190 0x7f84e4198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84eb591700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4068b30 0x7f84e4199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:21.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84eb591700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e41067a0 0x7f84e419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84eb591700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e419de50 con 0x7f84e41067a0 2023-10-26T06:39:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84eb591700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e419dfc0 con 0x7f84e4068190 2023-10-26T06:39:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84eb591700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e419e180 con 0x7f84e4068b30 2023-10-26T06:39:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e41067a0 0x7f84e419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:21.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e41067a0 0x7f84e419d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46098/0 (socket says 172.21.15.89:46098) 2023-10-26T06:39:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e932d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e4068190 0x7f84e4198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 -- 172.21.15.89:0/1361145840 learned_addr learned my addr 172.21.15.89:0/1361145840 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e8b2c700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4068b30 0x7f84e4199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 -- 172.21.15.89:0/1361145840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4068b30 msgr2=0x7f84e4199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4068b30 0x7f84e4199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:21.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 -- 172.21.15.89:0/1361145840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e4068190 msgr2=0x7f84e4198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:21.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e4068190 0x7f84e4198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:21.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.779+0000 7f84e9b2e700 1 -- 172.21.15.89:0/1361145840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc009d00 con 0x7f84e41067a0 2023-10-26T06:39:21.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.780+0000 7f84e9b2e700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e41067a0 0x7f84e419d840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00b070 tx=0x7f84dc007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:21.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.780+0000 7f84da7fc700 1 -- 172.21.15.89:0/1361145840 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc00e040 con 0x7f84e41067a0 2023-10-26T06:39:21.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.780+0000 7f84da7fc700 1 -- 172.21.15.89:0/1361145840 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84dc01d470 con 0x7f84e41067a0 2023-10-26T06:39:21.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.780+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e419e400 con 0x7f84e41067a0 2023-10-26T06:39:21.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.780+0000 7f84da7fc700 1 -- 172.21.15.89:0/1361145840 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0052a0 con 0x7f84e41067a0 2023-10-26T06:39:21.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.780+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e41a5cc0 con 0x7f84e41067a0 2023-10-26T06:39:21.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.781+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e40fed60 con 0x7f84e41067a0 2023-10-26T06:39:21.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.783+0000 7f84da7fc700 1 -- 172.21.15.89:0/1361145840 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) v1 ==== 90808+0+0 (secure 0 0 0) 0x7f84dc01d850 con 0x7f84e41067a0 2023-10-26T06:39:21.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.784+0000 7f84da7fc700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f84d006cfc0 0x7f84d006f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:21.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.784+0000 7f84da7fc700 1 -- 172.21.15.89:0/1361145840 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f84dc08c6f0 con 0x7f84e41067a0 2023-10-26T06:39:21.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.784+0000 7f84e932d700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f84d006cfc0 0x7f84d006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:21.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.784+0000 7f84e932d700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f84d006cfc0 0x7f84d006f480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007b60 tx=0x7f84e00073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:21.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:21.787+0000 7f84da7fc700 1 -- 172.21.15.89:0/1361145840 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f84dc057c90 con 0x7f84e41067a0 2023-10-26T06:39:21.967 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:39:21 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y[133749]: 2023-10-26T06:39:21.510+0000 7f8369cce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T06:39:22.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.014+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f84e4101ee0 con 0x7f84e41067a0 2023-10-26T06:39:22.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.018+0000 7f84da7fc700 1 -- 172.21.15.89:0/1361145840 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v20)=0 v20) v1 ==== 119+0+0 (secure 0 0 0) 0x7f84dc024350 con 0x7f84e41067a0 2023-10-26T06:39:22.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.020+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f84d006cfc0 msgr2=0x7f84d006f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:22.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.020+0000 7f84eb591700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f84d006cfc0 0x7f84d006f480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007b60 tx=0x7f84e00073d0 comp rx=0 tx=0).stop 2023-10-26T06:39:22.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.020+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e41067a0 msgr2=0x7f84e419d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:22.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.020+0000 7f84eb591700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e41067a0 0x7f84e419d840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00b070 tx=0x7f84dc007930 comp rx=0 tx=0).stop 2023-10-26T06:39:22.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 shutdown_connections 2023-10-26T06:39:22.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f84d006cfc0 0x7f84d006f480 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:22.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84e41067a0 0x7f84e419d840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:22.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84e4068190 0x7f84e4198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:22.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 --2- 172.21.15.89:0/1361145840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84e4068b30 0x7f84e4199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:22.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 >> 172.21.15.89:0/1361145840 conn(0x7f84e4075790 msgr2=0x7f84e410bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:22.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 shutdown_connections 2023-10-26T06:39:22.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.021+0000 7f84eb591700 1 -- 172.21.15.89:0/1361145840 wait complete. 2023-10-26T06:39:22.134 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:22 smithi089 ceph-mon[95031]: Standby manager daemon y started 2023-10-26T06:39:22.134 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:22 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-26T06:39:22.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:22 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-26T06:39:22.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:22 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-26T06:39:22.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:22 smithi089 ceph-mon[95031]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-26T06:39:22.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:22 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1361145840' entity='client.admin' 2023-10-26T06:39:22.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:22 smithi192 ceph-mon[131661]: Standby manager daemon y started 2023-10-26T06:39:22.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:22 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-26T06:39:22.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:22 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-26T06:39:22.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:22 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-26T06:39:22.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:22 smithi192 ceph-mon[131661]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-26T06:39:22.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:22 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1361145840' entity='client.admin' 2023-10-26T06:39:22.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:22 smithi135 ceph-mon[131403]: Standby manager daemon y started 2023-10-26T06:39:22.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:22 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-26T06:39:22.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:22 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-26T06:39:22.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:22 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-26T06:39:22.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:22 smithi135 ceph-mon[131403]: from='mgr.? 172.21.15.135:0/4112075607' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-26T06:39:22.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:22 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1361145840' entity='client.admin' 2023-10-26T06:39:23.117 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:23 smithi089 ceph-mon[95031]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:23.117 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:23 smithi089 ceph-mon[95031]: mgrmap e19: x(active, since 20s), standbys: z, y 2023-10-26T06:39:23.118 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:23 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-26T06:39:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.149+0000 7efd54235700 1 -- 172.21.15.89:0/12278745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd4c074770 msgr2=0x7efd4c072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.149+0000 7efd54235700 1 --2- 172.21.15.89:0/12278745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd4c074770 0x7efd4c072a20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efd48009990 tx=0x7efd48009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.150+0000 7efd54235700 1 -- 172.21.15.89:0/12278745 shutdown_connections 2023-10-26T06:39:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.150+0000 7efd54235700 1 --2- 172.21.15.89:0/12278745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd4c074770 0x7efd4c072a20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.150+0000 7efd54235700 1 --2- 172.21.15.89:0/12278745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c111570 0x7efd4c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.150+0000 7efd54235700 1 --2- 172.21.15.89:0/12278745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd4c072f60 0x7efd4c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.150+0000 7efd54235700 1 -- 172.21.15.89:0/12278745 >> 172.21.15.89:0/12278745 conn(0x7efd4c0779e0 msgr2=0x7efd4c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.150+0000 7efd54235700 1 -- 172.21.15.89:0/12278745 shutdown_connections 2023-10-26T06:39:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.150+0000 7efd54235700 1 -- 172.21.15.89:0/12278745 wait complete. 2023-10-26T06:39:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.151+0000 7efd54235700 1 Processor -- start 2023-10-26T06:39:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.151+0000 7efd54235700 1 -- start start 2023-10-26T06:39:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd54235700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd4c072f60 0x7efd4c19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd54235700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c074770 0x7efd4c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd54235700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd4c111570 0x7efd4c1a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd54235700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c1a21b0 con 0x7efd4c072f60 2023-10-26T06:39:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd54235700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c1a2320 con 0x7efd4c111570 2023-10-26T06:39:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd54235700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c1a24e0 con 0x7efd4c074770 2023-10-26T06:39:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd517d0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c074770 0x7efd4c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd517d0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c074770 0x7efd4c19d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41150/0 (socket says 172.21.15.89:41150) 2023-10-26T06:39:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd517d0700 1 -- 172.21.15.89:0/3303353241 learned_addr learned my addr 172.21.15.89:0/3303353241 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.152+0000 7efd527d2700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd4c111570 0x7efd4c1a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd517d0700 1 -- 172.21.15.89:0/3303353241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd4c111570 msgr2=0x7efd4c1a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd517d0700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd4c111570 0x7efd4c1a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd517d0700 1 -- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd4c072f60 msgr2=0x7efd4c19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd517d0700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd4c072f60 0x7efd4c19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd517d0700 1 -- 172.21.15.89:0/3303353241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480095f0 con 0x7efd4c074770 2023-10-26T06:39:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd527d2700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd4c111570 0x7efd4c1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd517d0700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c074770 0x7efd4c19d7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e9a0 tx=0x7efd3c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd42ffd700 1 -- 172.21.15.89:0/3303353241 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c009790 con 0x7efd4c074770 2023-10-26T06:39:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd42ffd700 1 -- 172.21.15.89:0/3303353241 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd3c00cbd0 con 0x7efd4c074770 2023-10-26T06:39:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.153+0000 7efd42ffd700 1 -- 172.21.15.89:0/3303353241 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c018680 con 0x7efd4c074770 2023-10-26T06:39:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.154+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c1a27c0 con 0x7efd4c074770 2023-10-26T06:39:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.154+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c1aa020 con 0x7efd4c074770 2023-10-26T06:39:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.155+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c0740b0 con 0x7efd4c074770 2023-10-26T06:39:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.158+0000 7efd42ffd700 1 -- 172.21.15.89:0/3303353241 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7efd3c00cd40 con 0x7efd4c074770 2023-10-26T06:39:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.159+0000 7efd42ffd700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd380a1540 0x7efd380a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.159+0000 7efd42ffd700 1 -- 172.21.15.89:0/3303353241 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7efd3c014070 con 0x7efd4c074770 2023-10-26T06:39:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.159+0000 7efd51fd1700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd380a1540 0x7efd380a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.160+0000 7efd51fd1700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd380a1540 0x7efd380a3a00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd480098e0 tx=0x7efd48005e80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.161+0000 7efd42ffd700 1 -- 172.21.15.89:0/3303353241 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd3c096620 con 0x7efd4c074770 2023-10-26T06:39:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:23 smithi192 ceph-mon[131661]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:23 smithi192 ceph-mon[131661]: mgrmap e19: x(active, since 20s), standbys: z, y 2023-10-26T06:39:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:23 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-26T06:39:23.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.414+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) v1 -- 0x7efd4c04e4c0 con 0x7efd4c074770 2023-10-26T06:39:23.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.420+0000 7efd42ffd700 1 -- 172.21.15.89:0/3303353241 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v22)=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7efd3c099c40 con 0x7efd4c074770 2023-10-26T06:39:23.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd380a1540 msgr2=0x7efd380a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd380a1540 0x7efd380a3a00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd480098e0 tx=0x7efd48005e80 comp rx=0 tx=0).stop 2023-10-26T06:39:23.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c074770 msgr2=0x7efd4c19d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c074770 0x7efd4c19d7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e9a0 tx=0x7efd3c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 shutdown_connections 2023-10-26T06:39:23.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd380a1540 0x7efd380a3a00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd4c072f60 0x7efd4c19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd4c111570 0x7efd4c1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 --2- 172.21.15.89:0/3303353241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd4c074770 0x7efd4c19d7f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 >> 172.21.15.89:0/3303353241 conn(0x7efd4c0779e0 msgr2=0x7efd4c1101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:23.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.423+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 shutdown_connections 2023-10-26T06:39:23.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.424+0000 7efd54235700 1 -- 172.21.15.89:0/3303353241 wait complete. 2023-10-26T06:39:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:23 smithi135 ceph-mon[131403]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:23 smithi135 ceph-mon[131403]: mgrmap e19: x(active, since 20s), standbys: z, y 2023-10-26T06:39:23.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:23 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-26T06:39:23.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.348+0000 7fd73ded4700 1 -- 172.21.15.89:0/139023891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738072a20 msgr2=0x7fd738072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.348+0000 7fd73ded4700 1 --2- 172.21.15.89:0/139023891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738072a20 0x7fd738072e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd728009990 tx=0x7fd728009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.348+0000 7fd73ded4700 1 -- 172.21.15.89:0/139023891 shutdown_connections 2023-10-26T06:39:23.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.348+0000 7fd73ded4700 1 --2- 172.21.15.89:0/139023891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738072a20 0x7fd738072e00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.348+0000 7fd73ded4700 1 --2- 172.21.15.89:0/139023891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd73810d250 0x7fd73810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.348+0000 7fd73ded4700 1 --2- 172.21.15.89:0/139023891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd738073340 0x7fd73810cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.348+0000 7fd73ded4700 1 -- 172.21.15.89:0/139023891 >> 172.21.15.89:0/139023891 conn(0x7fd7380779e0 msgr2=0x7fd738077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:23.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.349+0000 7fd73ded4700 1 -- 172.21.15.89:0/139023891 shutdown_connections 2023-10-26T06:39:23.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.349+0000 7fd73ded4700 1 -- 172.21.15.89:0/139023891 wait complete. 2023-10-26T06:39:23.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.350+0000 7fd73ded4700 1 Processor -- start 2023-10-26T06:39:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.351+0000 7fd73ded4700 1 -- start start 2023-10-26T06:39:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.352+0000 7fd73ded4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd738072a20 0x7fd73819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.352+0000 7fd73ded4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738073340 0x7fd7381a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.352+0000 7fd73ded4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd73810d250 0x7fd73819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.352+0000 7fd73ded4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73819e780 con 0x7fd738073340 2023-10-26T06:39:23.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.352+0000 7fd73ded4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73819e8f0 con 0x7fd738072a20 2023-10-26T06:39:23.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.352+0000 7fd73ded4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73819eab0 con 0x7fd73810d250 2023-10-26T06:39:23.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.353+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738073340 0x7fd7381a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:23.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.353+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd738072a20 0x7fd73819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:23.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.353+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd738072a20 0x7fd73819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58034/0 (socket says 172.21.15.89:58034) 2023-10-26T06:39:23.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.353+0000 7fd7377fe700 1 -- 172.21.15.89:0/594479440 learned_addr learned my addr 172.21.15.89:0/594479440 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:23.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.353+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738073340 0x7fd7381a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46156/0 (socket says 172.21.15.89:46156) 2023-10-26T06:39:23.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.353+0000 7fd737fff700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd73810d250 0x7fd73819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:23.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.353+0000 7fd72ffff700 1 -- 172.21.15.89:0/594479440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd73810d250 msgr2=0x7fd73819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.354+0000 7fd72ffff700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd73810d250 0x7fd73819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.354+0000 7fd72ffff700 1 -- 172.21.15.89:0/594479440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd738072a20 msgr2=0x7fd73819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.354+0000 7fd72ffff700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd738072a20 0x7fd73819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.354+0000 7fd72ffff700 1 -- 172.21.15.89:0/594479440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280095f0 con 0x7fd738073340 2023-10-26T06:39:23.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.354+0000 7fd72ffff700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738073340 0x7fd7381a0270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd72000c970 tx=0x7fd72000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:23.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.354+0000 7fd7357fa700 1 -- 172.21.15.89:0/594479440 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720004440 con 0x7fd738073340 2023-10-26T06:39:23.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.354+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73810bcd0 con 0x7fd738073340 2023-10-26T06:39:23.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.355+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73810c220 con 0x7fd738073340 2023-10-26T06:39:23.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.356+0000 7fd7357fa700 1 -- 172.21.15.89:0/594479440 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd720003b40 con 0x7fd738073340 2023-10-26T06:39:23.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.356+0000 7fd7357fa700 1 -- 172.21.15.89:0/594479440 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720007ba0 con 0x7fd738073340 2023-10-26T06:39:23.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.358+0000 7fd7357fa700 1 -- 172.21.15.89:0/594479440 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fd720007d00 con 0x7fd738073340 2023-10-26T06:39:23.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.358+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7180052f0 con 0x7fd738073340 2023-10-26T06:39:23.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.360+0000 7fd7357fa700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7240a14c0 0x7fd7240a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:23.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.360+0000 7fd7357fa700 1 -- 172.21.15.89:0/594479440 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd7200cb120 con 0x7fd738073340 2023-10-26T06:39:23.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.361+0000 7fd7377fe700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7240a14c0 0x7fd7240a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:23.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.362+0000 7fd7377fe700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7240a14c0 0x7fd7240a3980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd728006010 tx=0x7fd72800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:23.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.364+0000 7fd7357fa700 1 -- 172.21.15.89:0/594479440 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd720096e40 con 0x7fd738073340 2023-10-26T06:39:23.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.606+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7fd718005b20 con 0x7fd738073340 2023-10-26T06:39:23.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.611+0000 7fd7357fa700 1 -- 172.21.15.89:0/594479440 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v23)=0 v23) v1 ==== 106+0+0 (secure 0 0 0) 0x7fd720014020 con 0x7fd738073340 2023-10-26T06:39:23.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.614+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7240a14c0 msgr2=0x7fd7240a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.614+0000 7fd73ded4700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7240a14c0 0x7fd7240a3980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd728006010 tx=0x7fd72800b540 comp rx=0 tx=0).stop 2023-10-26T06:39:23.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.614+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738073340 msgr2=0x7fd7381a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:23.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.614+0000 7fd73ded4700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738073340 0x7fd7381a0270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd72000c970 tx=0x7fd72000cc80 comp rx=0 tx=0).stop 2023-10-26T06:39:23.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 shutdown_connections 2023-10-26T06:39:23.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7240a14c0 0x7fd7240a3980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd73810d250 0x7fd73819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd738072a20 0x7fd73819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 --2- 172.21.15.89:0/594479440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738073340 0x7fd7381a0270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:23.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 >> 172.21.15.89:0/594479440 conn(0x7fd7380779e0 msgr2=0x7fd738102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:23.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 shutdown_connections 2023-10-26T06:39:23.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.615+0000 7fd73ded4700 1 -- 172.21.15.89:0/594479440 wait complete. 2023-10-26T06:39:24.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 -- 172.21.15.89:0/2008613431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 msgr2=0x7ff5e410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 --2- 172.21.15.89:0/2008613431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e410f640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b0a0 tx=0x7ff5dc00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 -- 172.21.15.89:0/2008613431 shutdown_connections 2023-10-26T06:39:24.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 --2- 172.21.15.89:0/2008613431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e410f640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 --2- 172.21.15.89:0/2008613431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 0x7ff5e4072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 --2- 172.21.15.89:0/2008613431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4073340 0x7ff5e410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 -- 172.21.15.89:0/2008613431 >> 172.21.15.89:0/2008613431 conn(0x7ff5e40779e0 msgr2=0x7ff5e4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 -- 172.21.15.89:0/2008613431 shutdown_connections 2023-10-26T06:39:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.191+0000 7ff5e985c700 1 -- 172.21.15.89:0/2008613431 wait complete. 2023-10-26T06:39:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 Processor -- start 2023-10-26T06:39:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 -- start start 2023-10-26T06:39:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 0x7ff5e4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4073340 0x7ff5e4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419def0 con 0x7ff5e410d250 2023-10-26T06:39:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419e060 con 0x7ff5e4072a20 2023-10-26T06:39:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.192+0000 7ff5e985c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419e220 con 0x7ff5e4073340 2023-10-26T06:39:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 0x7ff5e4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 0x7ff5e4199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58056/0 (socket says 172.21.15.89:58056) 2023-10-26T06:39:24.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46174/0 (socket says 172.21.15.89:46174) 2023-10-26T06:39:24.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 -- 172.21.15.89:0/777273792 learned_addr learned my addr 172.21.15.89:0/777273792 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:24.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e27fc700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4073340 0x7ff5e4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 -- 172.21.15.89:0/777273792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4073340 msgr2=0x7ff5e4199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4073340 0x7ff5e4199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 -- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 msgr2=0x7ff5e419d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e419d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 -- 172.21.15.89:0/777273792 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc009d00 con 0x7ff5e4072a20 2023-10-26T06:39:24.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e37fe700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:24.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.193+0000 7ff5e2ffd700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 0x7ff5e4199040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e9a0 tx=0x7ff5d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:24.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.194+0000 7ff5e885a700 1 -- 172.21.15.89:0/777273792 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4009790 con 0x7ff5e4072a20 2023-10-26T06:39:24.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.194+0000 7ff5e885a700 1 -- 172.21.15.89:0/777273792 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d4010430 con 0x7ff5e4072a20 2023-10-26T06:39:24.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.194+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e419e500 con 0x7ff5e4072a20 2023-10-26T06:39:24.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.194+0000 7ff5e885a700 1 -- 172.21.15.89:0/777273792 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4018680 con 0x7ff5e4072a20 2023-10-26T06:39:24.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.194+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e41a5d60 con 0x7ff5e4072a20 2023-10-26T06:39:24.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.195+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e4108460 con 0x7ff5e4072a20 2023-10-26T06:39:24.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.197+0000 7ff5e885a700 1 -- 172.21.15.89:0/777273792 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7ff5d40105a0 con 0x7ff5e4072a20 2023-10-26T06:39:24.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.198+0000 7ff5e885a700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff5cc0a14c0 0x7ff5cc0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.198+0000 7ff5e885a700 1 -- 172.21.15.89:0/777273792 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff5d4014070 con 0x7ff5e4072a20 2023-10-26T06:39:24.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.198+0000 7ff5e27fc700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff5cc0a14c0 0x7ff5cc0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.199+0000 7ff5e27fc700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff5cc0a14c0 0x7ff5cc0a3980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5e419a550 tx=0x7ff5d8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:24.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.203+0000 7ff5e885a700 1 -- 172.21.15.89:0/777273792 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff5d40964d0 con 0x7ff5e4072a20 2023-10-26T06:39:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 -- 172.21.15.89:0/2245475446 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 msgr2=0x7f400810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 --2- 172.21.15.89:0/2245475446 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 0x7f400810d650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f400000b0a0 tx=0x7f400000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 -- 172.21.15.89:0/2245475446 shutdown_connections 2023-10-26T06:39:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 --2- 172.21.15.89:0/2245475446 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 0x7f400810d650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 --2- 172.21.15.89:0/2245475446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40081057c0 0x7f4008105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 --2- 172.21.15.89:0/2245475446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40081008a0 0x7f4008100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 -- 172.21.15.89:0/2245475446 >> 172.21.15.89:0/2245475446 conn(0x7f40080779e0 msgr2=0x7f4008077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 -- 172.21.15.89:0/2245475446 shutdown_connections 2023-10-26T06:39:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.884+0000 7f400ec33700 1 -- 172.21.15.89:0/2245475446 wait complete. 2023-10-26T06:39:24.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.885+0000 7f400ec33700 1 Processor -- start 2023-10-26T06:39:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.885+0000 7f400ec33700 1 -- start start 2023-10-26T06:39:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400ec33700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40081008a0 0x7f4008198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400ec33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 0x7f4008199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400ec33700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40081057c0 0x7f400819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400ec33700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819de10 con 0x7f4008101260 2023-10-26T06:39:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400ec33700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819df80 con 0x7f40081057c0 2023-10-26T06:39:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400ec33700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819e140 con 0x7f40081008a0 2023-10-26T06:39:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400c9cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40081008a0 0x7f4008198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f4007fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 0x7f4008199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400c9cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40081008a0 0x7f4008198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41138/0 (socket says 172.21.15.89:41138) 2023-10-26T06:39:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400c9cf700 1 -- 172.21.15.89:0/2819448270 learned_addr learned my addr 172.21.15.89:0/2819448270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f400d1d0700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40081057c0 0x7f400819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f4007fff700 1 -- 172.21.15.89:0/2819448270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40081008a0 msgr2=0x7f4008198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f4007fff700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40081008a0 0x7f4008198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f4007fff700 1 -- 172.21.15.89:0/2819448270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40081057c0 msgr2=0x7f400819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f4007fff700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40081057c0 0x7f400819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.886+0000 7f4007fff700 1 -- 172.21.15.89:0/2819448270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000009d00 con 0x7f4008101260 2023-10-26T06:39:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.887+0000 7f4007fff700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 0x7f4008199450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e9f0 tx=0x7f3ff400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.887+0000 7f4005ffb700 1 -- 172.21.15.89:0/2819448270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400c9c0 con 0x7f4008101260 2023-10-26T06:39:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.887+0000 7f4005ffb700 1 -- 172.21.15.89:0/2819448270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff4010430 con 0x7f4008101260 2023-10-26T06:39:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.887+0000 7f4005ffb700 1 -- 172.21.15.89:0/2819448270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4018680 con 0x7f4008101260 2023-10-26T06:39:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.887+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400819e420 con 0x7f4008101260 2023-10-26T06:39:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.887+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40081a5c80 con 0x7f4008101260 2023-10-26T06:39:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.888+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008101f30 con 0x7f4008101260 2023-10-26T06:39:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.890+0000 7f4005ffb700 1 -- 172.21.15.89:0/2819448270 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f3ff4010860 con 0x7f4008101260 2023-10-26T06:39:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.891+0000 7f4005ffb700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3ff80a14c0 0x7f3ff80a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.891+0000 7f4005ffb700 1 -- 172.21.15.89:0/2819448270 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3ff4014070 con 0x7f4008101260 2023-10-26T06:39:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.891+0000 7f400c9cf700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3ff80a14c0 0x7f3ff80a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.892+0000 7f400c9cf700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3ff80a14c0 0x7f3ff80a3980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005950 tx=0x7f3ffc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:22.895+0000 7f4005ffb700 1 -- 172.21.15.89:0/2819448270 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3ff4052100 con 0x7f4008101260 2023-10-26T06:39:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.120+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f4008110780 con 0x7f4008101260 2023-10-26T06:39:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.126+0000 7f4005ffb700 1 -- 172.21.15.89:0/2819448270 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v21)=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3ff4096420 con 0x7f4008101260 2023-10-26T06:39:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3ff80a14c0 msgr2=0x7f3ff80a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3ff80a14c0 0x7f3ff80a3980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005950 tx=0x7f3ffc009260 comp rx=0 tx=0).stop 2023-10-26T06:39:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 msgr2=0x7f4008199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 0x7f4008199450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e9f0 tx=0x7f3ff400ed00 comp rx=0 tx=0).stop 2023-10-26T06:39:24.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 shutdown_connections 2023-10-26T06:39:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3ff80a14c0 0x7f3ff80a3980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40081057c0 0x7f400819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40081008a0 0x7f4008198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 --2- 172.21.15.89:0/2819448270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4008101260 0x7f4008199450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 >> 172.21.15.89:0/2819448270 conn(0x7f40080779e0 msgr2=0x7f400810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 shutdown_connections 2023-10-26T06:39:24.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:23.129+0000 7f400ec33700 1 -- 172.21.15.89:0/2819448270 wait complete. 2023-10-26T06:39:24.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:24 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2819448270' entity='client.admin' 2023-10-26T06:39:24.402 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:24 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:24.402 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:24 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/594479440' entity='client.admin' 2023-10-26T06:39:24.402 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:24 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2819448270' entity='client.admin' 2023-10-26T06:39:24.402 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:24 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:24.402 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:24 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/594479440' entity='client.admin' 2023-10-26T06:39:24.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:24 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2819448270' entity='client.admin' 2023-10-26T06:39:24.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:24 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:24.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:24 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/594479440' entity='client.admin' 2023-10-26T06:39:24.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.489+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7ff5e40740b0 con 0x7ff5e4072a20 2023-10-26T06:39:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.495+0000 7ff5e885a700 1 -- 172.21.15.89:0/777273792 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v24)=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff5d4099af0 con 0x7ff5e4072a20 2023-10-26T06:39:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff5cc0a14c0 msgr2=0x7ff5cc0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff5cc0a14c0 0x7ff5cc0a3980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5e419a550 tx=0x7ff5d8009260 comp rx=0 tx=0).stop 2023-10-26T06:39:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 msgr2=0x7ff5e4199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 0x7ff5e4199040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e9a0 tx=0x7ff5d400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 shutdown_connections 2023-10-26T06:39:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff5cc0a14c0 0x7ff5cc0a3980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5e410d250 0x7ff5e419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5e4072a20 0x7ff5e4199040 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 --2- 172.21.15.89:0/777273792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5e4073340 0x7ff5e4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.497+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 >> 172.21.15.89:0/777273792 conn(0x7ff5e40779e0 msgr2=0x7ff5e40fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.498+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 shutdown_connections 2023-10-26T06:39:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.498+0000 7ff5e985c700 1 -- 172.21.15.89:0/777273792 wait complete. 2023-10-26T06:39:24.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.280+0000 7f1df93de700 1 -- 172.21.15.89:0/2048153913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068190 msgr2=0x7f1df4068570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.280+0000 7f1df93de700 1 --2- 172.21.15.89:0/2048153913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068190 0x7f1df4068570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009990 tx=0x7f1de4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.280+0000 7f1df93de700 1 -- 172.21.15.89:0/2048153913 shutdown_connections 2023-10-26T06:39:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.280+0000 7f1df93de700 1 --2- 172.21.15.89:0/2048153913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1df410d300 0x7f1df410f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.280+0000 7f1df93de700 1 --2- 172.21.15.89:0/2048153913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068190 0x7f1df4068570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.280+0000 7f1df93de700 1 --2- 172.21.15.89:0/2048153913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1df4068ab0 0x7f1df410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.280+0000 7f1df93de700 1 -- 172.21.15.89:0/2048153913 >> 172.21.15.89:0/2048153913 conn(0x7f1df4075790 msgr2=0x7f1df4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.281+0000 7f1df93de700 1 -- 172.21.15.89:0/2048153913 shutdown_connections 2023-10-26T06:39:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.281+0000 7f1df93de700 1 -- 172.21.15.89:0/2048153913 wait complete. 2023-10-26T06:39:24.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.282+0000 7f1df93de700 1 Processor -- start 2023-10-26T06:39:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.282+0000 7f1df93de700 1 -- start start 2023-10-26T06:39:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df93de700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1df4068190 0x7f1df4071e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df93de700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068ab0 0x7f1df40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df93de700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1df410d300 0x7f1df406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df93de700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df406b7c0 con 0x7f1df410d300 2023-10-26T06:39:24.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df93de700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df406b930 con 0x7f1df4068190 2023-10-26T06:39:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df93de700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df406baf0 con 0x7f1df4068ab0 2023-10-26T06:39:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1df4068190 0x7f1df4071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df27fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068ab0 0x7f1df40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1df4068190 0x7f1df4071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58068/0 (socket says 172.21.15.89:58068) 2023-10-26T06:39:24.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df2ffd700 1 -- 172.21.15.89:0/1484327332 learned_addr learned my addr 172.21.15.89:0/1484327332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df27fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068ab0 0x7f1df40723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41190/0 (socket says 172.21.15.89:41190) 2023-10-26T06:39:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.283+0000 7f1df37fe700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1df410d300 0x7f1df406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df27fc700 1 -- 172.21.15.89:0/1484327332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1df4068190 msgr2=0x7f1df4071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df27fc700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1df4068190 0x7f1df4071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df27fc700 1 -- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1df410d300 msgr2=0x7f1df406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df27fc700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1df410d300 0x7f1df406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df27fc700 1 -- 172.21.15.89:0/1484327332 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de40095f0 con 0x7f1df4068ab0 2023-10-26T06:39:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df27fc700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068ab0 0x7f1df40723a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1de800e9f0 tx=0x7f1de800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1ddbfff700 1 -- 172.21.15.89:0/1484327332 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800eee0 con 0x7f1df4068ab0 2023-10-26T06:39:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1ddbfff700 1 -- 172.21.15.89:0/1484327332 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1de800cbd0 con 0x7f1df4068ab0 2023-10-26T06:39:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df406f9c0 con 0x7f1df4068ab0 2023-10-26T06:39:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1ddbfff700 1 -- 172.21.15.89:0/1484327332 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8018680 con 0x7f1df4068ab0 2023-10-26T06:39:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.284+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df406fec0 con 0x7f1df4068ab0 2023-10-26T06:39:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.286+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df406bd70 con 0x7f1df4068ab0 2023-10-26T06:39:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.287+0000 7f1ddbfff700 1 -- 172.21.15.89:0/1484327332 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f1de800cd40 con 0x7f1df4068ab0 2023-10-26T06:39:24.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.288+0000 7f1ddbfff700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ddc0a1440 0x7f1ddc0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.289+0000 7f1ddbfff700 1 -- 172.21.15.89:0/1484327332 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1de8014070 con 0x7f1df4068ab0 2023-10-26T06:39:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.289+0000 7f1df2ffd700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ddc0a1440 0x7f1ddc0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.289+0000 7f1df2ffd700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ddc0a1440 0x7f1ddc0a3900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1de4006010 tx=0x7f1de400b280 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.293+0000 7f1ddbfff700 1 -- 172.21.15.89:0/1484327332 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1df406bd70 con 0x7f1df4068ab0 2023-10-26T06:39:24.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.544+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f1df406bd70 con 0x7f1df4068ab0 2023-10-26T06:39:24.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.554+0000 7f1ddbfff700 1 -- 172.21.15.89:0/1484327332 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v25)=0 v25) v1 ==== 116+0+0 (secure 0 0 0) 0x7f1df406bd70 con 0x7f1df4068ab0 2023-10-26T06:39:24.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ddc0a1440 msgr2=0x7f1ddc0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ddc0a1440 0x7f1ddc0a3900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1de4006010 tx=0x7f1de400b280 comp rx=0 tx=0).stop 2023-10-26T06:39:24.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068ab0 msgr2=0x7f1df40723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:24.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068ab0 0x7f1df40723a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1de800e9f0 tx=0x7f1de800ed00 comp rx=0 tx=0).stop 2023-10-26T06:39:24.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 shutdown_connections 2023-10-26T06:39:24.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ddc0a1440 0x7f1ddc0a3900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1df410d300 0x7f1df406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1df4068190 0x7f1df4071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 --2- 172.21.15.89:0/1484327332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1df4068ab0 0x7f1df40723a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:24.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.556+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 >> 172.21.15.89:0/1484327332 conn(0x7f1df4075790 msgr2=0x7f1df410ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:24.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.557+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 shutdown_connections 2023-10-26T06:39:24.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:24.557+0000 7f1df93de700 1 -- 172.21.15.89:0/1484327332 wait complete. 2023-10-26T06:39:25.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 -- 172.21.15.89:0/2031465894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 msgr2=0x7fc5ec105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:25.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 --2- 172.21.15.89:0/2031465894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec105ba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009990 tx=0x7fc5dc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 -- 172.21.15.89:0/2031465894 shutdown_connections 2023-10-26T06:39:25.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 --2- 172.21.15.89:0/2031465894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec105ba0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 --2- 172.21.15.89:0/2031465894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ec101260 0x7fc5ec10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 --2- 172.21.15.89:0/2031465894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5ec1008a0 0x7fc5ec100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 -- 172.21.15.89:0/2031465894 >> 172.21.15.89:0/2031465894 conn(0x7fc5ec0779e0 msgr2=0x7fc5ec077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 -- 172.21.15.89:0/2031465894 shutdown_connections 2023-10-26T06:39:25.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.230+0000 7fc5f0904700 1 -- 172.21.15.89:0/2031465894 wait complete. 2023-10-26T06:39:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.231+0000 7fc5f0904700 1 Processor -- start 2023-10-26T06:39:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.231+0000 7fc5f0904700 1 -- start start 2023-10-26T06:39:25.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5f0904700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ec1008a0 0x7fc5ec198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5f0904700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5ec101260 0x7fc5ec1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5f0904700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5f0904700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec19dd60 con 0x7fc5ec1057c0 2023-10-26T06:39:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5f0904700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec19ded0 con 0x7fc5ec1008a0 2023-10-26T06:39:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5f0904700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec19e090 con 0x7fc5ec101260 2023-10-26T06:39:25.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5ead9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5e9d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5ec101260 0x7fc5ec1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5ea59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ec1008a0 0x7fc5ec198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5ead9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec19d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46216/0 (socket says 172.21.15.89:46216) 2023-10-26T06:39:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5ead9d700 1 -- 172.21.15.89:0/1831547098 learned_addr learned my addr 172.21.15.89:0/1831547098 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:25.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5ea59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ec1008a0 0x7fc5ec198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58080/0 (socket says 172.21.15.89:58080) 2023-10-26T06:39:25.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.232+0000 7fc5e9d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5ec101260 0x7fc5ec1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41200/0 (socket says 172.21.15.89:41200) 2023-10-26T06:39:25.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5ead9d700 1 -- 172.21.15.89:0/1831547098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5ec101260 msgr2=0x7fc5ec1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:25.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5ead9d700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5ec101260 0x7fc5ec1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5ead9d700 1 -- 172.21.15.89:0/1831547098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ec1008a0 msgr2=0x7fc5ec198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5ead9d700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ec1008a0 0x7fc5ec198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5ead9d700 1 -- 172.21.15.89:0/1831547098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc0095f0 con 0x7fc5ec1057c0 2023-10-26T06:39:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5ead9d700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec19d750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400efc0 tx=0x7fc5e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5db7fe700 1 -- 172.21.15.89:0/1831547098 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4018070 con 0x7fc5ec1057c0 2023-10-26T06:39:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5db7fe700 1 -- 172.21.15.89:0/1831547098 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e400f040 con 0x7fc5ec1057c0 2023-10-26T06:39:25.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec19e310 con 0x7fc5ec1057c0 2023-10-26T06:39:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5db7fe700 1 -- 172.21.15.89:0/1831547098 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e400cc10 con 0x7fc5ec1057c0 2023-10-26T06:39:25.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.233+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec1a5bd0 con 0x7fc5ec1057c0 2023-10-26T06:39:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.234+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec101f30 con 0x7fc5ec1057c0 2023-10-26T06:39:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.236+0000 7fc5db7fe700 1 -- 172.21.15.89:0/1831547098 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fc5e4003a00 con 0x7fc5ec1057c0 2023-10-26T06:39:25.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.237+0000 7fc5db7fe700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc5d40a14c0 0x7fc5d40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.237+0000 7fc5db7fe700 1 -- 172.21.15.89:0/1831547098 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc5e40caa70 con 0x7fc5ec1057c0 2023-10-26T06:39:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.238+0000 7fc5ea59c700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc5d40a14c0 0x7fc5d40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.238+0000 7fc5ea59c700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc5d40a14c0 0x7fc5d40a3980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b2e0 tx=0x7fc5dc005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:25.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.242+0000 7fc5db7fe700 1 -- 172.21.15.89:0/1831547098 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc5e4095cc0 con 0x7fc5ec1057c0 2023-10-26T06:39:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:25 smithi192 ceph-mon[131661]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:25 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:25 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.456+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7fc5ec04e4c0 con 0x7fc5ec1057c0 2023-10-26T06:39:25.462 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:25 smithi089 ceph-mon[95031]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:25.463 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:25 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:25.463 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:25 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.461+0000 7fc5db7fe700 1 -- 172.21.15.89:0/1831547098 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v26)=0 v26) v1 ==== 117+0+0 (secure 0 0 0) 0x7fc5e40992e0 con 0x7fc5ec1057c0 2023-10-26T06:39:25.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc5d40a14c0 msgr2=0x7fc5d40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc5d40a14c0 0x7fc5d40a3980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b2e0 tx=0x7fc5dc005fd0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 msgr2=0x7fc5ec19d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec19d750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400efc0 tx=0x7fc5e400c3a0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 shutdown_connections 2023-10-26T06:39:25.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc5d40a14c0 0x7fc5d40a3980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec1057c0 0x7fc5ec19d750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ec1008a0 0x7fc5ec198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 --2- 172.21.15.89:0/1831547098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5ec101260 0x7fc5ec1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 >> 172.21.15.89:0/1831547098 conn(0x7fc5ec0779e0 msgr2=0x7fc5ec10ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 shutdown_connections 2023-10-26T06:39:25.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:25.464+0000 7fc5f0904700 1 -- 172.21.15.89:0/1831547098 wait complete. 2023-10-26T06:39:25.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:25 smithi135 ceph-mon[131403]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:25.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:25 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:25.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:25 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.385+0000 7f41ba453700 1 -- 172.21.15.89:0/3017549841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 msgr2=0x7f41b410d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:26.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.385+0000 7f41ba453700 1 --2- 172.21.15.89:0/3017549841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 0x7f41b410d650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00b0a0 tx=0x7f41ac00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.386+0000 7f41ba453700 1 -- 172.21.15.89:0/3017549841 shutdown_connections 2023-10-26T06:39:26.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.386+0000 7f41ba453700 1 --2- 172.21.15.89:0/3017549841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 0x7f41b410d650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.386+0000 7f41ba453700 1 --2- 172.21.15.89:0/3017549841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41b41057c0 0x7f41b4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.386+0000 7f41ba453700 1 --2- 172.21.15.89:0/3017549841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41b41008a0 0x7f41b4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.386+0000 7f41ba453700 1 -- 172.21.15.89:0/3017549841 >> 172.21.15.89:0/3017549841 conn(0x7f41b40779e0 msgr2=0x7f41b4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:26.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.386+0000 7f41ba453700 1 -- 172.21.15.89:0/3017549841 shutdown_connections 2023-10-26T06:39:26.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.386+0000 7f41ba453700 1 -- 172.21.15.89:0/3017549841 wait complete. 2023-10-26T06:39:26.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.387+0000 7f41ba453700 1 Processor -- start 2023-10-26T06:39:26.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.387+0000 7f41ba453700 1 -- start start 2023-10-26T06:39:26.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.388+0000 7f41ba453700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41b41008a0 0x7f41b4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:26.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.388+0000 7f41ba453700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 0x7f41b4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:26.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.388+0000 7f41ba453700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41b41057c0 0x7f41b4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:26.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.388+0000 7f41ba453700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41a5050 con 0x7f41b4101260 2023-10-26T06:39:26.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.388+0000 7f41ba453700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41a51c0 con 0x7f41b41008a0 2023-10-26T06:39:26.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.388+0000 7f41ba453700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41a5380 con 0x7f41b41057c0 2023-10-26T06:39:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41b41008a0 0x7f41b4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b89f0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41b41057c0 0x7f41b4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:26.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b89f0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41b41057c0 0x7f41b4073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41230/0 (socket says 172.21.15.89:41230) 2023-10-26T06:39:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b89f0700 1 -- 172.21.15.89:0/3348225578 learned_addr learned my addr 172.21.15.89:0/3348225578 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b37fe700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 0x7f41b4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b3fff700 1 -- 172.21.15.89:0/3348225578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41b41057c0 msgr2=0x7f41b4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b3fff700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41b41057c0 0x7f41b4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b3fff700 1 -- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 msgr2=0x7f41b4073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:26.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b3fff700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 0x7f41b4073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.389+0000 7f41b3fff700 1 -- 172.21.15.89:0/3348225578 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac009d00 con 0x7f41b41008a0 2023-10-26T06:39:26.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.390+0000 7f41b3fff700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41b41008a0 0x7f41b4072af0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9c0 tx=0x7f41a400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:26.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.390+0000 7f41b17fa700 1 -- 172.21.15.89:0/3348225578 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a400cbd0 con 0x7f41b41008a0 2023-10-26T06:39:26.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.390+0000 7f41b17fa700 1 -- 172.21.15.89:0/3348225578 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41a400eeb0 con 0x7f41b41008a0 2023-10-26T06:39:26.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.390+0000 7f41b17fa700 1 -- 172.21.15.89:0/3348225578 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4003d10 con 0x7f41b41008a0 2023-10-26T06:39:26.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.391+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b41a5660 con 0x7f41b41008a0 2023-10-26T06:39:26.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.391+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b41a5ae0 con 0x7f41b41008a0 2023-10-26T06:39:26.395 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:26.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.392+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b4101f30 con 0x7f41b41008a0 2023-10-26T06:39:26.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.395+0000 7f41b17fa700 1 -- 172.21.15.89:0/3348225578 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f41a400cd30 con 0x7f41b41008a0 2023-10-26T06:39:26.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.397+0000 7f41b17fa700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f419c0a14c0 0x7f419c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:26.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.397+0000 7f41b17fa700 1 -- 172.21.15.89:0/3348225578 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f41a4014070 con 0x7f41b41008a0 2023-10-26T06:39:26.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.397+0000 7f41b37fe700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f419c0a14c0 0x7f419c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:26.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.398+0000 7f41b37fe700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f419c0a14c0 0x7f419c0a3980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41b4074220 tx=0x7f41a8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:26.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.401+0000 7f41b17fa700 1 -- 172.21.15.89:0/3348225578 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f41a40965b0 con 0x7f41b41008a0 2023-10-26T06:39:26.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:26 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1831547098' entity='client.admin' 2023-10-26T06:39:26.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:26 smithi089 ceph-mon[95031]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:26.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.650+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f41b4110780 con 0x7f41b41008a0 2023-10-26T06:39:26.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.655+0000 7f41b17fa700 1 -- 172.21.15.89:0/3348225578 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v27)=0 v27) v1 ==== 120+0+0 (secure 0 0 0) 0x7f41a4099bd0 con 0x7f41b41008a0 2023-10-26T06:39:26.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f419c0a14c0 msgr2=0x7f419c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:26.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f419c0a14c0 0x7f419c0a3980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41b4074220 tx=0x7f41a8009260 comp rx=0 tx=0).stop 2023-10-26T06:39:26.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41b41008a0 msgr2=0x7f41b4072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:26.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41b41008a0 0x7f41b4072af0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9c0 tx=0x7f41a400ecd0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 shutdown_connections 2023-10-26T06:39:26.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f419c0a14c0 0x7f419c0a3980 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41b41057c0 0x7f41b4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41b41008a0 0x7f41b4072af0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 --2- 172.21.15.89:0/3348225578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41b4101260 0x7f41b4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:26.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 >> 172.21.15.89:0/3348225578 conn(0x7f41b40779e0 msgr2=0x7f41b40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:26.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 shutdown_connections 2023-10-26T06:39:26.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.658+0000 7f41ba453700 1 -- 172.21.15.89:0/3348225578 wait complete. 2023-10-26T06:39:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:26.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:26 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1831547098' entity='client.admin' 2023-10-26T06:39:26.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:26 smithi135 ceph-mon[131403]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:26.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:26 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1831547098' entity='client.admin' 2023-10-26T06:39:26.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:26 smithi192 ceph-mon[131661]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:27.029 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi089') 2023-10-26T06:39:27.621 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:27.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.778+0000 7ff7d0dca700 1 -- 172.21.15.89:0/731499834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc074770 msgr2=0x7ff7cc072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.778+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/731499834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc074770 0x7ff7cc072a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009990 tx=0x7ff7bc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.778+0000 7ff7d0dca700 1 -- 172.21.15.89:0/731499834 shutdown_connections 2023-10-26T06:39:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.778+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/731499834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc074770 0x7ff7cc072a20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.778+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/731499834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc111570 0x7ff7cc1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.778+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/731499834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7cc072f60 0x7ff7cc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.778+0000 7ff7d0dca700 1 -- 172.21.15.89:0/731499834 >> 172.21.15.89:0/731499834 conn(0x7ff7cc0779e0 msgr2=0x7ff7cc077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.779+0000 7ff7d0dca700 1 -- 172.21.15.89:0/731499834 shutdown_connections 2023-10-26T06:39:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.779+0000 7ff7d0dca700 1 -- 172.21.15.89:0/731499834 wait complete. 2023-10-26T06:39:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.780+0000 7ff7d0dca700 1 Processor -- start 2023-10-26T06:39:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.780+0000 7ff7d0dca700 1 -- start start 2023-10-26T06:39:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7d0dca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc072f60 0x7ff7cc19d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7d0dca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7cc074770 0x7ff7cc19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7ca59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc072f60 0x7ff7cc19d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7ca59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc072f60 0x7ff7cc19d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46252/0 (socket says 172.21.15.89:46252) 2023-10-26T06:39:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7d0dca700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc111570 0x7ff7cc1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7d0dca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc1a1fc0 con 0x7ff7cc072f60 2023-10-26T06:39:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7d0dca700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc1a2130 con 0x7ff7cc074770 2023-10-26T06:39:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7d0dca700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc1a22f0 con 0x7ff7cc111570 2023-10-26T06:39:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7ca59c700 1 -- 172.21.15.89:0/4245911714 learned_addr learned my addr 172.21.15.89:0/4245911714 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:27.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7c9d9b700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7cc074770 0x7ff7cc19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.781+0000 7ff7ca59c700 1 -- 172.21.15.89:0/4245911714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc111570 msgr2=0x7ff7cc1a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7cad9d700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc111570 0x7ff7cc1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7ca59c700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc111570 0x7ff7cc1a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7ca59c700 1 -- 172.21.15.89:0/4245911714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7cc074770 msgr2=0x7ff7cc19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:27.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7ca59c700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7cc074770 0x7ff7cc19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7ca59c700 1 -- 172.21.15.89:0/4245911714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0095f0 con 0x7ff7cc072f60 2023-10-26T06:39:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7c9d9b700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7cc074770 0x7ff7cc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:39:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7cad9d700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc111570 0x7ff7cc1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:39:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7ca59c700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc072f60 0x7ff7cc19d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00b2e0 tx=0x7ff7bc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:27.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7bb7fe700 1 -- 172.21.15.89:0/4245911714 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc01d070 con 0x7ff7cc072f60 2023-10-26T06:39:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7bb7fe700 1 -- 172.21.15.89:0/4245911714 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7bc0218d0 con 0x7ff7cc072f60 2023-10-26T06:39:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc1a2570 con 0x7ff7cc072f60 2023-10-26T06:39:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.782+0000 7ff7bb7fe700 1 -- 172.21.15.89:0/4245911714 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc017650 con 0x7ff7cc072f60 2023-10-26T06:39:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.783+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc1a9e30 con 0x7ff7cc072f60 2023-10-26T06:39:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.784+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc0740b0 con 0x7ff7cc072f60 2023-10-26T06:39:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.786+0000 7ff7bb7fe700 1 -- 172.21.15.89:0/4245911714 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7ff7bc021a40 con 0x7ff7cc072f60 2023-10-26T06:39:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.787+0000 7ff7bb7fe700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7b40a14c0 0x7ff7b40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.787+0000 7ff7bb7fe700 1 -- 172.21.15.89:0/4245911714 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff7bc0cc580 con 0x7ff7cc072f60 2023-10-26T06:39:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.787+0000 7ff7c9d9b700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7b40a14c0 0x7ff7b40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.788+0000 7ff7c9d9b700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7b40a14c0 0x7ff7b40a3980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0005fd0 tx=0x7ff7c0005f60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:26.791+0000 7ff7bb7fe700 1 -- 172.21.15.89:0/4245911714 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff7bc097870 con 0x7ff7cc072f60 2023-10-26T06:39:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.009+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7ff7cc04e4c0 con 0x7ff7cc072f60 2023-10-26T06:39:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.013+0000 7ff7bb7fe700 1 -- 172.21.15.89:0/4245911714 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff7bc09ae90 con 0x7ff7cc072f60 2023-10-26T06:39:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7b40a14c0 msgr2=0x7ff7b40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7b40a14c0 0x7ff7b40a3980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0005fd0 tx=0x7ff7c0005f60 comp rx=0 tx=0).stop 2023-10-26T06:39:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc072f60 msgr2=0x7ff7cc19d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc072f60 0x7ff7cc19d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00b2e0 tx=0x7ff7bc00f740 comp rx=0 tx=0).stop 2023-10-26T06:39:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 shutdown_connections 2023-10-26T06:39:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7b40a14c0 0x7ff7b40a3980 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7cc072f60 0x7ff7cc19d0c0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00b2e0 tx=0x7ff7bc00f740 comp rx=0 tx=0).stop 2023-10-26T06:39:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc111570 0x7ff7cc1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 --2- 172.21.15.89:0/4245911714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7cc074770 0x7ff7cc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.016+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 >> 172.21.15.89:0/4245911714 conn(0x7ff7cc0779e0 msgr2=0x7ff7cc10ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.017+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 shutdown_connections 2023-10-26T06:39:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:27.017+0000 7ff7d0dca700 1 -- 172.21.15.89:0/4245911714 wait complete. 2023-10-26T06:39:27.885 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi089') 2023-10-26T06:39:27.886 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi089') 2023-10-26T06:39:27.886 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi089') 2023-10-26T06:39:27.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:27 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:27.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:27 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4245911714' entity='client.admin' 2023-10-26T06:39:28.019 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:27 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:28.019 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:27 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4245911714' entity='client.admin' 2023-10-26T06:39:28.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:27 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:28.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:27 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4245911714' entity='client.admin' 2023-10-26T06:39:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: Reconfiguring mon.c (monmap changed)... 2023-10-26T06:39:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:39:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:39:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:29.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:29 smithi192 ceph-mon[131661]: Reconfiguring daemon mon.c on smithi192 2023-10-26T06:39:29.408 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: Reconfiguring mon.c (monmap changed)... 2023-10-26T06:39:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:39:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:39:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:29 smithi135 ceph-mon[131403]: Reconfiguring daemon mon.c on smithi192 2023-10-26T06:39:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: Reconfiguring mon.c (monmap changed)... 2023-10-26T06:39:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-26T06:39:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-26T06:39:29.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:29.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:29 smithi089 ceph-mon[95031]: Reconfiguring daemon mon.c on smithi192 2023-10-26T06:39:30.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 -- 172.21.15.89:0/3674047005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 msgr2=0x7fae3c072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:30.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 --2- 172.21.15.89:0/3674047005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 0x7fae3c072e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae38009990 tx=0x7fae38009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 -- 172.21.15.89:0/3674047005 shutdown_connections 2023-10-26T06:39:30.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 --2- 172.21.15.89:0/3674047005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 0x7fae3c072e00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 --2- 172.21.15.89:0/3674047005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae3c10d250 0x7fae3c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 --2- 172.21.15.89:0/3674047005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c073340 0x7fae3c10cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 -- 172.21.15.89:0/3674047005 >> 172.21.15.89:0/3674047005 conn(0x7fae3c0779e0 msgr2=0x7fae3c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:30.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 -- 172.21.15.89:0/3674047005 shutdown_connections 2023-10-26T06:39:30.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.050+0000 7fae440d4700 1 -- 172.21.15.89:0/3674047005 wait complete. 2023-10-26T06:39:30.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.051+0000 7fae440d4700 1 Processor -- start 2023-10-26T06:39:30.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.051+0000 7fae440d4700 1 -- start start 2023-10-26T06:39:30.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae440d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 0x7fae3c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:30.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae440d4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c073340 0x7fae3c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:30.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae440d4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae3c10d250 0x7fae3c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:30.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae440d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19ddb0 con 0x7fae3c072a20 2023-10-26T06:39:30.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae440d4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19df20 con 0x7fae3c073340 2023-10-26T06:39:30.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae440d4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19e0e0 con 0x7fae3c10d250 2023-10-26T06:39:30.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae41e70700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 0x7fae3c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:30.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae41e70700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 0x7fae3c198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48846/0 (socket says 172.21.15.89:48846) 2023-10-26T06:39:30.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae41e70700 1 -- 172.21.15.89:0/894789079 learned_addr learned my addr 172.21.15.89:0/894789079 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:30.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae4166f700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c073340 0x7fae3c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:30.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.052+0000 7fae42671700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae3c10d250 0x7fae3c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:30.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae4166f700 1 -- 172.21.15.89:0/894789079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae3c10d250 msgr2=0x7fae3c19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:30.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae4166f700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae3c10d250 0x7fae3c19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae4166f700 1 -- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 msgr2=0x7fae3c198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:30.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae4166f700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 0x7fae3c198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae4166f700 1 -- 172.21.15.89:0/894789079 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae380095f0 con 0x7fae3c073340 2023-10-26T06:39:30.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae4166f700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c073340 0x7fae3c199440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e9f0 tx=0x7fae2c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:30.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae32ffd700 1 -- 172.21.15.89:0/894789079 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00eee0 con 0x7fae3c073340 2023-10-26T06:39:30.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c19e3c0 con 0x7fae3c073340 2023-10-26T06:39:30.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae32ffd700 1 -- 172.21.15.89:0/894789079 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae2c00cbd0 con 0x7fae3c073340 2023-10-26T06:39:30.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.053+0000 7fae32ffd700 1 -- 172.21.15.89:0/894789079 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c018680 con 0x7fae3c073340 2023-10-26T06:39:30.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.054+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c1a5c20 con 0x7fae3c073340 2023-10-26T06:39:30.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.055+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c108460 con 0x7fae3c073340 2023-10-26T06:39:30.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.057+0000 7fae32ffd700 1 -- 172.21.15.89:0/894789079 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fae2c00cd40 con 0x7fae3c073340 2023-10-26T06:39:30.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.059+0000 7fae32ffd700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae280a1440 0x7fae280a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:30.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.059+0000 7fae32ffd700 1 -- 172.21.15.89:0/894789079 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fae2c014070 con 0x7fae3c073340 2023-10-26T06:39:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.059+0000 7fae41e70700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae280a1440 0x7fae280a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:30.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.060+0000 7fae41e70700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae280a1440 0x7fae280a3900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fae38006010 tx=0x7fae3800b280 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:30.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.060+0000 7fae32ffd700 1 -- 172.21.15.89:0/894789079 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae2c0965e0 con 0x7fae3c073340 2023-10-26T06:39:30.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.302+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7fae3c19a1c0 con 0x7fae3c073340 2023-10-26T06:39:30.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.308+0000 7fae32ffd700 1 -- 172.21.15.89:0/894789079 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v29)=0 v29) v1 ==== 116+0+0 (secure 0 0 0) 0x7fae2c099c00 con 0x7fae3c073340 2023-10-26T06:39:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae280a1440 msgr2=0x7fae280a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae280a1440 0x7fae280a3900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fae38006010 tx=0x7fae3800b280 comp rx=0 tx=0).stop 2023-10-26T06:39:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c073340 msgr2=0x7fae3c199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c073340 0x7fae3c199440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e9f0 tx=0x7fae2c00ed00 comp rx=0 tx=0).stop 2023-10-26T06:39:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 shutdown_connections 2023-10-26T06:39:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae280a1440 0x7fae280a3900 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae3c072a20 0x7fae3c198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae3c10d250 0x7fae3c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 --2- 172.21.15.89:0/894789079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c073340 0x7fae3c199440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 >> 172.21.15.89:0/894789079 conn(0x7fae3c0779e0 msgr2=0x7fae3c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 shutdown_connections 2023-10-26T06:39:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:30.311+0000 7fae440d4700 1 -- 172.21.15.89:0/894789079 wait complete. 2023-10-26T06:39:30.688 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.689 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.689 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:30.689 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:30.689 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:30.690 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.690 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:30.690 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:30 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:30.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:30.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:30.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:30.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:30.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:30 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:30.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:30.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:30.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:30 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:31.034 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi089') 2023-10-26T06:39:31.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi089') 2023-10-26T06:39:31.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi089') 2023-10-26T06:39:31.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi089') 2023-10-26T06:39:31.036 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi089') 2023-10-26T06:39:31.036 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi089') 2023-10-26T06:39:31.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 -- 172.21.15.89:0/3597362322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 msgr2=0x7fdf88072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 --2- 172.21.15.89:0/3597362322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88072e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098f0 tx=0x7fdf84009c00 comp rx=0 tx=0).stop 2023-10-26T06:39:31.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 -- 172.21.15.89:0/3597362322 shutdown_connections 2023-10-26T06:39:31.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 --2- 172.21.15.89:0/3597362322 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88072e00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 --2- 172.21.15.89:0/3597362322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf8810d250 0x7fdf8810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 --2- 172.21.15.89:0/3597362322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf88073340 0x7fdf8810cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 -- 172.21.15.89:0/3597362322 >> 172.21.15.89:0/3597362322 conn(0x7fdf880779e0 msgr2=0x7fdf88077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:31.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 -- 172.21.15.89:0/3597362322 shutdown_connections 2023-10-26T06:39:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.319+0000 7fdf90040700 1 -- 172.21.15.89:0/3597362322 wait complete. 2023-10-26T06:39:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.320+0000 7fdf90040700 1 Processor -- start 2023-10-26T06:39:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf90040700 1 -- start start 2023-10-26T06:39:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf90040700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf90040700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf88073340 0x7fdf88199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf90040700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf8810d250 0x7fdf8819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf90040700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8819de20 con 0x7fdf88072a20 2023-10-26T06:39:31.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf90040700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8819df90 con 0x7fdf8810d250 2023-10-26T06:39:31.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf90040700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8819e150 con 0x7fdf88073340 2023-10-26T06:39:31.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf8dddc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf8dddc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48864/0 (socket says 172.21.15.89:48864) 2023-10-26T06:39:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf8dddc700 1 -- 172.21.15.89:0/167172179 learned_addr learned my addr 172.21.15.89:0/167172179 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.321+0000 7fdf8e5dd700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf8810d250 0x7fdf8819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf8d5db700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf88073340 0x7fdf88199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf8dddc700 1 -- 172.21.15.89:0/167172179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf88073340 msgr2=0x7fdf88199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf8dddc700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf88073340 0x7fdf88199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf8dddc700 1 -- 172.21.15.89:0/167172179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf8810d250 msgr2=0x7fdf8819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf8dddc700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf8810d250 0x7fdf8819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf8dddc700 1 -- 172.21.15.89:0/167172179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84009520 con 0x7fdf88072a20 2023-10-26T06:39:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf8dddc700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88198f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf84005950 tx=0x7fdf8400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf7effd700 1 -- 172.21.15.89:0/167172179 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8401c070 con 0x7fdf88072a20 2023-10-26T06:39:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf7effd700 1 -- 172.21.15.89:0/167172179 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8400bb20 con 0x7fdf88072a20 2023-10-26T06:39:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8819e3d0 con 0x7fdf88072a20 2023-10-26T06:39:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf7effd700 1 -- 172.21.15.89:0/167172179 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf840038e0 con 0x7fdf88072a20 2023-10-26T06:39:31.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.322+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf881a5c90 con 0x7fdf88072a20 2023-10-26T06:39:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.324+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf88108460 con 0x7fdf88072a20 2023-10-26T06:39:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.325+0000 7fdf7effd700 1 -- 172.21.15.89:0/167172179 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fdf84020030 con 0x7fdf88072a20 2023-10-26T06:39:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.326+0000 7fdf7effd700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf740a1440 0x7fdf740a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.327+0000 7fdf7effd700 1 -- 172.21.15.89:0/167172179 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fdf840cc030 con 0x7fdf88072a20 2023-10-26T06:39:31.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.327+0000 7fdf8d5db700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf740a1440 0x7fdf740a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.327+0000 7fdf8d5db700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf740a1440 0x7fdf740a3900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf8819a480 tx=0x7fdf78009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:31.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.331+0000 7fdf7effd700 1 -- 172.21.15.89:0/167172179 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdf84097280 con 0x7fdf88072a20 2023-10-26T06:39:31.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.417+0000 7f59b6b40700 1 -- 172.21.15.89:0/471353354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0106680 msgr2=0x7f59b0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.417+0000 7f59b6b40700 1 --2- 172.21.15.89:0/471353354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0106680 0x7f59b0106a60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009990 tx=0x7f59ac009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.417+0000 7f59b6b40700 1 -- 172.21.15.89:0/471353354 shutdown_connections 2023-10-26T06:39:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.417+0000 7f59b6b40700 1 --2- 172.21.15.89:0/471353354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b0100fb0 0x7f59b010d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.417+0000 7f59b6b40700 1 --2- 172.21.15.89:0/471353354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0106680 0x7f59b0106a60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.417+0000 7f59b6b40700 1 --2- 172.21.15.89:0/471353354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b01005f0 0x7f59b0100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.418+0000 7f59b6b40700 1 -- 172.21.15.89:0/471353354 >> 172.21.15.89:0/471353354 conn(0x7f59b00fc120 msgr2=0x7f59b00fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.418+0000 7f59b6b40700 1 -- 172.21.15.89:0/471353354 shutdown_connections 2023-10-26T06:39:31.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.418+0000 7f59b6b40700 1 -- 172.21.15.89:0/471353354 wait complete. 2023-10-26T06:39:31.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.419+0000 7f59b6b40700 1 Processor -- start 2023-10-26T06:39:31.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.419+0000 7f59b6b40700 1 -- start start 2023-10-26T06:39:31.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.419+0000 7f59b6b40700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b01005f0 0x7f59b019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.419+0000 7f59b6b40700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0100fb0 0x7f59b019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b6b40700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b0106680 0x7f59b0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b6b40700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0199110 con 0x7f59b01005f0 2023-10-26T06:39:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b6b40700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0199280 con 0x7f59b0100fb0 2023-10-26T06:39:31.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b6b40700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0199440 con 0x7f59b0106680 2023-10-26T06:39:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b01005f0 0x7f59b019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b01005f0 0x7f59b019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48888/0 (socket says 172.21.15.89:48888) 2023-10-26T06:39:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 -- 172.21.15.89:0/774868653 learned_addr learned my addr 172.21.15.89:0/774868653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b533d700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0100fb0 0x7f59b019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b633f700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b0106680 0x7f59b0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 -- 172.21.15.89:0/774868653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b0106680 msgr2=0x7f59b0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b0106680 0x7f59b0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 -- 172.21.15.89:0/774868653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0100fb0 msgr2=0x7f59b019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0100fb0 0x7f59b019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.420+0000 7f59b5b3e700 1 -- 172.21.15.89:0/774868653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac0095f0 con 0x7f59b01005f0 2023-10-26T06:39:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.421+0000 7f59b5b3e700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b01005f0 0x7f59b019e920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000c00 tx=0x7f59ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.421+0000 7f59a6ffd700 1 -- 172.21.15.89:0/774868653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac01d070 con 0x7f59b01005f0 2023-10-26T06:39:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.421+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b01996c0 con 0x7f59b01005f0 2023-10-26T06:39:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.421+0000 7f59a6ffd700 1 -- 172.21.15.89:0/774868653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59ac005030 con 0x7f59b01005f0 2023-10-26T06:39:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.421+0000 7f59a6ffd700 1 -- 172.21.15.89:0/774868653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac004a00 con 0x7f59b01005f0 2023-10-26T06:39:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.421+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b01a5910 con 0x7f59b01005f0 2023-10-26T06:39:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.422+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b0101d30 con 0x7f59b01005f0 2023-10-26T06:39:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.425+0000 7f59a6ffd700 1 -- 172.21.15.89:0/774868653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f59ac005460 con 0x7f59b01005f0 2023-10-26T06:39:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.428+0000 7f59a6ffd700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f599c0a1440 0x7f599c0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.428+0000 7f59a6ffd700 1 -- 172.21.15.89:0/774868653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f59ac0cc620 con 0x7f59b01005f0 2023-10-26T06:39:31.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.428+0000 7f59b533d700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f599c0a1440 0x7f599c0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.428+0000 7f59b533d700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f599c0a1440 0x7f599c0a3900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f59b0199c20 tx=0x7f59a0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.429+0000 7f59a6ffd700 1 -- 172.21.15.89:0/774868653 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f59ac097870 con 0x7f59b01005f0 2023-10-26T06:39:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.545+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdf88074120 con 0x7fdf88072a20 2023-10-26T06:39:31.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.549+0000 7fdf7effd700 1 -- 172.21.15.89:0/167172179 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdf88074120 con 0x7fdf88072a20 2023-10-26T06:39:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.551+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf740a1440 msgr2=0x7fdf740a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.551+0000 7fdf90040700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf740a1440 0x7fdf740a3900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdf8819a480 tx=0x7fdf78009260 comp rx=0 tx=0).stop 2023-10-26T06:39:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.551+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 msgr2=0x7fdf88198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.551+0000 7fdf90040700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88198f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf84005950 tx=0x7fdf8400fa00 comp rx=0 tx=0).stop 2023-10-26T06:39:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 shutdown_connections 2023-10-26T06:39:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf740a1440 0x7fdf740a3900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf88072a20 0x7fdf88198f20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf8810d250 0x7fdf8819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 --2- 172.21.15.89:0/167172179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf88073340 0x7fdf88199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 >> 172.21.15.89:0/167172179 conn(0x7fdf880779e0 msgr2=0x7fdf880fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:31.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 shutdown_connections 2023-10-26T06:39:31.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.552+0000 7fdf90040700 1 -- 172.21.15.89:0/167172179 wait complete. 2023-10-26T06:39:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.645+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f59b004e4c0 con 0x7f59b01005f0 2023-10-26T06:39:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.650+0000 7f59a6ffd700 1 -- 172.21.15.89:0/774868653 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v31)=0 v31) v1 ==== 120+0+0 (secure 0 0 0) 0x7f59ac09ae90 con 0x7f59b01005f0 2023-10-26T06:39:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.653+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f599c0a1440 msgr2=0x7f599c0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.653+0000 7f59b6b40700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f599c0a1440 0x7f599c0a3900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f59b0199c20 tx=0x7f59a0009260 comp rx=0 tx=0).stop 2023-10-26T06:39:31.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b01005f0 msgr2=0x7f59b019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:31.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b01005f0 0x7f59b019e920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000c00 tx=0x7f59ac004730 comp rx=0 tx=0).stop 2023-10-26T06:39:31.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 shutdown_connections 2023-10-26T06:39:31.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f599c0a1440 0x7f599c0a3900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59b01005f0 0x7f59b019e920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b0106680 0x7f59b0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 --2- 172.21.15.89:0/774868653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59b0100fb0 0x7f59b019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:31.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 >> 172.21.15.89:0/774868653 conn(0x7f59b00fc120 msgr2=0x7f59b00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:31.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 shutdown_connections 2023-10-26T06:39:31.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:31.654+0000 7f59b6b40700 1 -- 172.21.15.89:0/774868653 wait complete. 2023-10-26T06:39:32.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:32 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/167172179' entity='client.admin' 2023-10-26T06:39:32.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:32.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:32.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:32.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:32.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:32 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/774868653' entity='client.admin' 2023-10-26T06:39:32.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:32 smithi089 ceph-mon[95031]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:32.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:32 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/167172179' entity='client.admin' 2023-10-26T06:39:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:32.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:32 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/774868653' entity='client.admin' 2023-10-26T06:39:32.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:32 smithi192 ceph-mon[131661]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:32.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:32 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/167172179' entity='client.admin' 2023-10-26T06:39:32.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:32.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:32.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:32.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:32.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:32 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/774868653' entity='client.admin' 2023-10-26T06:39:32.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:32 smithi135 ceph-mon[131403]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:33.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 -- 172.21.15.89:0/3389971909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 msgr2=0x7fa728100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 --2- 172.21.15.89:0/3389971909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 0x7fa728100d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098f0 tx=0x7fa718009c00 comp rx=0 tx=0).stop 2023-10-26T06:39:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 -- 172.21.15.89:0/3389971909 shutdown_connections 2023-10-26T06:39:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 --2- 172.21.15.89:0/3389971909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 0x7fa72810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 --2- 172.21.15.89:0/3389971909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7281057c0 0x7fa728105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 --2- 172.21.15.89:0/3389971909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 0x7fa728100d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 -- 172.21.15.89:0/3389971909 >> 172.21.15.89:0/3389971909 conn(0x7fa7280779e0 msgr2=0x7fa728077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 -- 172.21.15.89:0/3389971909 shutdown_connections 2023-10-26T06:39:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.127+0000 7fa72f53d700 1 -- 172.21.15.89:0/3389971909 wait complete. 2023-10-26T06:39:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.128+0000 7fa72f53d700 1 Processor -- start 2023-10-26T06:39:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72f53d700 1 -- start start 2023-10-26T06:39:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72f53d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 0x7fa728198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72f53d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 0x7fa7281993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72f53d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7281057c0 0x7fa72819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72f53d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819dd80 con 0x7fa728101260 2023-10-26T06:39:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72f53d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819def0 con 0x7fa7281057c0 2023-10-26T06:39:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72f53d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819e0b0 con 0x7fa7281008a0 2023-10-26T06:39:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72cad8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 0x7fa7281993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72cad8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 0x7fa7281993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48900/0 (socket says 172.21.15.89:48900) 2023-10-26T06:39:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72cad8700 1 -- 172.21.15.89:0/2572636081 learned_addr learned my addr 172.21.15.89:0/2572636081 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72d2d9700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 0x7fa728198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.129+0000 7fa72dada700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7281057c0 0x7fa72819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72cad8700 1 -- 172.21.15.89:0/2572636081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 msgr2=0x7fa728198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72cad8700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 0x7fa728198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72cad8700 1 -- 172.21.15.89:0/2572636081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7281057c0 msgr2=0x7fa72819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72cad8700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7281057c0 0x7fa72819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72cad8700 1 -- 172.21.15.89:0/2572636081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718009520 con 0x7fa728101260 2023-10-26T06:39:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72d2d9700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 0x7fa728198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72cad8700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 0x7fa7281993c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa718005950 tx=0x7fa71800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa71e7fc700 1 -- 172.21.15.89:0/2572636081 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71801c070 con 0x7fa728101260 2023-10-26T06:39:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72819e330 con 0x7fa728101260 2023-10-26T06:39:33.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa71e7fc700 1 -- 172.21.15.89:0/2572636081 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71800bb20 con 0x7fa728101260 2023-10-26T06:39:33.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa71e7fc700 1 -- 172.21.15.89:0/2572636081 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180038e0 con 0x7fa728101260 2023-10-26T06:39:33.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.130+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7281a5bf0 con 0x7fa728101260 2023-10-26T06:39:33.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.131+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa728101f30 con 0x7fa728101260 2023-10-26T06:39:33.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.133+0000 7fa71e7fc700 1 -- 172.21.15.89:0/2572636081 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fa718020030 con 0x7fa728101260 2023-10-26T06:39:33.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.135+0000 7fa71e7fc700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa7140a1440 0x7fa7140a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.136+0000 7fa71e7fc700 1 -- 172.21.15.89:0/2572636081 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa7180cbf80 con 0x7fa728101260 2023-10-26T06:39:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.136+0000 7fa72d2d9700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa7140a1440 0x7fa7140a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.136+0000 7fa72d2d9700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa7140a1440 0x7fa7140a3900 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa728101930 tx=0x7fa724009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.139+0000 7fa71e7fc700 1 -- 172.21.15.89:0/2572636081 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa7180971d0 con 0x7fa728101260 2023-10-26T06:39:33.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.349+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fa72819a190 con 0x7fa728101260 2023-10-26T06:39:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.352+0000 7fa71e7fc700 1 -- 172.21.15.89:0/2572636081 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v32)=0 v32) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa71809a7f0 con 0x7fa728101260 2023-10-26T06:39:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa7140a1440 msgr2=0x7fa7140a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa7140a1440 0x7fa7140a3900 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa728101930 tx=0x7fa724009310 comp rx=0 tx=0).stop 2023-10-26T06:39:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 msgr2=0x7fa7281993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 0x7fa7281993c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa718005950 tx=0x7fa71800fa00 comp rx=0 tx=0).stop 2023-10-26T06:39:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 shutdown_connections 2023-10-26T06:39:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa7140a1440 0x7fa7140a3900 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7281057c0 0x7fa72819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7281008a0 0x7fa728198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 --2- 172.21.15.89:0/2572636081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa728101260 0x7fa7281993c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 >> 172.21.15.89:0/2572636081 conn(0x7fa7280779e0 msgr2=0x7fa72810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 shutdown_connections 2023-10-26T06:39:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.355+0000 7fa72f53d700 1 -- 172.21.15.89:0/2572636081 wait complete. 2023-10-26T06:39:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 -- 172.21.15.89:0/1909469655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714072a20 msgr2=0x7f7714072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 --2- 172.21.15.89:0/1909469655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714072a20 0x7f7714072e00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7710009990 tx=0x7f7710009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 -- 172.21.15.89:0/1909469655 shutdown_connections 2023-10-26T06:39:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 --2- 172.21.15.89:0/1909469655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f771410d1f0 0x7f771410f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 --2- 172.21.15.89:0/1909469655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714072a20 0x7f7714072e00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 --2- 172.21.15.89:0/1909469655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7714073340 0x7f771410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 -- 172.21.15.89:0/1909469655 >> 172.21.15.89:0/1909469655 conn(0x7f7714077950 msgr2=0x7f7714077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 -- 172.21.15.89:0/1909469655 shutdown_connections 2023-10-26T06:39:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.406+0000 7f771bf75700 1 -- 172.21.15.89:0/1909469655 wait complete. 2023-10-26T06:39:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.407+0000 7f771bf75700 1 Processor -- start 2023-10-26T06:39:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.407+0000 7f771bf75700 1 -- start start 2023-10-26T06:39:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f771bf75700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7714072a20 0x7f7714198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f771bf75700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714073340 0x7f7714199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f771bf75700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f771410d1f0 0x7f771419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f771bf75700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771419dcc0 con 0x7f7714072a20 2023-10-26T06:39:33.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f771bf75700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771419de30 con 0x7f7714073340 2023-10-26T06:39:33.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f771bf75700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771419dff0 con 0x7f771410d1f0 2023-10-26T06:39:33.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f7719d11700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7714072a20 0x7f7714198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f7719d11700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7714072a20 0x7f7714198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48924/0 (socket says 172.21.15.89:48924) 2023-10-26T06:39:33.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f7719d11700 1 -- 172.21.15.89:0/3481281393 learned_addr learned my addr 172.21.15.89:0/3481281393 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:33.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f7719510700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714073340 0x7f7714199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.408+0000 7f771a512700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f771410d1f0 0x7f771419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771a512700 1 -- 172.21.15.89:0/3481281393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714073340 msgr2=0x7f7714199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771a512700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714073340 0x7f7714199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771a512700 1 -- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7714072a20 msgr2=0x7f7714198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771a512700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7714072a20 0x7f7714198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771a512700 1 -- 172.21.15.89:0/3481281393 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77100095f0 con 0x7f771410d1f0 2023-10-26T06:39:33.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f7719510700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714073340 0x7f7714199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:39:33.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771a512700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f771410d1f0 0x7f771419d6b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f770c007f10 tx=0x7f770c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:33.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f770affd700 1 -- 172.21.15.89:0/3481281393 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c004db0 con 0x7f771410d1f0 2023-10-26T06:39:33.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771419e2d0 con 0x7f771410d1f0 2023-10-26T06:39:33.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f770affd700 1 -- 172.21.15.89:0/3481281393 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f770c00f040 con 0x7f771410d1f0 2023-10-26T06:39:33.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f770affd700 1 -- 172.21.15.89:0/3481281393 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c005990 con 0x7f771410d1f0 2023-10-26T06:39:33.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.409+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77141a5b30 con 0x7f771410d1f0 2023-10-26T06:39:33.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.410+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7714108400 con 0x7f771410d1f0 2023-10-26T06:39:33.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.412+0000 7f770affd700 1 -- 172.21.15.89:0/3481281393 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f770c00db20 con 0x7f771410d1f0 2023-10-26T06:39:33.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.413+0000 7f770affd700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77000a1440 0x7f77000a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:33.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.414+0000 7f770affd700 1 -- 172.21.15.89:0/3481281393 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f770c0ca9f0 con 0x7f771410d1f0 2023-10-26T06:39:33.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.414+0000 7f7719d11700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77000a1440 0x7f77000a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:33.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.414+0000 7f7719d11700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77000a1440 0x7f77000a3900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7710006010 tx=0x7f771000b560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:33.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.417+0000 7f770affd700 1 -- 172.21.15.89:0/3481281393 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f770c04f100 con 0x7f771410d1f0 2023-10-26T06:39:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.629+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f77140740c0 con 0x7f771410d1f0 2023-10-26T06:39:33.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.635+0000 7f770affd700 1 -- 172.21.15.89:0/3481281393 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f770c095bb0 con 0x7f771410d1f0 2023-10-26T06:39:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77000a1440 msgr2=0x7f77000a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77000a1440 0x7f77000a3900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7710006010 tx=0x7f771000b560 comp rx=0 tx=0).stop 2023-10-26T06:39:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f771410d1f0 msgr2=0x7f771419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f771410d1f0 0x7f771419d6b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f770c007f10 tx=0x7f770c00d3b0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 shutdown_connections 2023-10-26T06:39:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77000a1440 0x7f77000a3900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7714072a20 0x7f7714198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f771410d1f0 0x7f771419d6b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 --2- 172.21.15.89:0/3481281393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714073340 0x7f7714199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:33.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.637+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 >> 172.21.15.89:0/3481281393 conn(0x7f7714077950 msgr2=0x7f77140fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.638+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 shutdown_connections 2023-10-26T06:39:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:33.638+0000 7f771bf75700 1 -- 172.21.15.89:0/3481281393 wait complete. 2023-10-26T06:39:34.043 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi089') 2023-10-26T06:39:34.043 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi089') 2023-10-26T06:39:34.043 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi089') 2023-10-26T06:39:34.044 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi089') 2023-10-26T06:39:34.285 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi089') 2023-10-26T06:39:34.286 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi089') 2023-10-26T06:39:34.286 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e22b2d1a-73c9-11ee-8db9-212e2dc638e7', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi089') 2023-10-26T06:39:34.287 INFO:tasks.cephadm:Deploying OSDs... 2023-10-26T06:39:34.287 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:39:34.287 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2023-10-26T06:39:34.335 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-26T06:39:34.335 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2023-10-26T06:39:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-26T06:39:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:34.427 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 451 Links: 1 2023-10-26T06:39:34.427 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:34.427 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:34.427 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-26 06:38:00.312565692 +0000 2023-10-26T06:39:34.427 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-26 06:30:02.730621263 +0000 2023-10-26T06:39:34.427 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-26 06:30:02.730621263 +0000 2023-10-26T06:39:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-26 06:30:02.701621910 +0000 2023-10-26T06:39:34.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-26T06:39:34.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:34 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2572636081' entity='client.admin' 2023-10-26T06:39:34.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:34 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' 2023-10-26T06:39:34.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:34.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:34 smithi089 ceph-mon[95031]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:34.535 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:34.535 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:34.535 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-26T06:39:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-26T06:39:34.540 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000290704 s, 1.8 MB/s 2023-10-26T06:39:34.543 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-26T06:39:34.589 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2023-10-26T06:39:34.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:34 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2572636081' entity='client.admin' 2023-10-26T06:39:34.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:34 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' 2023-10-26T06:39:34.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:34.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:34 smithi192 ceph-mon[131661]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:34.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:34.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:34.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:34.676 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-26T06:39:34.676 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:34.677 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 457 Links: 1 2023-10-26T06:39:34.677 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:34.677 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:34.677 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-26 06:38:00.399563741 +0000 2023-10-26T06:39:34.677 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-26 06:30:03.551602942 +0000 2023-10-26T06:39:34.678 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-26 06:30:03.551602942 +0000 2023-10-26T06:39:34.678 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-26 06:30:03.523603567 +0000 2023-10-26T06:39:34.678 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-26T06:39:34.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:34 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2572636081' entity='client.admin' 2023-10-26T06:39:34.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:34 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' 2023-10-26T06:39:34.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:39:34.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:34 smithi135 ceph-mon[131403]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:34.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:34.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:39:34.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:39:34.787 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-26T06:39:34.787 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-26T06:39:34.789 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000327769 s, 1.6 MB/s 2023-10-26T06:39:34.790 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-26T06:39:34.837 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2023-10-26T06:39:34.923 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-26T06:39:34.924 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:34.924 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 463 Links: 1 2023-10-26T06:39:34.924 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:34.925 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:34.925 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-26 06:38:00.480561925 +0000 2023-10-26T06:39:34.925 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-26 06:30:04.401583975 +0000 2023-10-26T06:39:34.925 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-26 06:30:04.401583975 +0000 2023-10-26T06:39:34.925 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-26 06:30:04.373584600 +0000 2023-10-26T06:39:34.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-26T06:39:35.032 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-26T06:39:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-26T06:39:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000321329 s, 1.6 MB/s 2023-10-26T06:39:35.035 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-26T06:39:35.081 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2023-10-26T06:39:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-26T06:39:35.165 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:35.165 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 469 Links: 1 2023-10-26T06:39:35.166 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:35.166 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:35.166 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-26 06:38:00.556560221 +0000 2023-10-26T06:39:35.166 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-26 06:30:05.254564941 +0000 2023-10-26T06:39:35.166 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-26 06:30:05.254564941 +0000 2023-10-26T06:39:35.167 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-26 06:30:05.228565521 +0000 2023-10-26T06:39:35.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-26T06:39:35.269 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-10-26T06:39:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-10-26T06:39:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000334791 s, 1.5 MB/s 2023-10-26T06:39:35.272 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-26T06:39:35.317 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-10-26T06:39:35.318 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2023-10-26T06:39:35.339 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-26T06:39:35.340 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2023-10-26T06:39:35.400 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-26T06:39:35.401 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:35.401 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 377919 Links: 1 2023-10-26T06:39:35.401 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:35.401 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:35.401 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-10-26 06:39:14.280062033 +0000 2023-10-26T06:39:35.402 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-10-26 06:26:58.556215898 +0000 2023-10-26T06:39:35.402 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-10-26 06:26:58.556215898 +0000 2023-10-26T06:39:35.402 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-10-26T06:39:35.403 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-26T06:39:35.473 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-10-26T06:39:35.474 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-10-26T06:39:35.474 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000333974 s, 1.5 MB/s 2023-10-26T06:39:35.475 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-26T06:39:35.535 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2023-10-26T06:39:35.594 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-26T06:39:35.595 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:35.595 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 378957 Links: 1 2023-10-26T06:39:35.595 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:35.595 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:35.595 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-10-26 06:39:14.280062033 +0000 2023-10-26T06:39:35.596 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-10-26 06:26:59.054205408 +0000 2023-10-26T06:39:35.596 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-10-26 06:26:59.054205408 +0000 2023-10-26T06:39:35.596 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-10-26T06:39:35.596 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-26T06:39:35.662 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-10-26T06:39:35.663 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-10-26T06:39:35.663 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000218117 s, 2.3 MB/s 2023-10-26T06:39:35.664 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-26T06:39:35.723 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2023-10-26T06:39:35.781 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-26T06:39:35.782 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:35.782 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 371110 Links: 1 2023-10-26T06:39:35.782 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:35.782 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:35.782 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-10-26 06:39:14.280062033 +0000 2023-10-26T06:39:35.783 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-10-26 06:26:59.624193400 +0000 2023-10-26T06:39:35.783 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-10-26 06:26:59.624193400 +0000 2023-10-26T06:39:35.783 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-10-26T06:39:35.784 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-26T06:39:35.848 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-10-26T06:39:35.849 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-10-26T06:39:35.849 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000222374 s, 2.3 MB/s 2023-10-26T06:39:35.850 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-26T06:39:35.909 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2023-10-26T06:39:35.967 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-26T06:39:35.967 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:35.967 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 373664 Links: 1 2023-10-26T06:39:35.967 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:35.968 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:35.968 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-10-26 06:39:14.280062033 +0000 2023-10-26T06:39:35.968 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-10-26 06:27:00.126182826 +0000 2023-10-26T06:39:35.968 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-10-26 06:27:00.126182826 +0000 2023-10-26T06:39:35.968 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-10-26T06:39:35.969 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-26T06:39:36.033 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-10-26T06:39:36.033 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-10-26T06:39:36.034 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.00019838 s, 2.6 MB/s 2023-10-26T06:39:36.037 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-26T06:39:36.093 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-10-26T06:39:36.094 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2023-10-26T06:39:36.115 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-26T06:39:36.116 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2023-10-26T06:39:36.176 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-26T06:39:36.176 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:36.176 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 377706 Links: 1 2023-10-26T06:39:36.177 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:36.177 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:36.177 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-10-26 06:39:12.148721830 +0000 2023-10-26T06:39:36.177 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-10-26 06:26:58.581327621 +0000 2023-10-26T06:39:36.177 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-10-26 06:26:58.581327621 +0000 2023-10-26T06:39:36.178 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-10-26T06:39:36.178 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-26T06:39:36.249 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-10-26T06:39:36.249 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-10-26T06:39:36.250 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000274213 s, 1.9 MB/s 2023-10-26T06:39:36.251 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-26T06:39:36.311 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2023-10-26T06:39:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-26T06:39:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:36.371 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 373057 Links: 1 2023-10-26T06:39:36.371 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:36.371 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:36.371 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-10-26 06:39:12.148721830 +0000 2023-10-26T06:39:36.372 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-10-26 06:26:59.083317753 +0000 2023-10-26T06:39:36.372 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-10-26 06:26:59.083317753 +0000 2023-10-26T06:39:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-10-26T06:39:36.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-26T06:39:36.440 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-10-26T06:39:36.441 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-10-26T06:39:36.441 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000237551 s, 2.2 MB/s 2023-10-26T06:39:36.442 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-26T06:39:36.502 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2023-10-26T06:39:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-26T06:39:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:36.562 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 378604 Links: 1 2023-10-26T06:39:36.562 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:36.562 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:36.563 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-10-26 06:39:12.148721830 +0000 2023-10-26T06:39:36.563 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-10-26 06:26:59.643306743 +0000 2023-10-26T06:39:36.563 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-10-26 06:26:59.643306743 +0000 2023-10-26T06:39:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-10-26T06:39:36.564 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-26T06:39:36.629 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-10-26T06:39:36.630 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-10-26T06:39:36.630 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000205245 s, 2.5 MB/s 2023-10-26T06:39:36.631 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-26T06:39:36.691 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2023-10-26T06:39:36.749 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-26T06:39:36.750 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T06:39:36.750 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 373174 Links: 1 2023-10-26T06:39:36.750 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:39:36.750 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T06:39:36.750 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-10-26 06:39:12.148721830 +0000 2023-10-26T06:39:36.751 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-10-26 06:27:00.149296796 +0000 2023-10-26T06:39:36.751 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-10-26 06:27:00.149296796 +0000 2023-10-26T06:39:36.751 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2023-10-26T06:39:36.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-26T06:39:36.816 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-10-26T06:39:36.816 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-10-26T06:39:36.816 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000248994 s, 2.1 MB/s 2023-10-26T06:39:36.817 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-26T06:39:36.876 INFO:tasks.cephadm:Deploying osd.0 on smithi089 with /dev/vg_nvme/lv_4... 2023-10-26T06:39:36.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-26T06:39:36.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:36 smithi192 ceph-mon[131661]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:36.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:36 smithi135 ceph-mon[131403]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:37.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:36 smithi089 ceph-mon[95031]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:38.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:38 smithi135 ceph-mon[131403]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:39.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:38 smithi089 ceph-mon[95031]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:39.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:38 smithi192 ceph-mon[131661]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:39.736 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-26T06:39:39.736 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-26T06:39:39.736 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-10-26T06:39:39.737 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-10-26T06:39:39.737 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0459575 s, 228 MB/s 2023-10-26T06:39:39.737 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-10-26T06:39:40.201 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2023-10-26T06:39:40.935 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:39:41.281 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:41 smithi089 ceph-mon[95031]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:41.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:41 smithi192 ceph-mon[131661]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:41 smithi135 ceph-mon[131403]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:42.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2319539012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd81057c0 msgr2=0x7f7cd8105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:42.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2319539012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd81057c0 0x7f7cd8105ba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009990 tx=0x7f7cd4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:39:42.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2319539012 shutdown_connections 2023-10-26T06:39:42.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2319539012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd81057c0 0x7f7cd8105ba0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2319539012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd8101260 0x7f7cd810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2319539012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd81008a0 0x7f7cd8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2319539012 >> 172.21.15.89:0/2319539012 conn(0x7f7cd80779e0 msgr2=0x7f7cd8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:39:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2319539012 shutdown_connections 2023-10-26T06:39:42.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.417+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2319539012 wait complete. 2023-10-26T06:39:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.418+0000 7f7cdf8ac700 1 Processor -- start 2023-10-26T06:39:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.418+0000 7f7cdf8ac700 1 -- start start 2023-10-26T06:39:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdf8ac700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd81008a0 0x7f7cd8198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdf8ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd8101260 0x7f7cd81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdf8ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd81057c0 0x7f7cd819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:42.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdf8ac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd819dd60 con 0x7f7cd8101260 2023-10-26T06:39:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdf8ac700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd819ded0 con 0x7f7cd81008a0 2023-10-26T06:39:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdf8ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd819e090 con 0x7f7cd81057c0 2023-10-26T06:39:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd8101260 0x7f7cd81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd8101260 0x7f7cd81993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58850/0 (socket says 172.21.15.89:58850) 2023-10-26T06:39:42.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 -- 172.21.15.89:0/2661576370 learned_addr learned my addr 172.21.15.89:0/2661576370 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:39:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdd648700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd81008a0 0x7f7cd8198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdde49700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd81057c0 0x7f7cd819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 -- 172.21.15.89:0/2661576370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd81057c0 msgr2=0x7f7cd819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd81057c0 0x7f7cd819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 -- 172.21.15.89:0/2661576370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd81008a0 msgr2=0x7f7cd8198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:39:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd81008a0 0x7f7cd8198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:39:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.419+0000 7f7cdce47700 1 -- 172.21.15.89:0/2661576370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd40095f0 con 0x7f7cd8101260 2023-10-26T06:39:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.420+0000 7f7cdce47700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd8101260 0x7f7cd81993a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800d980 tx=0x7f7cc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.420+0000 7f7cce7fc700 1 -- 172.21.15.89:0/2661576370 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8004220 con 0x7f7cd8101260 2023-10-26T06:39:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.420+0000 7f7cce7fc700 1 -- 172.21.15.89:0/2661576370 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cc8007640 con 0x7f7cd8101260 2023-10-26T06:39:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.420+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd819e370 con 0x7f7cd8101260 2023-10-26T06:39:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.420+0000 7f7cce7fc700 1 -- 172.21.15.89:0/2661576370 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8005690 con 0x7f7cd8101260 2023-10-26T06:39:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.420+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd81a5bd0 con 0x7f7cd8101260 2023-10-26T06:39:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.421+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd8101f30 con 0x7f7cd8101260 2023-10-26T06:39:42.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.423+0000 7f7cce7fc700 1 -- 172.21.15.89:0/2661576370 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f7cc8004750 con 0x7f7cd8101260 2023-10-26T06:39:42.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.424+0000 7f7cce7fc700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7cc40a1440 0x7f7cc40a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:39:42.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.424+0000 7f7cce7fc700 1 -- 172.21.15.89:0/2661576370 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7cc80caf20 con 0x7f7cd8101260 2023-10-26T06:39:42.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.425+0000 7f7cdd648700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7cc40a1440 0x7f7cc40a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:39:42.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.425+0000 7f7cdd648700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7cc40a1440 0x7f7cc40a3900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400b2e0 tx=0x7f7cd4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:39:42.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.429+0000 7f7cce7fc700 1 -- 172.21.15.89:0/2661576370 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7cc80960e0 con 0x7f7cd8101260 2023-10-26T06:39:42.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:39:42.640+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cd8110760 con 0x7f7cc40a1440 2023-10-26T06:39:43.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:43 smithi089 ceph-mon[95031]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:43.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:43 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:39:43.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:43 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:39:43.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:43 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:43 smithi192 ceph-mon[131661]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:43 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:39:43.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:43 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:39:43.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:43 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:43 smithi135 ceph-mon[131403]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:43 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:39:43.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:43 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:39:43.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:43 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:44.187 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:44 smithi089 ceph-mon[95031]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:39:44.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:44 smithi192 ceph-mon[131661]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:39:44.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:44 smithi135 ceph-mon[131403]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:39:45.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:45 smithi089 ceph-mon[95031]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:45.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:45 smithi192 ceph-mon[131661]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:45.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:45 smithi135 ceph-mon[131403]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:46.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:46 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2080816555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54028a78-a197-4bc3-b38b-f3e792bb42b5"}]: dispatch 2023-10-26T06:39:46.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:46 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2080816555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54028a78-a197-4bc3-b38b-f3e792bb42b5"}]': finished 2023-10-26T06:39:46.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:46 smithi089 ceph-mon[95031]: osdmap e6: 1 total, 0 up, 1 in 2023-10-26T06:39:46.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:46 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:39:46.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:46 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2080816555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54028a78-a197-4bc3-b38b-f3e792bb42b5"}]: dispatch 2023-10-26T06:39:46.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:46 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2080816555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54028a78-a197-4bc3-b38b-f3e792bb42b5"}]': finished 2023-10-26T06:39:46.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:46 smithi192 ceph-mon[131661]: osdmap e6: 1 total, 0 up, 1 in 2023-10-26T06:39:46.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:46 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:39:46.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:46 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2080816555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54028a78-a197-4bc3-b38b-f3e792bb42b5"}]: dispatch 2023-10-26T06:39:46.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:46 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2080816555' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54028a78-a197-4bc3-b38b-f3e792bb42b5"}]': finished 2023-10-26T06:39:46.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:46 smithi135 ceph-mon[131403]: osdmap e6: 1 total, 0 up, 1 in 2023-10-26T06:39:46.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:46 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:39:47.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:47 smithi192 ceph-mon[131661]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:47.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:47 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1007405658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:39:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:47 smithi135 ceph-mon[131403]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:47 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1007405658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:39:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:47 smithi089 ceph-mon[95031]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:47 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1007405658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:39:49.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:49 smithi192 ceph-mon[131661]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:49.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:49 smithi135 ceph-mon[131403]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:49 smithi089 ceph-mon[95031]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:51.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:51 smithi192 ceph-mon[131661]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:51.410 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:51 smithi089 ceph-mon[95031]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:51.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:51 smithi135 ceph-mon[131403]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:53.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:53 smithi089 ceph-mon[95031]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:53.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:53 smithi192 ceph-mon[131661]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:53.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:53 smithi135 ceph-mon[131403]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:54.333 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-26T06:39:54.334 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:54.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-26T06:39:54.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-26T06:39:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:39:55.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:55 smithi192 ceph-mon[131661]: Deploying daemon osd.0 on smithi089 2023-10-26T06:39:55.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:55 smithi192 ceph-mon[131661]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:55 smithi135 ceph-mon[131403]: Deploying daemon osd.0 on smithi089 2023-10-26T06:39:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:55 smithi135 ceph-mon[131403]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:55.512 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:55 smithi089 ceph-mon[95031]: Deploying daemon osd.0 on smithi089 2023-10-26T06:39:55.512 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:55 smithi089 ceph-mon[95031]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:57.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:57 smithi192 ceph-mon[131661]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:57 smithi135 ceph-mon[131403]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:57 smithi089 ceph-mon[95031]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:59.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:39:59 smithi192 ceph-mon[131661]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:39:59 smithi135 ceph-mon[131403]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:39:59.514 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:39:59 smithi089 ceph-mon[95031]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:01.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:00 smithi192 ceph-mon[131661]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:01.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:01.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:01.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:01.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:00 smithi192 ceph-mon[131661]: overall HEALTH_OK 2023-10-26T06:40:01.172 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:00 smithi089 ceph-mon[95031]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:01.172 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:01.172 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:01.172 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:01.172 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:00 smithi089 ceph-mon[95031]: overall HEALTH_OK 2023-10-26T06:40:01.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:00 smithi135 ceph-mon[131403]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:01.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:01.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:01.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:01.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:00 smithi135 ceph-mon[131403]: overall HEALTH_OK 2023-10-26T06:40:03.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:02 smithi192 ceph-mon[131661]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:03.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:03.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:03.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:02 smithi135 ceph-mon[131403]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:03.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:03.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:03.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.242 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:02 smithi089 ceph-mon[95031]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:03.242 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.242 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.242 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:03.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:03.243 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.253+0000 7f7cce7fc700 1 -- 172.21.15.89:0/2661576370 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7cd8110760 con 0x7f7cc40a1440 2023-10-26T06:40:03.256 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 0 on host 'smithi089' 2023-10-26T06:40:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7cc40a1440 msgr2=0x7f7cc40a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7cc40a1440 0x7f7cc40a3900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400b2e0 tx=0x7f7cd4005fd0 comp rx=0 tx=0).stop 2023-10-26T06:40:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd8101260 msgr2=0x7f7cd81993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd8101260 0x7f7cd81993a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800d980 tx=0x7f7cc800dc90 comp rx=0 tx=0).stop 2023-10-26T06:40:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 shutdown_connections 2023-10-26T06:40:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7cc40a1440 0x7f7cc40a3900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd81057c0 0x7f7cd819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cd81008a0 0x7f7cd8198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 --2- 172.21.15.89:0/2661576370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cd8101260 0x7f7cd81993a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 >> 172.21.15.89:0/2661576370 conn(0x7f7cd80779e0 msgr2=0x7f7cd810ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:40:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 shutdown_connections 2023-10-26T06:40:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:03.256+0000 7f7cdf8ac700 1 -- 172.21.15.89:0/2661576370 wait complete. 2023-10-26T06:40:03.959 DEBUG:teuthology.orchestra.run.smithi089:osd.0> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.0.service 2023-10-26T06:40:03.964 INFO:tasks.cephadm:Deploying osd.1 on smithi135 with /dev/vg_nvme/lv_4... 2023-10-26T06:40:03.964 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-26T06:40:04.038 INFO:journalctl@ceph.osd.0.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:40:04.537 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:40:04.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:04 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:04.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:04 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:04.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:04 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:04.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:04 smithi089 ceph-mon[95031]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:04.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:04 smithi089 ceph-mon[95031]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-26T06:40:04.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:04 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:04.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:04 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:04.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:04 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:04.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:04 smithi192 ceph-mon[131661]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:04.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:04 smithi192 ceph-mon[131661]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-26T06:40:04.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:04 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:04.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:04 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:04.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:04 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:04.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:04 smithi135 ceph-mon[131403]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:04.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:04 smithi135 ceph-mon[131403]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-26T06:40:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:05 smithi135 ceph-mon[131403]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-26T06:40:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:05 smithi135 ceph-mon[131403]: osdmap e7: 1 total, 0 up, 1 in 2023-10-26T06:40:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:05 smithi135 ceph-mon[131403]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:40:05.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:05.510 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:05 smithi089 ceph-mon[95031]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-26T06:40:05.513 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:05 smithi089 ceph-mon[95031]: osdmap e7: 1 total, 0 up, 1 in 2023-10-26T06:40:05.514 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:05 smithi089 ceph-mon[95031]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:40:05.514 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:05.515 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:05 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-0[117229]: 2023-10-26T06:40:05.282+0000 7fa16026a700 -1 osd.0 0 waiting for initial osdmap 2023-10-26T06:40:05.515 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:05 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-0[117229]: 2023-10-26T06:40:05.286+0000 7fa15a85c700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:40:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:05 smithi192 ceph-mon[131661]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-26T06:40:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:05 smithi192 ceph-mon[131661]: osdmap e7: 1 total, 0 up, 1 in 2023-10-26T06:40:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:05 smithi192 ceph-mon[131661]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:40:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:06.141 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-26T06:40:06.142 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-26T06:40:06.142 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2023-10-26T06:40:06.143 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2023-10-26T06:40:06.143 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110516 s, 949 MB/s 2023-10-26T06:40:06.143 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2023-10-26T06:40:06.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:06 smithi135 ceph-mon[131403]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:40:06.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:06 smithi135 ceph-mon[131403]: osdmap e8: 1 total, 0 up, 1 in 2023-10-26T06:40:06.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:06.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:06.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:06 smithi135 ceph-mon[131403]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:06.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:06 smithi089 ceph-mon[95031]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:40:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:06 smithi089 ceph-mon[95031]: osdmap e8: 1 total, 0 up, 1 in 2023-10-26T06:40:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:06.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:06 smithi089 ceph-mon[95031]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:06.593 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi135:vg_nvme/lv_4 2023-10-26T06:40:06.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:06 smithi192 ceph-mon[131661]: from='osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:40:06.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:06 smithi192 ceph-mon[131661]: osdmap e8: 1 total, 0 up, 1 in 2023-10-26T06:40:06.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:06.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:06.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:06 smithi192 ceph-mon[131661]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-26T06:40:07.149 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:40:07.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:07 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:40:07.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:07 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:40:07.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:07.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:07 smithi089 ceph-mon[95031]: osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] boot 2023-10-26T06:40:07.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:07 smithi089 ceph-mon[95031]: osdmap e9: 1 total, 1 up, 1 in 2023-10-26T06:40:07.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:07.575 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:07 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:40:07.575 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:07 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:40:07.576 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:07.576 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:07 smithi135 ceph-mon[131403]: osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] boot 2023-10-26T06:40:07.576 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:07 smithi135 ceph-mon[131403]: osdmap e9: 1 total, 1 up, 1 in 2023-10-26T06:40:07.577 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:07 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:40:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:07 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:40:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:07 smithi192 ceph-mon[131661]: osd.0 [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] boot 2023-10-26T06:40:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:07 smithi192 ceph-mon[131661]: osdmap e9: 1 total, 1 up, 1 in 2023-10-26T06:40:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-26T06:40:08.309 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 -- 172.21.15.135:0/1768771808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 msgr2=0x7f25b80ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:08.309 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 --2- 172.21.15.135:0/1768771808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 0x7f25b80ff5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009990 tx=0x7f25a8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:40:08.309 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 -- 172.21.15.135:0/1768771808 shutdown_connections 2023-10-26T06:40:08.309 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 --2- 172.21.15.135:0/1768771808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b81058d0 0x7f25b8105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:08.309 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 --2- 172.21.15.135:0/1768771808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25b80ffb30 0x7f25b810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:08.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 --2- 172.21.15.135:0/1768771808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 0x7f25b80ff5f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:08.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 -- 172.21.15.135:0/1768771808 >> 172.21.15.135:0/1768771808 conn(0x7f25b8074a10 msgr2=0x7f25b8074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:40:08.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 -- 172.21.15.135:0/1768771808 shutdown_connections 2023-10-26T06:40:08.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 -- 172.21.15.135:0/1768771808 wait complete. 2023-10-26T06:40:08.312 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.307+0000 7f25bf71d700 1 Processor -- start 2023-10-26T06:40:08.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bf71d700 1 -- start start 2023-10-26T06:40:08.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bf71d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 0x7f25b819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:08.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bf71d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25b80ffb30 0x7f25b819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:08.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bf71d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b81058d0 0x7f25b8196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:08.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bf71d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8197320 con 0x7f25b81058d0 2023-10-26T06:40:08.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bf71d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8197490 con 0x7f25b80ff190 2023-10-26T06:40:08.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bf71d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8197650 con 0x7f25b80ffb30 2023-10-26T06:40:08.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bccb8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25b80ffb30 0x7f25b819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:08.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bd4b9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 0x7f25b819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:08.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bccb8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25b80ffb30 0x7f25b819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.135:35518/0 (socket says 172.21.15.135:35518) 2023-10-26T06:40:08.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bccb8700 1 -- 172.21.15.135:0/3305517691 learned_addr learned my addr 172.21.15.135:0/3305517691 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-10-26T06:40:08.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bd4b9700 1 -- 172.21.15.135:0/3305517691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25b80ffb30 msgr2=0x7f25b819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:08.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bd4b9700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25b80ffb30 0x7f25b819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:08.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bd4b9700 1 -- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b81058d0 msgr2=0x7f25b8196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:08.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bd4b9700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b81058d0 0x7f25b8196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:08.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bd4b9700 1 -- 172.21.15.135:0/3305517691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80095f0 con 0x7f25b80ff190 2023-10-26T06:40:08.316 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25bd4b9700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 0x7f25b819ca50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f25b8068160 tx=0x7f25b400b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:40:08.316 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.308+0000 7f25ae7fc700 1 -- 172.21.15.135:0/3305517691 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4011830 con 0x7f25b80ff190 2023-10-26T06:40:08.316 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.309+0000 7f25ae7fc700 1 -- 172.21.15.135:0/3305517691 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b4011e50 con 0x7f25b80ff190 2023-10-26T06:40:08.316 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.309+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b8197930 con 0x7f25b80ff190 2023-10-26T06:40:08.316 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.309+0000 7f25ae7fc700 1 -- 172.21.15.135:0/3305517691 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4010de0 con 0x7f25b80ff190 2023-10-26T06:40:08.317 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.309+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b81a3a20 con 0x7f25b80ff190 2023-10-26T06:40:08.317 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.310+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b8108d30 con 0x7f25b80ff190 2023-10-26T06:40:08.317 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.310+0000 7f25ae7fc700 1 -- 172.21.15.135:0/3305517691 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f25b4010480 con 0x7f25b80ff190 2023-10-26T06:40:08.317 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.311+0000 7f25ae7fc700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25a40a14c0 0x7f25a40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:08.317 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.311+0000 7f25ae7fc700 1 -- 172.21.15.135:0/3305517691 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f25b40cacd0 con 0x7f25b80ff190 2023-10-26T06:40:08.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.311+0000 7f25bccb8700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25a40a14c0 0x7f25a40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:08.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.312+0000 7f25bccb8700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25a40a14c0 0x7f25a40a3980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f25b8198270 tx=0x7f25a800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:40:08.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.315+0000 7f25ae7fc700 1 -- 172.21.15.135:0/3305517691 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f25b4095cd0 con 0x7f25b80ff190 2023-10-26T06:40:08.514 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:08.512+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f25b8197fe0 con 0x7f25a40a14c0 2023-10-26T06:40:08.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: osdmap e10: 1 total, 1 up, 1 in 2023-10-26T06:40:08.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi089 2023-10-26T06:40:08.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:08.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi089 to 17271M 2023-10-26T06:40:08.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:08.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:08.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:08 smithi192 ceph-mon[131661]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:08.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: osdmap e10: 1 total, 1 up, 1 in 2023-10-26T06:40:08.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi089 2023-10-26T06:40:08.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:08.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi089 to 17271M 2023-10-26T06:40:08.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:08.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:08.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:08 smithi135 ceph-mon[131403]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:08.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: osdmap e10: 1 total, 1 up, 1 in 2023-10-26T06:40:08.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi089 2023-10-26T06:40:08.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:08.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi089 to 17271M 2023-10-26T06:40:08.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:08.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:08.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:08.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:08 smithi089 ceph-mon[95031]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:09.649 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:09 smithi135 ceph-mon[131403]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:09.649 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:09.650 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:09.650 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:09.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:09 smithi089 ceph-mon[95031]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:09.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:09.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:09.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:09.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:09 smithi192 ceph-mon[131661]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:09.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:09.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:09.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:10 smithi089 ceph-mon[95031]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:10.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:10 smithi192 ceph-mon[131661]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:10.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:10 smithi135 ceph-mon[131403]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:11.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:11 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/1369120203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]: dispatch 2023-10-26T06:40:11.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:11 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]: dispatch 2023-10-26T06:40:11.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:11 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]': finished 2023-10-26T06:40:11.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:11 smithi089 ceph-mon[95031]: osdmap e11: 2 total, 1 up, 2 in 2023-10-26T06:40:11.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:11.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:11 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/1369120203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]: dispatch 2023-10-26T06:40:11.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:11 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]: dispatch 2023-10-26T06:40:11.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:11 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]': finished 2023-10-26T06:40:11.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:11 smithi192 ceph-mon[131661]: osdmap e11: 2 total, 1 up, 2 in 2023-10-26T06:40:11.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:11.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:11 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/1369120203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]: dispatch 2023-10-26T06:40:11.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:11 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]: dispatch 2023-10-26T06:40:11.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:11 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3"}]': finished 2023-10-26T06:40:11.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:11 smithi135 ceph-mon[131403]: osdmap e11: 2 total, 1 up, 2 in 2023-10-26T06:40:11.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:12.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:12 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/1909580109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:12.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:12 smithi089 ceph-mon[95031]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:12.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:12 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/1909580109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:12.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:12 smithi192 ceph-mon[131661]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:12.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:12 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/1909580109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:12.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:12 smithi135 ceph-mon[131403]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:15.128 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:15 smithi135 ceph-mon[131403]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:15.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:15 smithi089 ceph-mon[95031]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:15.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:15 smithi192 ceph-mon[131661]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:17.198 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:17 smithi135 ceph-mon[131403]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:17.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:17 smithi089 ceph-mon[95031]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:17.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:17 smithi192 ceph-mon[131661]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:18.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-26T06:40:18.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:18.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-26T06:40:18.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:18.415 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-26T06:40:18.415 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:19.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:19 smithi089 ceph-mon[95031]: Deploying daemon osd.1 on smithi135 2023-10-26T06:40:19.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:19 smithi089 ceph-mon[95031]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:19 smithi192 ceph-mon[131661]: Deploying daemon osd.1 on smithi135 2023-10-26T06:40:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:19 smithi192 ceph-mon[131661]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:19.432 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:19 smithi135 ceph-mon[131403]: Deploying daemon osd.1 on smithi135 2023-10-26T06:40:19.432 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:19 smithi135 ceph-mon[131403]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:21 smithi192 ceph-mon[131661]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:21 smithi135 ceph-mon[131403]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:21.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:21 smithi089 ceph-mon[95031]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:22.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:22 smithi135 ceph-mon[131403]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:22.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:22.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:22.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:23.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:22 smithi192 ceph-mon[131661]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:23.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:23.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:23.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:23.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:22 smithi089 ceph-mon[95031]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:23.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:23.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:23.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:24.579 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 1 on host 'smithi135' 2023-10-26T06:40:24.579 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.577+0000 7f25ae7fc700 1 -- 172.21.15.135:0/3305517691 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f25b8197fe0 con 0x7f25a40a14c0 2023-10-26T06:40:24.581 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25a40a14c0 msgr2=0x7f25a40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:24.581 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25a40a14c0 0x7f25a40a3980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f25b8198270 tx=0x7f25a800b540 comp rx=0 tx=0).stop 2023-10-26T06:40:24.582 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 msgr2=0x7f25b819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:24.582 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 0x7f25b819ca50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f25b8068160 tx=0x7f25b400b730 comp rx=0 tx=0).stop 2023-10-26T06:40:24.582 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 shutdown_connections 2023-10-26T06:40:24.582 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25a40a14c0 0x7f25a40a3980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:24.582 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25b81058d0 0x7f25b8196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:24.583 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25b80ff190 0x7f25b819ca50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:24.583 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 --2- 172.21.15.135:0/3305517691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25b80ffb30 0x7f25b819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:24.583 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 >> 172.21.15.135:0/3305517691 conn(0x7f25b8074a10 msgr2=0x7f25b80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:40:24.583 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 shutdown_connections 2023-10-26T06:40:24.584 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:40:24.579+0000 7f25bf71d700 1 -- 172.21.15.135:0/3305517691 wait complete. 2023-10-26T06:40:25.032 DEBUG:teuthology.orchestra.run.smithi135:osd.1> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.1.service 2023-10-26T06:40:25.034 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2023-10-26T06:40:25.035 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-26T06:40:25.068 INFO:journalctl@ceph.osd.1.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:40:25.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:25.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:25.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:25.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:25.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:25.597 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:40:26.136 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:25 smithi192 ceph-mon[131661]: from='osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-26T06:40:26.137 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:25 smithi192 ceph-mon[131661]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-26T06:40:26.193 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:25 smithi135 ceph-mon[131403]: from='osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-26T06:40:26.193 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:25 smithi135 ceph-mon[131403]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-26T06:40:26.194 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:25 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1[137576]: 2023-10-26T06:40:25.905+0000 7f328dbd0640 -1 osd.1 0 log_to_monitors true 2023-10-26T06:40:26.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:25 smithi089 ceph-mon[95031]: from='osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-26T06:40:26.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:25 smithi089 ceph-mon[95031]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-26T06:40:27.124 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-26T06:40:27.124 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-26T06:40:27.125 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-10-26T06:40:27.125 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-10-26T06:40:27.125 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010993 s, 954 MB/s 2023-10-26T06:40:27.125 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-10-26T06:40:27.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:27 smithi192 ceph-mon[131661]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:27.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:27 smithi192 ceph-mon[131661]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-26T06:40:27.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:27 smithi192 ceph-mon[131661]: from='osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:40:27.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:27 smithi192 ceph-mon[131661]: osdmap e12: 2 total, 1 up, 2 in 2023-10-26T06:40:27.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:27.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:27 smithi192 ceph-mon[131661]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:40:27.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-mon[131403]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:27.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-mon[131403]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-26T06:40:27.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-mon[131403]: from='osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:40:27.282 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-mon[131403]: osdmap e12: 2 total, 1 up, 2 in 2023-10-26T06:40:27.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:27.283 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-mon[131403]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:40:27.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1[137576]: 2023-10-26T06:40:27.008+0000 7f3282a46700 -1 osd.1 0 waiting for initial osdmap 2023-10-26T06:40:27.283 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:27 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1[137576]: 2023-10-26T06:40:27.012+0000 7f327d038700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:40:27.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:27 smithi089 ceph-mon[95031]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:27.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:27 smithi089 ceph-mon[95031]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-26T06:40:27.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:27 smithi089 ceph-mon[95031]: from='osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:40:27.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:27 smithi089 ceph-mon[95031]: osdmap e12: 2 total, 1 up, 2 in 2023-10-26T06:40:27.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:27.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:27 smithi089 ceph-mon[95031]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:40:27.553 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2023-10-26T06:40:28.109 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:40:28.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:40:28.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: osdmap e13: 2 total, 1 up, 2 in 2023-10-26T06:40:28.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:28.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:28.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:28.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:28.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:28.309 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:40:28.309 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: osdmap e13: 2 total, 1 up, 2 in 2023-10-26T06:40:28.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:28.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:28.310 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:28.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:28.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.311 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:28.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:40:28.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: osdmap e13: 2 total, 1 up, 2 in 2023-10-26T06:40:28.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:28.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:28.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:28.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:28.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:28.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:29.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:40:29.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:40:29.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi135 2023-10-26T06:40:29.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:29.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi135 to 17168M 2023-10-26T06:40:29.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] boot 2023-10-26T06:40:29.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: osdmap e14: 2 total, 2 up, 2 in 2023-10-26T06:40:29.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:29.294 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 -- 172.21.15.192:0/732215663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb560104720 msgr2=0x7fb560104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:29.294 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 --2- 172.21.15.192:0/732215663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb560104720 0x7fb560104b00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009990 tx=0x7fb55c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:40:29.294 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 -- 172.21.15.192:0/732215663 shutdown_connections 2023-10-26T06:40:29.294 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 --2- 172.21.15.192:0/732215663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb560100c60 0x7fb56010c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:29.295 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 --2- 172.21.15.192:0/732215663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb560104720 0x7fb560104b00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:29.295 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 --2- 172.21.15.192:0/732215663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5601002a0 0x7fb560100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:29.295 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 -- 172.21.15.192:0/732215663 >> 172.21.15.192:0/732215663 conn(0x7fb560074a10 msgr2=0x7fb560074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:40:29.295 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 -- 172.21.15.192:0/732215663 shutdown_connections 2023-10-26T06:40:29.296 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 -- 172.21.15.192:0/732215663 wait complete. 2023-10-26T06:40:29.296 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 Processor -- start 2023-10-26T06:40:29.296 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.291+0000 7fb56781f700 1 -- start start 2023-10-26T06:40:29.296 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5601002a0 0x7fb56019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:29.297 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb560100c60 0x7fb56019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:29.297 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb560104720 0x7fb560195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:29.297 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560195f80 con 0x7fb560100c60 2023-10-26T06:40:29.297 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601960f0 con 0x7fb560104720 2023-10-26T06:40:29.297 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601962b0 con 0x7fb5601002a0 2023-10-26T06:40:29.298 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5601002a0 0x7fb56019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:29.298 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5601002a0 0x7fb56019b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:53966/0 (socket says 172.21.15.192:53966) 2023-10-26T06:40:29.298 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 -- 172.21.15.192:0/2961144499 learned_addr learned my addr 172.21.15.192:0/2961144499 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-10-26T06:40:29.298 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb565dbc700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb560104720 0x7fb560195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:29.298 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb564dba700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb560100c60 0x7fb56019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:29.298 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 -- 172.21.15.192:0/2961144499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb560104720 msgr2=0x7fb560195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:29.299 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb560104720 0x7fb560195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:29.299 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 -- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb560100c60 msgr2=0x7fb56019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:29.299 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb560100c60 0x7fb56019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:29.299 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 -- 172.21.15.192:0/2961144499 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0095f0 con 0x7fb5601002a0 2023-10-26T06:40:29.299 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb564dba700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb560100c60 0x7fb56019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:40:29.300 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5655bb700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5601002a0 0x7fb56019b7e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0098e0 tx=0x7fb55c00f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:40:29.300 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5567fc700 1 -- 172.21.15.192:0/2961144499 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c01d070 con 0x7fb5601002a0 2023-10-26T06:40:29.300 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5567fc700 1 -- 172.21.15.192:0/2961144499 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb55c022470 con 0x7fb5601002a0 2023-10-26T06:40:29.300 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb5567fc700 1 -- 172.21.15.192:0/2961144499 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c017650 con 0x7fb5601002a0 2023-10-26T06:40:29.300 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb560196530 con 0x7fb5601002a0 2023-10-26T06:40:29.301 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.292+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a27d0 con 0x7fb5601002a0 2023-10-26T06:40:29.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.293+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb560107c80 con 0x7fb5601002a0 2023-10-26T06:40:29.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.294+0000 7fb5567fc700 1 -- 172.21.15.192:0/2961144499 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fb55c0225e0 con 0x7fb5601002a0 2023-10-26T06:40:29.302 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.295+0000 7fb5567fc700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb54c0a1440 0x7fb54c0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:29.303 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.295+0000 7fb5567fc700 1 -- 172.21.15.192:0/2961144499 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2524+0+0 (secure 0 0 0) 0x7fb55c0cc960 con 0x7fb5601002a0 2023-10-26T06:40:29.303 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.295+0000 7fb564dba700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb54c0a1440 0x7fb54c0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:29.303 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.296+0000 7fb564dba700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb54c0a1440 0x7fb54c0a3900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb550005fd0 tx=0x7fb550005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:40:29.303 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.298+0000 7fb5567fc700 1 -- 172.21.15.192:0/2961144499 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb55c0977d0 con 0x7fb5601002a0 2023-10-26T06:40:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:40:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:40:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi135 2023-10-26T06:40:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi135 to 17168M 2023-10-26T06:40:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] boot 2023-10-26T06:40:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: osdmap e14: 2 total, 2 up, 2 in 2023-10-26T06:40:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:40:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:40:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi135 2023-10-26T06:40:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-26T06:40:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi135 to 17168M 2023-10-26T06:40:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: osd.1 [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] boot 2023-10-26T06:40:29.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: osdmap e14: 2 total, 2 up, 2 in 2023-10-26T06:40:29.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-26T06:40:29.495 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:29.494+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5601966c0 con 0x7fb54c0a1440 2023-10-26T06:40:30.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:30.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:30.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:30.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:30 smithi089 ceph-mon[95031]: osdmap e15: 2 total, 2 up, 2 in 2023-10-26T06:40:30.342 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:30.343 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:30.343 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:30.343 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:30 smithi192 ceph-mon[131661]: osdmap e15: 2 total, 2 up, 2 in 2023-10-26T06:40:30.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:30.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:30.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:30.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:30 smithi135 ceph-mon[131403]: osdmap e15: 2 total, 2 up, 2 in 2023-10-26T06:40:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:31 smithi089 ceph-mon[95031]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:31 smithi089 ceph-mon[95031]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:31.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:31 smithi192 ceph-mon[131661]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:31.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:31 smithi192 ceph-mon[131661]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:31.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:31 smithi135 ceph-mon[131403]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:31.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:31 smithi135 ceph-mon[131403]: pgmap v46: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:32.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:32 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/1530643531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]: dispatch 2023-10-26T06:40:32.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:32 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]: dispatch 2023-10-26T06:40:32.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:32 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]': finished 2023-10-26T06:40:32.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:32 smithi089 ceph-mon[95031]: osdmap e16: 3 total, 2 up, 3 in 2023-10-26T06:40:32.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:32.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:32 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/1530643531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]: dispatch 2023-10-26T06:40:32.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:32 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]: dispatch 2023-10-26T06:40:32.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:32 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]': finished 2023-10-26T06:40:32.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:32 smithi192 ceph-mon[131661]: osdmap e16: 3 total, 2 up, 3 in 2023-10-26T06:40:32.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:32.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:32 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/1530643531' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]: dispatch 2023-10-26T06:40:32.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:32 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]: dispatch 2023-10-26T06:40:32.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:32 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05df73a-080f-410a-84ad-50df3f2717f7"}]': finished 2023-10-26T06:40:32.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:32 smithi135 ceph-mon[131403]: osdmap e16: 3 total, 2 up, 3 in 2023-10-26T06:40:32.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:33 smithi192 ceph-mon[131661]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:33 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/507796098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:33.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:33 smithi135 ceph-mon[131403]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:33 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/507796098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:33.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:33 smithi089 ceph-mon[95031]: pgmap v48: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:33.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:33 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/507796098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:35.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:35 smithi192 ceph-mon[131661]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:35 smithi135 ceph-mon[131403]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:35 smithi089 ceph-mon[95031]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:37.360 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:37 smithi192 ceph-mon[131661]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:37.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:37 smithi135 ceph-mon[131403]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:37.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:37 smithi089 ceph-mon[95031]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:39 smithi192 ceph-mon[131661]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-26T06:40:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:39 smithi135 ceph-mon[131403]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-26T06:40:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:39.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:39 smithi089 ceph-mon[95031]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:39.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-26T06:40:39.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:40.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:40 smithi192 ceph-mon[131661]: Deploying daemon osd.2 on smithi192 2023-10-26T06:40:40.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:40 smithi135 ceph-mon[131403]: Deploying daemon osd.2 on smithi192 2023-10-26T06:40:40.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:40 smithi089 ceph-mon[95031]: Deploying daemon osd.2 on smithi192 2023-10-26T06:40:41.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:41 smithi192 ceph-mon[131661]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:41 smithi135 ceph-mon[131403]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:41.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:41 smithi089 ceph-mon[95031]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:43 smithi192 ceph-mon[131661]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:43 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:43.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:43 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:43.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:43 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:43 smithi135 ceph-mon[131403]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:43 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:43 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:43.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:43 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:43.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:43 smithi089 ceph-mon[95031]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:43.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:43 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:43.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:43 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:43.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:43 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:45.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:45 smithi192 ceph-mon[131661]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:45.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:45 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:45 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:45 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:45.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:45 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:45.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:45 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:45 smithi135 ceph-mon[131403]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:45.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:45 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:45 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:45 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:45.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:45 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:45.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:45 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:45 smithi089 ceph-mon[95031]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:45.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:45 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:45 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:45 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:45.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:45 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:45.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:45 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:45.559 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.557+0000 7fb5567fc700 1 -- 172.21.15.192:0/2961144499 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb5601966c0 con 0x7fb54c0a1440 2023-10-26T06:40:45.559 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2023-10-26T06:40:45.561 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb54c0a1440 msgr2=0x7fb54c0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:45.561 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb54c0a1440 0x7fb54c0a3900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb550005fd0 tx=0x7fb550005ee0 comp rx=0 tx=0).stop 2023-10-26T06:40:45.562 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5601002a0 msgr2=0x7fb56019b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:45.562 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5601002a0 0x7fb56019b7e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0098e0 tx=0x7fb55c00f670 comp rx=0 tx=0).stop 2023-10-26T06:40:45.562 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 shutdown_connections 2023-10-26T06:40:45.562 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb54c0a1440 0x7fb54c0a3900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:45.562 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb560104720 0x7fb560195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:45.563 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5601002a0 0x7fb56019b7e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:45.563 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 --2- 172.21.15.192:0/2961144499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb560100c60 0x7fb56019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:45.563 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 >> 172.21.15.192:0/2961144499 conn(0x7fb560074a10 msgr2=0x7fb5600fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:40:45.563 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 shutdown_connections 2023-10-26T06:40:45.563 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:40:45.559+0000 7fb56781f700 1 -- 172.21.15.192:0/2961144499 wait complete. 2023-10-26T06:40:45.905 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.2.service 2023-10-26T06:40:45.908 INFO:tasks.cephadm:Deploying osd.3 on smithi089 with /dev/vg_nvme/lv_3... 2023-10-26T06:40:45.909 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-26T06:40:45.950 INFO:journalctl@ceph.osd.2.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:40:46.670 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:40:46.725 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:46 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:46.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:46 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:46.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:46 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:46.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:46 smithi192 ceph-mon[131661]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:46.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:46 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:46.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:46 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:46.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:46 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:46.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:46 smithi089 ceph-mon[95031]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:46.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:46 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:46.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:46 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:46.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:46 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:40:46.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:46 smithi135 ceph-mon[131403]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:47.146 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:46 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2[138885]: 2023-10-26T06:40:46.955+0000 7f0509a57640 -1 osd.2 0 log_to_monitors true 2023-10-26T06:40:47.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:47 smithi089 ceph-mon[95031]: from='osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-26T06:40:47.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:47 smithi089 ceph-mon[95031]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-26T06:40:47.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:47 smithi192 ceph-mon[131661]: from='osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-26T06:40:47.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:47 smithi192 ceph-mon[131661]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-26T06:40:47.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:47 smithi135 ceph-mon[131403]: from='osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-26T06:40:47.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:47 smithi135 ceph-mon[131403]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-26T06:40:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-26T06:40:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-26T06:40:48.761 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-10-26T06:40:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-10-26T06:40:48.762 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0307958 s, 340 MB/s 2023-10-26T06:40:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-10-26T06:40:48.895 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2[138885]: 2023-10-26T06:40:48.545+0000 7f04fe8cd700 -1 osd.2 0 waiting for initial osdmap 2023-10-26T06:40:48.895 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2[138885]: 2023-10-26T06:40:48.551+0000 7f04fa6c2700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:40:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-26T06:40:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:40:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: osdmap e17: 3 total, 2 up, 3 in 2023-10-26T06:40:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:40:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.905 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-26T06:40:48.905 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:40:48.905 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: osdmap e17: 3 total, 2 up, 3 in 2023-10-26T06:40:48.905 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:48.906 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:40:48.906 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:48.906 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.907 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.907 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:48.907 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-26T06:40:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:40:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: osdmap e17: 3 total, 2 up, 3 in 2023-10-26T06:40:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:40:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:48.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:48.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:40:48.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:49.263 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2023-10-26T06:40:49.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi192 2023-10-26T06:40:49.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi192 to 17168M 2023-10-26T06:40:49.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:49.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:49.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:40:49.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: osdmap e18: 3 total, 2 up, 3 in 2023-10-26T06:40:49.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:49.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:49.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:49.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi192 2023-10-26T06:40:49.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi192 to 17168M 2023-10-26T06:40:49.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:49.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:49.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:40:49.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: osdmap e18: 3 total, 2 up, 3 in 2023-10-26T06:40:49.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:49.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:49.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:49.997 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:40:50.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi192 2023-10-26T06:40:50.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi192 to 17168M 2023-10-26T06:40:50.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:50.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:40:50.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:40:50.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: osdmap e18: 3 total, 2 up, 3 in 2023-10-26T06:40:50.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:50.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:50.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:50.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:50 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:40:50.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:50 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:40:50.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:50 smithi089 ceph-mon[95031]: osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] boot 2023-10-26T06:40:50.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:50 smithi089 ceph-mon[95031]: osdmap e19: 3 total, 3 up, 3 in 2023-10-26T06:40:50.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:50 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:50.829 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:50 smithi089 ceph-mon[95031]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:50.829 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:50 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:40:50.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:50 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:40:50.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:50 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:40:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:50 smithi192 ceph-mon[131661]: osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] boot 2023-10-26T06:40:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:50 smithi192 ceph-mon[131661]: osdmap e19: 3 total, 3 up, 3 in 2023-10-26T06:40:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:50 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:50 smithi192 ceph-mon[131661]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:50 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:40:50.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:50 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:40:50.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:50 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:40:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:50 smithi135 ceph-mon[131403]: osd.2 [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] boot 2023-10-26T06:40:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:50 smithi135 ceph-mon[131403]: osdmap e19: 3 total, 3 up, 3 in 2023-10-26T06:40:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:50 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-26T06:40:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:50 smithi135 ceph-mon[131403]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-26T06:40:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:50 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:40:51.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.497+0000 7f0939490700 1 -- 172.21.15.89:0/145902300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09341005f0 msgr2=0x7f0934100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:51.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.497+0000 7f0939490700 1 --2- 172.21.15.89:0/145902300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09341005f0 0x7f0934100a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0928009990 tx=0x7f0928009ca0 comp rx=0 tx=0).stop 2023-10-26T06:40:51.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.498+0000 7f0939490700 1 -- 172.21.15.89:0/145902300 shutdown_connections 2023-10-26T06:40:51.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.498+0000 7f0939490700 1 --2- 172.21.15.89:0/145902300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0934100fb0 0x7f093410d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:51.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.498+0000 7f0939490700 1 --2- 172.21.15.89:0/145902300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0934106680 0x7f0934106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:51.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.498+0000 7f0939490700 1 --2- 172.21.15.89:0/145902300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09341005f0 0x7f0934100a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:51.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.498+0000 7f0939490700 1 -- 172.21.15.89:0/145902300 >> 172.21.15.89:0/145902300 conn(0x7f09340fc120 msgr2=0x7f09340fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:40:51.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.498+0000 7f0939490700 1 -- 172.21.15.89:0/145902300 shutdown_connections 2023-10-26T06:40:51.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.498+0000 7f0939490700 1 -- 172.21.15.89:0/145902300 wait complete. 2023-10-26T06:40:51.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.499+0000 7f0939490700 1 Processor -- start 2023-10-26T06:40:51.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.499+0000 7f0939490700 1 -- start start 2023-10-26T06:40:51.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0939490700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09341005f0 0x7f093419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:51.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0939490700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934100fb0 0x7f093419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:51.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0939490700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0934106680 0x7f0934196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:51.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0939490700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934196eb0 con 0x7f0934100fb0 2023-10-26T06:40:51.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0939490700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934197020 con 0x7f0934106680 2023-10-26T06:40:51.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0939490700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09341971e0 con 0x7f09341005f0 2023-10-26T06:40:51.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f09337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934100fb0 0x7f093419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:51.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f09337fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934100fb0 0x7f093419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51574/0 (socket says 172.21.15.89:51574) 2023-10-26T06:40:51.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f09337fe700 1 -- 172.21.15.89:0/1909726613 learned_addr learned my addr 172.21.15.89:0/1909726613 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:40:51.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0933fff700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09341005f0 0x7f093419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:51.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.500+0000 7f0938c8f700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0934106680 0x7f0934196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:51.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09337fe700 1 -- 172.21.15.89:0/1909726613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09341005f0 msgr2=0x7f093419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:51.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09337fe700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09341005f0 0x7f093419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:51.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09337fe700 1 -- 172.21.15.89:0/1909726613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0934106680 msgr2=0x7f0934196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:40:51.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09337fe700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0934106680 0x7f0934196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:40:51.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09337fe700 1 -- 172.21.15.89:0/1909726613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280095f0 con 0x7f0934100fb0 2023-10-26T06:40:51.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09337fe700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934100fb0 0x7f093419cc40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0928005910 tx=0x7f0928004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:40:51.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092801d070 con 0x7f0934100fb0 2023-10-26T06:40:51.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0928022470 con 0x7f0934100fb0 2023-10-26T06:40:51.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928017650 con 0x7f0934100fb0 2023-10-26T06:40:51.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0934197460 con 0x7f0934100fb0 2023-10-26T06:40:51.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.501+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09341a36f0 con 0x7f0934100fb0 2023-10-26T06:40:51.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.502+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934108c10 con 0x7f0934100fb0 2023-10-26T06:40:51.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.504+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f09280225e0 con 0x7f0934100fb0 2023-10-26T06:40:51.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.506+0000 7f09317fa700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f091c0a14c0 0x7f091c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:40:51.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.506+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3441+0+0 (secure 0 0 0) 0x7f09280cd040 con 0x7f0934100fb0 2023-10-26T06:40:51.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.506+0000 7f0933fff700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f091c0a14c0 0x7f091c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:40:51.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.506+0000 7f0933fff700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f091c0a14c0 0x7f091c0a3980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0934101730 tx=0x7f0924007400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:40:51.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.509+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0928097a70 con 0x7f0934100fb0 2023-10-26T06:40:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:51.737+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0934197d00 con 0x7f091c0a14c0 2023-10-26T06:40:51.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-26T06:40:51.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:51 smithi192 ceph-mon[131661]: osdmap e20: 3 total, 3 up, 3 in 2023-10-26T06:40:51.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:40:51.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-26T06:40:51.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:51 smithi135 ceph-mon[131403]: osdmap e20: 3 total, 3 up, 3 in 2023-10-26T06:40:51.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:40:52.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-26T06:40:52.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:51 smithi089 ceph-mon[95031]: osdmap e20: 3 total, 3 up, 3 in 2023-10-26T06:40:52.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:40:52.539 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123320]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-26T06:40:52.539 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:52.540 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123320]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:52.540 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123320]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:52.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:40:52.585+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f0928097e40 con 0x7f0934100fb0 2023-10-26T06:40:52.834 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123366]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-26T06:40:52.834 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123366]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:52.835 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123366]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:52.835 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:40:52 smithi089 sudo[123366]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:52.835 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:52 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-26T06:40:52.835 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:52 smithi089 ceph-mon[95031]: osdmap e21: 3 total, 3 up, 3 in 2023-10-26T06:40:52.836 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:52 smithi089 ceph-mon[95031]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:52.836 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:52 smithi089 ceph-mon[95031]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:52.836 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:52 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:52.836 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:52 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:52.836 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:52 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:52.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:52 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-26T06:40:52.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:52 smithi192 ceph-mon[131661]: osdmap e21: 3 total, 3 up, 3 in 2023-10-26T06:40:52.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:52 smithi192 ceph-mon[131661]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:52.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:52 smithi192 ceph-mon[131661]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:52.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:52 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:52.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:52 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:52.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:52 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:52.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:52 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-26T06:40:52.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:52 smithi135 ceph-mon[131403]: osdmap e21: 3 total, 3 up, 3 in 2023-10-26T06:40:52.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:52 smithi135 ceph-mon[131403]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:52.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:52 smithi135 ceph-mon[131403]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:40:52.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:52 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:40:52.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:52 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:40:52.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:52 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:40:53.315 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143192]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-26T06:40:53.316 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:53.316 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143192]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:53.316 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143192]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:53.582 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143195]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-26T06:40:53.582 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:53.582 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143195]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:53.582 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:40:53 smithi135 sudo[143195]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:53 smithi089 ceph-mon[95031]: osdmap e22: 3 total, 3 up, 3 in 2023-10-26T06:40:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:53 smithi089 ceph-mon[95031]: mgrmap e20: x(active, since 110s), standbys: z, y 2023-10-26T06:40:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:53.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:53.839 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:53 smithi192 ceph-mon[131661]: osdmap e22: 3 total, 3 up, 3 in 2023-10-26T06:40:53.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:53 smithi192 ceph-mon[131661]: mgrmap e20: x(active, since 110s), standbys: z, y 2023-10-26T06:40:53.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:53.840 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:53.841 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143544]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-26T06:40:53.841 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143544]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:53.841 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143544]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:53.841 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143544]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:53.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:53 smithi135 ceph-mon[131403]: osdmap e22: 3 total, 3 up, 3 in 2023-10-26T06:40:53.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:53 smithi135 ceph-mon[131403]: mgrmap e20: x(active, since 110s), standbys: z, y 2023-10-26T06:40:53.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:53.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:54.145 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143547]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-26T06:40:54.145 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143547]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:54.146 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143547]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:54.146 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:40:53 smithi192 sudo[143547]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:54.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123484]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-26T06:40:54.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123484]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:54.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123484]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:54.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123484]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:55.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:55.233 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:54 smithi192 ceph-mon[131661]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:55.233 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143550]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-26T06:40:55.233 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143550]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:55.233 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143550]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:55.233 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143550]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:55.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123517]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-26T06:40:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123517]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 sudo[123517]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:54 smithi089 ceph-mon[95031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:40:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:54 smithi135 ceph-mon[131403]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143553]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-26T06:40:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143553]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 sudo[143553]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:55.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143198]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-26T06:40:55.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:55.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143198]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:55.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143198]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:56.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:56.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1995427248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]: dispatch 2023-10-26T06:40:56.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]: dispatch 2023-10-26T06:40:56.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:56.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]': finished 2023-10-26T06:40:56.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:56.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: osdmap e23: 4 total, 3 up, 4 in 2023-10-26T06:40:56.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:40:56.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.294 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:55 smithi089 ceph-mon[95031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:56.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1995427248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]: dispatch 2023-10-26T06:40:56.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]: dispatch 2023-10-26T06:40:56.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:56.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]': finished 2023-10-26T06:40:56.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:56.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: osdmap e23: 4 total, 3 up, 4 in 2023-10-26T06:40:56.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:40:56.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:55 smithi192 ceph-mon[131661]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143201]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-26T06:40:56.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143201]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-26T06:40:56.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143201]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-26T06:40:56.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 sudo[143201]: pam_unix(sudo:session): session closed for user root 2023-10-26T06:40:56.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:56.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1995427248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]: dispatch 2023-10-26T06:40:56.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]: dispatch 2023-10-26T06:40:56.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:56.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:56.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b4e4079-2075-4d49-9a5b-457e210aa444"}]': finished 2023-10-26T06:40:56.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-26T06:40:56.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: osdmap e23: 4 total, 3 up, 4 in 2023-10-26T06:40:56.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:40:56.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-26T06:40:56.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-26T06:40:56.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-26T06:40:56.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:55 smithi135 ceph-mon[131403]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-26T06:40:57.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:56 smithi089 ceph-mon[95031]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:57.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:56 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2583743667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:57.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:56 smithi192 ceph-mon[131661]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:57.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:56 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2583743667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:56 smithi135 ceph-mon[131403]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:56 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2583743667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:40:59.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:40:58 smithi089 ceph-mon[95031]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:59.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:40:58 smithi192 ceph-mon[131661]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:40:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:40:58 smithi135 ceph-mon[131403]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:01.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:00 smithi089 ceph-mon[95031]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 63 KiB/s wr, 5 op/s 2023-10-26T06:41:01.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:00 smithi192 ceph-mon[131661]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 63 KiB/s wr, 5 op/s 2023-10-26T06:41:01.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:00 smithi135 ceph-mon[131403]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 63 KiB/s wr, 5 op/s 2023-10-26T06:41:03.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:02 smithi089 ceph-mon[95031]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2023-10-26T06:41:03.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:02 smithi192 ceph-mon[131661]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2023-10-26T06:41:03.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:02 smithi135 ceph-mon[131403]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2023-10-26T06:41:04.250 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-26T06:41:04.250 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:04.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-26T06:41:04.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:04.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-26T06:41:04.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:04 smithi089 ceph-mon[95031]: Deploying daemon osd.3 on smithi089 2023-10-26T06:41:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:04 smithi089 ceph-mon[95031]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-10-26T06:41:05.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:04 smithi192 ceph-mon[131661]: Deploying daemon osd.3 on smithi089 2023-10-26T06:41:05.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:04 smithi192 ceph-mon[131661]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-10-26T06:41:05.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:04 smithi135 ceph-mon[131403]: Deploying daemon osd.3 on smithi089 2023-10-26T06:41:05.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:04 smithi135 ceph-mon[131403]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2023-10-26T06:41:07.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:07 smithi089 ceph-mon[95031]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1004 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-26T06:41:07.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:07 smithi192 ceph-mon[131661]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1004 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-26T06:41:07.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:07 smithi135 ceph-mon[131403]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1004 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-26T06:41:09.268 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:09 smithi089 ceph-mon[95031]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-26T06:41:09.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:09 smithi192 ceph-mon[131661]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-26T06:41:09.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:09 smithi135 ceph-mon[131403]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-26T06:41:10.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:10 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:10.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:10 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:10.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:10 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:10.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:10 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:10.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:10 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:10.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:10 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:10.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:10 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:10.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:10 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:10.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:10 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:11.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:11 smithi089 ceph-mon[95031]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-26T06:41:11.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:11 smithi192 ceph-mon[131661]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-26T06:41:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:11 smithi135 ceph-mon[131403]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-26T06:41:13.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.055+0000 7f09317fa700 1 -- 172.21.15.89:0/1909726613 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0934197d00 con 0x7f091c0a14c0 2023-10-26T06:41:13.057 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 3 on host 'smithi089' 2023-10-26T06:41:13.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f091c0a14c0 msgr2=0x7f091c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:13.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f091c0a14c0 0x7f091c0a3980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0934101730 tx=0x7f0924007400 comp rx=0 tx=0).stop 2023-10-26T06:41:13.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934100fb0 msgr2=0x7f093419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:13.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934100fb0 0x7f093419cc40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0928005910 tx=0x7f0928004730 comp rx=0 tx=0).stop 2023-10-26T06:41:13.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 shutdown_connections 2023-10-26T06:41:13.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f091c0a14c0 0x7f091c0a3980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:13.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0934106680 0x7f0934196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:13.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09341005f0 0x7f093419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:13.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 --2- 172.21.15.89:0/1909726613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0934100fb0 0x7f093419cc40 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0928005910 tx=0x7f0928004730 comp rx=0 tx=0).stop 2023-10-26T06:41:13.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.058+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 >> 172.21.15.89:0/1909726613 conn(0x7f09340fc120 msgr2=0x7f09340fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:41:13.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.059+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 shutdown_connections 2023-10-26T06:41:13.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:41:13.059+0000 7f0939490700 1 -- 172.21.15.89:0/1909726613 wait complete. 2023-10-26T06:41:13.238 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:13 smithi089 ceph-mon[95031]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:13.238 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.238 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.239 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:13.239 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:13.239 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:13 smithi192 ceph-mon[131661]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:13.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:13.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:13.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:13 smithi135 ceph-mon[131403]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:13.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:13.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:13.523 DEBUG:teuthology.orchestra.run.smithi089:osd.3> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.3.service 2023-10-26T06:41:13.528 INFO:tasks.cephadm:Deploying osd.4 on smithi135 with /dev/vg_nvme/lv_3... 2023-10-26T06:41:13.529 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-26T06:41:13.616 INFO:journalctl@ceph.osd.3.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:41:14.111 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:41:14.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:14.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:14.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:14.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:14 smithi089 ceph-mon[95031]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-26T06:41:14.299 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:14.300 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:14.300 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:14.300 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:14 smithi135 ceph-mon[131403]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-26T06:41:14.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:14.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:14.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:14.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:14 smithi192 ceph-mon[131661]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-26T06:41:15.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:15 smithi192 ceph-mon[131661]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:15.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:15 smithi192 ceph-mon[131661]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-26T06:41:15.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:15 smithi192 ceph-mon[131661]: osdmap e24: 4 total, 3 up, 4 in 2023-10-26T06:41:15.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:15 smithi192 ceph-mon[131661]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:41:15.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:15 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:15.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:15 smithi135 ceph-mon[131403]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:15.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:15 smithi135 ceph-mon[131403]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-26T06:41:15.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:15 smithi135 ceph-mon[131403]: osdmap e24: 4 total, 3 up, 4 in 2023-10-26T06:41:15.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:15 smithi135 ceph-mon[131403]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:41:15.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:15 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:15.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:15 smithi089 ceph-mon[95031]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:15.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:15 smithi089 ceph-mon[95031]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-26T06:41:15.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:15 smithi089 ceph-mon[95031]: osdmap e24: 4 total, 3 up, 4 in 2023-10-26T06:41:15.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:15 smithi089 ceph-mon[95031]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:41:15.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:15 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:15.540 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:41:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:41:15.075+0000 7f7a0a8a6700 -1 osd.3 0 waiting for initial osdmap 2023-10-26T06:41:15.541 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:41:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:41:15.081+0000 7f7a06e9c700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:41:15.570 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-26T06:41:15.570 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-26T06:41:15.570 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2023-10-26T06:41:15.570 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2023-10-26T06:41:15.571 INFO:teuthology.orchestra.run.smithi135.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121663 s, 862 MB/s 2023-10-26T06:41:15.571 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 2023-10-26T06:41:15.571 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2023-10-26T06:41:15.963 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi135:vg_nvme/lv_3 2023-10-26T06:41:16.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:16 smithi135 ceph-mon[131403]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:41:16.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:16 smithi135 ceph-mon[131403]: osdmap e25: 4 total, 3 up, 4 in 2023-10-26T06:41:16.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:16.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:16.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:16 smithi192 ceph-mon[131661]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:41:16.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:16 smithi192 ceph-mon[131661]: osdmap e25: 4 total, 3 up, 4 in 2023-10-26T06:41:16.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:16.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:16.522 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:41:16.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:16 smithi089 ceph-mon[95031]: from='osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:41:16.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:16 smithi089 ceph-mon[95031]: osdmap e25: 4 total, 3 up, 4 in 2023-10-26T06:41:16.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:16.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:17.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:41:17.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:41:17.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:17.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] boot 2023-10-26T06:41:17.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: osdmap e26: 4 total, 4 up, 4 in 2023-10-26T06:41:17.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:17.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:17.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:17.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:17.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:17.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:17 smithi089 ceph-mon[95031]: osdmap e27: 4 total, 4 up, 4 in 2023-10-26T06:41:17.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:41:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:41:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] boot 2023-10-26T06:41:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: osdmap e26: 4 total, 4 up, 4 in 2023-10-26T06:41:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:17.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:17.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:17.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:17.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:17.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:17 smithi192 ceph-mon[131661]: osdmap e27: 4 total, 4 up, 4 in 2023-10-26T06:41:17.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:41:17.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:41:17.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-26T06:41:17.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: osd.3 [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] boot 2023-10-26T06:41:17.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: osdmap e26: 4 total, 4 up, 4 in 2023-10-26T06:41:17.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-26T06:41:17.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:17.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:17.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:17.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:17.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:17.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:17 smithi135 ceph-mon[131403]: osdmap e27: 4 total, 4 up, 4 in 2023-10-26T06:41:17.738 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 -- 172.21.15.135:0/2868849957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05680ff6e0 msgr2=0x7f05680ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:17.738 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 --2- 172.21.15.135:0/2868849957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05680ff6e0 0x7f05680ffb60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0558009990 tx=0x7f0558009ca0 comp rx=0 tx=0).stop 2023-10-26T06:41:17.738 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 -- 172.21.15.135:0/2868849957 shutdown_connections 2023-10-26T06:41:17.738 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 --2- 172.21.15.135:0/2868849957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05681000a0 0x7f056810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:17.739 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 --2- 172.21.15.135:0/2868849957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05681045b0 0x7f0568104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:17.739 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 --2- 172.21.15.135:0/2868849957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05680ff6e0 0x7f05680ffb60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:17.739 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 -- 172.21.15.135:0/2868849957 >> 172.21.15.135:0/2868849957 conn(0x7f05680766c0 msgr2=0x7f0568076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:41:17.739 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 -- 172.21.15.135:0/2868849957 shutdown_connections 2023-10-26T06:41:17.739 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 -- 172.21.15.135:0/2868849957 wait complete. 2023-10-26T06:41:17.740 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 Processor -- start 2023-10-26T06:41:17.740 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 -- start start 2023-10-26T06:41:17.740 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.734+0000 7f056fb6e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05680ff6e0 0x7f056819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:17.740 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056fb6e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05681000a0 0x7f056819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:17.740 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056fb6e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05681045b0 0x7f0568196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:17.741 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056fb6e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681971e0 con 0x7f05681000a0 2023-10-26T06:41:17.741 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056fb6e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0568197350 con 0x7f05681045b0 2023-10-26T06:41:17.741 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056fb6e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0568197510 con 0x7f05680ff6e0 2023-10-26T06:41:17.741 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05681045b0 0x7f0568196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:41:17.741 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05681045b0 0x7f0568196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58600/0 (socket says 172.21.15.135:58600) 2023-10-26T06:41:17.742 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 -- 172.21.15.135:0/1142759801 learned_addr learned my addr 172.21.15.135:0/1142759801 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-10-26T06:41:17.742 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 -- 172.21.15.135:0/1142759801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05680ff6e0 msgr2=0x7f056819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:17.742 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056d90a700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05680ff6e0 0x7f056819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:41:17.742 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05680ff6e0 0x7f056819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:17.742 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 -- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05681000a0 msgr2=0x7f056819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:17.743 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05681000a0 0x7f056819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:17.743 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 -- 172.21.15.135:0/1142759801 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05580095f0 con 0x7f05681045b0 2023-10-26T06:41:17.743 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056d90a700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05680ff6e0 0x7f056819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:41:17.743 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056e10b700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05681045b0 0x7f0568196b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f056000b970 tx=0x7f056000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:41:17.743 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f055effd700 1 -- 172.21.15.135:0/1142759801 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000c9b0 con 0x7f05681045b0 2023-10-26T06:41:17.744 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f055effd700 1 -- 172.21.15.135:0/1142759801 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0560014440 con 0x7f05681045b0 2023-10-26T06:41:17.744 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05681977f0 con 0x7f05681045b0 2023-10-26T06:41:17.744 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f055effd700 1 -- 172.21.15.135:0/1142759801 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600075d0 con 0x7f05681045b0 2023-10-26T06:41:17.744 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.735+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05681a39e0 con 0x7f05681045b0 2023-10-26T06:41:17.744 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.736+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0568108c30 con 0x7f05681045b0 2023-10-26T06:41:17.745 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.737+0000 7f055effd700 1 -- 172.21.15.135:0/1142759801 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f05600145b0 con 0x7f05681045b0 2023-10-26T06:41:17.746 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.738+0000 7f055effd700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05540a14d0 0x7f05540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:17.746 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.738+0000 7f055effd700 1 -- 172.21.15.135:0/1142759801 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3913+0+0 (secure 0 0 0) 0x7f0560006e40 con 0x7f05681045b0 2023-10-26T06:41:17.746 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.738+0000 7f056d90a700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05540a14d0 0x7f05540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:41:17.746 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.739+0000 7f056d90a700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05540a14d0 0x7f05540a3990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0564005fd0 tx=0x7f0564005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:41:17.746 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.741+0000 7f055effd700 1 -- 172.21.15.135:0/1142759801 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0560095fb0 con 0x7f05681045b0 2023-10-26T06:41:17.939 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:17.938+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0568197ff0 con 0x7f05540a14d0 2023-10-26T06:41:18.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:18 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi089 2023-10-26T06:41:18.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:18 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi089 to 8635M 2023-10-26T06:41:18.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:41:18.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:41:18.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:18.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:18 smithi192 ceph-mon[131661]: osdmap e28: 4 total, 4 up, 4 in 2023-10-26T06:41:18.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:18 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi089 2023-10-26T06:41:18.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:18 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi089 to 8635M 2023-10-26T06:41:18.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:41:18.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:41:18.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:18.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:18 smithi135 ceph-mon[131403]: osdmap e28: 4 total, 4 up, 4 in 2023-10-26T06:41:18.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:18 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi089 2023-10-26T06:41:18.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:18 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi089 to 8635M 2023-10-26T06:41:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:41:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:41:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:18 smithi089 ceph-mon[95031]: osdmap e28: 4 total, 4 up, 4 in 2023-10-26T06:41:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:19 smithi192 ceph-mon[131661]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:19.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:19 smithi192 ceph-mon[131661]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:41:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:19 smithi135 ceph-mon[131403]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:19 smithi135 ceph-mon[131403]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:41:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:19 smithi089 ceph-mon[95031]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:19 smithi089 ceph-mon[95031]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:41:20.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:20 smithi192 ceph-mon[131661]: osdmap e29: 4 total, 4 up, 4 in 2023-10-26T06:41:20.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:20 smithi135 ceph-mon[131403]: osdmap e29: 4 total, 4 up, 4 in 2023-10-26T06:41:20.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:20 smithi089 ceph-mon[95031]: osdmap e29: 4 total, 4 up, 4 in 2023-10-26T06:41:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: pgmap v86: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-26T06:41:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: osdmap e30: 4 total, 4 up, 4 in 2023-10-26T06:41:21.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/1284029680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]: dispatch 2023-10-26T06:41:21.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]: dispatch 2023-10-26T06:41:21.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]': finished 2023-10-26T06:41:21.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: osdmap e31: 5 total, 4 up, 5 in 2023-10-26T06:41:21.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:21.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:21 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/3725947790' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:41:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: pgmap v86: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-26T06:41:21.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: osdmap e30: 4 total, 4 up, 4 in 2023-10-26T06:41:21.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/1284029680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]: dispatch 2023-10-26T06:41:21.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]: dispatch 2023-10-26T06:41:21.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]': finished 2023-10-26T06:41:21.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: osdmap e31: 5 total, 4 up, 5 in 2023-10-26T06:41:21.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:21.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:21 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/3725947790' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:41:21.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: pgmap v86: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-26T06:41:21.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: osdmap e30: 4 total, 4 up, 4 in 2023-10-26T06:41:21.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/1284029680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]: dispatch 2023-10-26T06:41:21.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]: dispatch 2023-10-26T06:41:21.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "391e09b6-9475-4b8e-914f-34ed06f36e07"}]': finished 2023-10-26T06:41:21.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: osdmap e31: 5 total, 4 up, 5 in 2023-10-26T06:41:21.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:21.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:21 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/3725947790' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:41:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:23 smithi192 ceph-mon[131661]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-26T06:41:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:23 smithi135 ceph-mon[131403]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-26T06:41:23.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:23 smithi089 ceph-mon[95031]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-26T06:41:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:25 smithi192 ceph-mon[131661]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-26T06:41:25.420 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:25 smithi135 ceph-mon[131403]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-26T06:41:25.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:25 smithi089 ceph-mon[95031]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-26T06:41:27.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:27 smithi192 ceph-mon[131661]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:27.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:27 smithi135 ceph-mon[131403]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:27.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:27 smithi089 ceph-mon[95031]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:28.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-26T06:41:28.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:28.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:28 smithi192 ceph-mon[131661]: Deploying daemon osd.4 on smithi135 2023-10-26T06:41:28.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-26T06:41:28.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:28.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:28 smithi135 ceph-mon[131403]: Deploying daemon osd.4 on smithi135 2023-10-26T06:41:28.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-26T06:41:28.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:28.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:28 smithi089 ceph-mon[95031]: Deploying daemon osd.4 on smithi135 2023-10-26T06:41:29.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:29 smithi135 ceph-mon[131403]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:29 smithi192 ceph-mon[131661]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:29 smithi089 ceph-mon[95031]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:31.374 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:31 smithi135 ceph-mon[131403]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:31 smithi192 ceph-mon[131661]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:31.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:31 smithi089 ceph-mon[95031]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:32.628 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:32.629 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:32.629 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:32.629 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:32 smithi135 ceph-mon[131403]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:32.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:32.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:32.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:32.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:32 smithi089 ceph-mon[95031]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:32.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:32.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:32.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:32 smithi192 ceph-mon[131661]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:33.960 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 4 on host 'smithi135' 2023-10-26T06:41:33.961 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.959+0000 7f055effd700 1 -- 172.21.15.135:0/1142759801 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0568197ff0 con 0x7f05540a14d0 2023-10-26T06:41:33.962 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05540a14d0 msgr2=0x7f05540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:33.963 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05540a14d0 0x7f05540a3990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0564005fd0 tx=0x7f0564005ee0 comp rx=0 tx=0).stop 2023-10-26T06:41:33.963 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05681045b0 msgr2=0x7f0568196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:33.963 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05681045b0 0x7f0568196b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f056000b970 tx=0x7f056000bd30 comp rx=0 tx=0).stop 2023-10-26T06:41:33.963 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 shutdown_connections 2023-10-26T06:41:33.963 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05540a14d0 0x7f05540a3990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:33.964 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05681045b0 0x7f0568196b80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:33.964 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05680ff6e0 0x7f056819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:33.964 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 --2- 172.21.15.135:0/1142759801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05681000a0 0x7f056819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:33.964 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 >> 172.21.15.135:0/1142759801 conn(0x7f05680766c0 msgr2=0x7f0568068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:41:33.964 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 shutdown_connections 2023-10-26T06:41:33.964 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:41:33.962+0000 7f056fb6e700 1 -- 172.21.15.135:0/1142759801 wait complete. 2023-10-26T06:41:34.408 DEBUG:teuthology.orchestra.run.smithi135:osd.4> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.4.service 2023-10-26T06:41:34.410 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2023-10-26T06:41:34.411 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-26T06:41:34.439 INFO:journalctl@ceph.osd.4.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:41:34.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:34 smithi192 ceph-mon[131661]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:34.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:34 smithi135 ceph-mon[131403]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:34.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:34.979 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:41:35.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:35.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:35.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:34 smithi089 ceph-mon[95031]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:35.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:35.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:35.717 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 06:41:35 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-4[146265]: 2023-10-26T06:41:35.355+0000 7fe20ce12640 -1 osd.4 0 log_to_monitors true 2023-10-26T06:41:36.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:35 smithi135 ceph-mon[131403]: from='osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-26T06:41:36.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:35 smithi135 ceph-mon[131403]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-26T06:41:36.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:35 smithi089 ceph-mon[95031]: from='osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-26T06:41:36.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:35 smithi089 ceph-mon[95031]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-26T06:41:36.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:35 smithi192 ceph-mon[131661]: from='osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-26T06:41:36.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:35 smithi192 ceph-mon[131661]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-26T06:41:36.509 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-26T06:41:36.509 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-26T06:41:36.510 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-10-26T06:41:36.510 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-10-26T06:41:36.510 INFO:teuthology.orchestra.run.smithi192.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124605 s, 842 MB/s 2023-10-26T06:41:36.510 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-10-26T06:41:36.928 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2023-10-26T06:41:37.217 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-4[146265]: 2023-10-26T06:41:37.000+0000 7fe201c88700 -1 osd.4 0 waiting for initial osdmap 2023-10-26T06:41:37.218 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-4[146265]: 2023-10-26T06:41:37.008+0000 7fe1fba79700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:41:37.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-mon[131403]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:37.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-mon[131403]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-26T06:41:37.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-mon[131403]: from='osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:41:37.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-mon[131403]: osdmap e32: 5 total, 4 up, 5 in 2023-10-26T06:41:37.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:37.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:37 smithi135 ceph-mon[131403]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:41:37.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:37 smithi089 ceph-mon[95031]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:37.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:37 smithi089 ceph-mon[95031]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-26T06:41:37.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:37 smithi089 ceph-mon[95031]: from='osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:41:37.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:37 smithi089 ceph-mon[95031]: osdmap e32: 5 total, 4 up, 5 in 2023-10-26T06:41:37.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:37.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:37 smithi089 ceph-mon[95031]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:41:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:37 smithi192 ceph-mon[131661]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:37 smithi192 ceph-mon[131661]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-26T06:41:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:37 smithi192 ceph-mon[131661]: from='osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:41:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:37 smithi192 ceph-mon[131661]: osdmap e32: 5 total, 4 up, 5 in 2023-10-26T06:41:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:37 smithi192 ceph-mon[131661]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:41:37.487 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:41:38.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:41:38.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: osdmap e33: 5 total, 4 up, 5 in 2023-10-26T06:41:38.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:38.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:38.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:38.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:38.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:38.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:38.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:38.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:38.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:38.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:41:38.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: osdmap e33: 5 total, 4 up, 5 in 2023-10-26T06:41:38.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:38.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:38.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:38.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:38.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:38.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:38.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:38.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:38.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:38.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:41:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: osdmap e33: 5 total, 4 up, 5 in 2023-10-26T06:41:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:38.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:38.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:38.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:38.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:38.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:38.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:38.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:38.658 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 -- 172.21.15.192:0/3884640872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 msgr2=0x7fb5f0104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:38.658 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/3884640872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f0104250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009990 tx=0x7fb5ec009ca0 comp rx=0 tx=0).stop 2023-10-26T06:41:38.659 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 -- 172.21.15.192:0/3884640872 shutdown_connections 2023-10-26T06:41:38.659 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/3884640872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f00ffb50 0x7fb5f010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:38.659 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/3884640872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f0104250 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/3884640872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5f00ff190 0x7fb5f00ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 -- 172.21.15.192:0/3884640872 >> 172.21.15.192:0/3884640872 conn(0x7fb5f0074a10 msgr2=0x7fb5f0074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:41:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 -- 172.21.15.192:0/3884640872 shutdown_connections 2023-10-26T06:41:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 -- 172.21.15.192:0/3884640872 wait complete. 2023-10-26T06:41:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 Processor -- start 2023-10-26T06:41:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.656+0000 7fb5f7a77700 1 -- start start 2023-10-26T06:41:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5f00ff190 0x7fb5f0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f00ffb50 0x7fb5f0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:38.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:38.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f019de00 con 0x7fb5f00ffb50 2023-10-26T06:41:38.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f019df70 con 0x7fb5f00ff190 2023-10-26T06:41:38.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f019e130 con 0x7fb5f0103e70 2023-10-26T06:41:38.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:41:38.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f019d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54052/0 (socket says 172.21.15.192:54052) 2023-10-26T06:41:38.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 -- 172.21.15.192:0/881150290 learned_addr learned my addr 172.21.15.192:0/881150290 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-10-26T06:41:38.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f5813700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5f00ff190 0x7fb5f0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:41:38.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 -- 172.21.15.192:0/881150290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5f00ff190 msgr2=0x7fb5f0198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:38.665 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5f00ff190 0x7fb5f0198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:38.666 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 -- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f00ffb50 msgr2=0x7fb5f0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:38.666 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f00ffb50 0x7fb5f0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:38.666 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 -- 172.21.15.192:0/881150290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0095f0 con 0x7fb5f0103e70 2023-10-26T06:41:38.666 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f6014700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f019d7f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800b970 tx=0x7fb5e800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:41:38.666 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5e6ffd700 1 -- 172.21.15.192:0/881150290 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800c9b0 con 0x7fb5f0103e70 2023-10-26T06:41:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5e6ffd700 1 -- 172.21.15.192:0/881150290 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5e8014440 con 0x7fb5f0103e70 2023-10-26T06:41:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f019e410 con 0x7fb5f0103e70 2023-10-26T06:41:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5e6ffd700 1 -- 172.21.15.192:0/881150290 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80075d0 con 0x7fb5f0103e70 2023-10-26T06:41:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.657+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f01a5c70 con 0x7fb5f0103e70 2023-10-26T06:41:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.658+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f01005b0 con 0x7fb5f0103e70 2023-10-26T06:41:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.659+0000 7fb5e6ffd700 1 -- 172.21.15.192:0/881150290 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb5e80145b0 con 0x7fb5f0103e70 2023-10-26T06:41:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.660+0000 7fb5e6ffd700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5dc0a14d0 0x7fb5dc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:41:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.660+0000 7fb5e6ffd700 1 -- 172.21.15.192:0/881150290 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4316+0+0 (secure 0 0 0) 0x7fb5e8006e40 con 0x7fb5f0103e70 2023-10-26T06:41:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.660+0000 7fb5f5813700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5dc0a14d0 0x7fb5dc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:41:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.661+0000 7fb5f5813700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5dc0a14d0 0x7fb5dc0a3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec00b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:41:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.664+0000 7fb5e6ffd700 1 -- 172.21.15.192:0/881150290 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb5e8095fb0 con 0x7fb5f0103e70 2023-10-26T06:41:38.865 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:38.863+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5f010edf0 con 0x7fb5dc0a14d0 2023-10-26T06:41:39.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:41:39.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:41:39.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi135 2023-10-26T06:41:39.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi135 to 8584M 2023-10-26T06:41:39.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:39.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:39.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] boot 2023-10-26T06:41:39.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: osdmap e34: 5 total, 5 up, 5 in 2023-10-26T06:41:39.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:39.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:41:39.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:41:39.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:41:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:41:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi135 2023-10-26T06:41:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi135 to 8584M 2023-10-26T06:41:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] boot 2023-10-26T06:41:39.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: osdmap e34: 5 total, 5 up, 5 in 2023-10-26T06:41:39.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:39.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:41:39.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:41:39.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:41:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:41:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi135 2023-10-26T06:41:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi135 to 8584M 2023-10-26T06:41:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-26T06:41:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: osd.4 [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] boot 2023-10-26T06:41:39.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: osdmap e34: 5 total, 5 up, 5 in 2023-10-26T06:41:39.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-26T06:41:39.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:41:39.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:41:39.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:40.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:40 smithi089 ceph-mon[95031]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:41:40.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:40 smithi089 ceph-mon[95031]: osdmap e35: 5 total, 5 up, 5 in 2023-10-26T06:41:40.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:40 smithi192 ceph-mon[131661]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:41:40.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:40 smithi192 ceph-mon[131661]: osdmap e35: 5 total, 5 up, 5 in 2023-10-26T06:41:40.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:40 smithi135 ceph-mon[131403]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:41:40.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:40 smithi135 ceph-mon[131403]: osdmap e35: 5 total, 5 up, 5 in 2023-10-26T06:41:41.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:41 smithi089 ceph-mon[95031]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:41.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:41 smithi192 ceph-mon[131661]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:41 smithi135 ceph-mon[131403]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:42.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:42 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/2988267050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]: dispatch 2023-10-26T06:41:42.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:42 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]: dispatch 2023-10-26T06:41:42.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:42 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]': finished 2023-10-26T06:41:42.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:42 smithi192 ceph-mon[131661]: osdmap e36: 6 total, 5 up, 6 in 2023-10-26T06:41:42.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:42.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:42 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/2640724910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:41:42.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:42 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/2988267050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]: dispatch 2023-10-26T06:41:42.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:42 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]: dispatch 2023-10-26T06:41:42.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:42 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]': finished 2023-10-26T06:41:42.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:42 smithi135 ceph-mon[131403]: osdmap e36: 6 total, 5 up, 6 in 2023-10-26T06:41:42.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:42.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:42 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/2640724910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:41:42.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:42 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/2988267050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]: dispatch 2023-10-26T06:41:42.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:42 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]: dispatch 2023-10-26T06:41:42.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:42 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "529984c4-0e50-4d78-b4ba-572a2f7df23c"}]': finished 2023-10-26T06:41:42.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:42 smithi089 ceph-mon[95031]: osdmap e36: 6 total, 5 up, 6 in 2023-10-26T06:41:42.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:42.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:42 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/2640724910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:41:43.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:43 smithi089 ceph-mon[95031]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:43.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:43 smithi192 ceph-mon[131661]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:43.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:43 smithi135 ceph-mon[131403]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:45.521 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:45 smithi192 ceph-mon[131661]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:45.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:45 smithi089 ceph-mon[95031]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:45.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:45 smithi135 ceph-mon[131403]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:47.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:47 smithi192 ceph-mon[131661]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:47 smithi089 ceph-mon[95031]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:47.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:47 smithi135 ceph-mon[131403]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:48.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-26T06:41:48.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:48.555 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-26T06:41:48.555 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:48.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-26T06:41:48.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:49 smithi089 ceph-mon[95031]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:49 smithi089 ceph-mon[95031]: Deploying daemon osd.5 on smithi192 2023-10-26T06:41:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:49 smithi192 ceph-mon[131661]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:49.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:49 smithi192 ceph-mon[131661]: Deploying daemon osd.5 on smithi192 2023-10-26T06:41:49.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:49 smithi135 ceph-mon[131403]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:49.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:49 smithi135 ceph-mon[131403]: Deploying daemon osd.5 on smithi192 2023-10-26T06:41:51.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:50 smithi192 ceph-mon[131661]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:51.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:50 smithi089 ceph-mon[95031]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:51.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:50 smithi135 ceph-mon[131403]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:53.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:53 smithi192 ceph-mon[131661]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:53.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:53.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:53.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:53 smithi089 ceph-mon[95031]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:53.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:53.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:53.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:53 smithi135 ceph-mon[131403]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:53.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:53.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:53.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:55.028 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.026+0000 7fb5e6ffd700 1 -- 172.21.15.192:0/881150290 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb5f010edf0 con 0x7fb5dc0a14d0 2023-10-26T06:41:55.029 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2023-10-26T06:41:55.030 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5dc0a14d0 msgr2=0x7fb5dc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:55.031 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5dc0a14d0 0x7fb5dc0a3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec006010 tx=0x7fb5ec00b260 comp rx=0 tx=0).stop 2023-10-26T06:41:55.031 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 msgr2=0x7fb5f019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:41:55.031 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f019d7f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800b970 tx=0x7fb5e800bd30 comp rx=0 tx=0).stop 2023-10-26T06:41:55.031 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 shutdown_connections 2023-10-26T06:41:55.031 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5dc0a14d0 0x7fb5dc0a3990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:55.032 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5f0103e70 0x7fb5f019d7f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:55.032 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5f00ff190 0x7fb5f0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:55.032 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 --2- 172.21.15.192:0/881150290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5f00ffb50 0x7fb5f0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:41:55.032 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 >> 172.21.15.192:0/881150290 conn(0x7fb5f0074a10 msgr2=0x7fb5f00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:41:55.033 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 shutdown_connections 2023-10-26T06:41:55.033 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:41:55.029+0000 7fb5f7a77700 1 -- 172.21.15.192:0/881150290 wait complete. 2023-10-26T06:41:55.365 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:55.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:55.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:55.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:55.368 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.5.service 2023-10-26T06:41:55.371 INFO:tasks.cephadm:Deploying osd.6 on smithi089 with /dev/vg_nvme/lv_2... 2023-10-26T06:41:55.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-26T06:41:55.400 INFO:journalctl@ceph.osd.5.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:41:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:55.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:55.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:55.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:41:56.115 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:41:56.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:56 smithi089 ceph-mon[95031]: from='osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-26T06:41:56.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:56 smithi089 ceph-mon[95031]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-26T06:41:56.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:56 smithi192 ceph-mon[131661]: from='osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-26T06:41:56.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:56 smithi192 ceph-mon[131661]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-26T06:41:56.646 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:41:56 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:41:56.231+0000 7f4af13d6640 -1 osd.5 0 log_to_monitors true 2023-10-26T06:41:56.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:56 smithi135 ceph-mon[131403]: from='osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-26T06:41:56.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:56 smithi135 ceph-mon[131403]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-26T06:41:57.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:57 smithi089 ceph-mon[95031]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:57.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:57 smithi089 ceph-mon[95031]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-26T06:41:57.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:57 smithi089 ceph-mon[95031]: osdmap e37: 6 total, 5 up, 6 in 2023-10-26T06:41:57.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:57 smithi089 ceph-mon[95031]: from='osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:41:57.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:57.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:57 smithi089 ceph-mon[95031]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:41:57.645 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:41:57.256+0000 7f4ae624c700 -1 osd.5 0 waiting for initial osdmap 2023-10-26T06:41:57.645 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:41:57.263+0000 7f4ae2842700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:41:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-mon[131661]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-mon[131661]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-26T06:41:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-mon[131661]: osdmap e37: 6 total, 5 up, 6 in 2023-10-26T06:41:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-mon[131661]: from='osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:41:57.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:57.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:57 smithi192 ceph-mon[131661]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:41:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:57 smithi135 ceph-mon[131403]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:57 smithi135 ceph-mon[131403]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-26T06:41:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:57 smithi135 ceph-mon[131403]: osdmap e37: 6 total, 5 up, 6 in 2023-10-26T06:41:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:57 smithi135 ceph-mon[131403]: from='osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:41:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:57 smithi135 ceph-mon[131403]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:41:58.236 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-26T06:41:58.236 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-26T06:41:58.237 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-10-26T06:41:58.237 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-10-26T06:41:58.237 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0570299 s, 184 MB/s 2023-10-26T06:41:58.237 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-10-26T06:41:58.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:41:58.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: osdmap e38: 6 total, 5 up, 6 in 2023-10-26T06:41:58.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:58.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:58.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:58.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:58.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:58.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:58.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:41:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: osdmap e38: 6 total, 5 up, 6 in 2023-10-26T06:41:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:58.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:58.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:58.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:58.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2023-10-26T06:41:58.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:41:58.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: osdmap e38: 6 total, 5 up, 6 in 2023-10-26T06:41:58.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:58.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:58.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:58.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:41:58.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:58.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:41:58.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:41:58.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:41:59.438 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:41:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:41:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:41:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi192 2023-10-26T06:41:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi192 to 8584M 2023-10-26T06:41:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] boot 2023-10-26T06:41:59.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: osdmap e39: 6 total, 6 up, 6 in 2023-10-26T06:41:59.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:41:59 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:41:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:41:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi192 2023-10-26T06:41:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi192 to 8584M 2023-10-26T06:41:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:59.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] boot 2023-10-26T06:41:59.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: osdmap e39: 6 total, 6 up, 6 in 2023-10-26T06:41:59.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:41:59 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:41:59.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:41:59.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-26T06:41:59.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi192 2023-10-26T06:41:59.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi192 to 8584M 2023-10-26T06:41:59.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:41:59.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: osd.5 [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] boot 2023-10-26T06:41:59.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: osdmap e39: 6 total, 6 up, 6 in 2023-10-26T06:41:59.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:41:59 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-26T06:42:00.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:00 smithi089 ceph-mon[95031]: osdmap e40: 6 total, 6 up, 6 in 2023-10-26T06:42:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:00 smithi089 ceph-mon[95031]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:00 smithi089 ceph-mon[95031]: osdmap e41: 6 total, 6 up, 6 in 2023-10-26T06:42:00.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:00 smithi192 ceph-mon[131661]: osdmap e40: 6 total, 6 up, 6 in 2023-10-26T06:42:00.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:00 smithi192 ceph-mon[131661]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:00.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:00 smithi192 ceph-mon[131661]: osdmap e41: 6 total, 6 up, 6 in 2023-10-26T06:42:00.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:00 smithi135 ceph-mon[131403]: osdmap e40: 6 total, 6 up, 6 in 2023-10-26T06:42:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:00 smithi135 ceph-mon[131403]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:00 smithi135 ceph-mon[131403]: osdmap e41: 6 total, 6 up, 6 in 2023-10-26T06:42:00.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 -- 172.21.15.89:0/1186499457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501057c0 msgr2=0x7f7850105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 --2- 172.21.15.89:0/1186499457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501057c0 0x7f7850105ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7840009990 tx=0x7f7840009ca0 comp rx=0 tx=0).stop 2023-10-26T06:42:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 -- 172.21.15.89:0/1186499457 shutdown_connections 2023-10-26T06:42:00.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 --2- 172.21.15.89:0/1186499457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7850101260 0x7f785010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 --2- 172.21.15.89:0/1186499457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501057c0 0x7f7850105ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 --2- 172.21.15.89:0/1186499457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78501008a0 0x7f7850100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:00.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 -- 172.21.15.89:0/1186499457 >> 172.21.15.89:0/1186499457 conn(0x7f78500779e0 msgr2=0x7f7850077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:42:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 -- 172.21.15.89:0/1186499457 shutdown_connections 2023-10-26T06:42:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.974+0000 7f7855164700 1 -- 172.21.15.89:0/1186499457 wait complete. 2023-10-26T06:42:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.975+0000 7f7855164700 1 Processor -- start 2023-10-26T06:42:00.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.975+0000 7f7855164700 1 -- start start 2023-10-26T06:42:00.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f7855164700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501008a0 0x7f7850198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:00.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f7855164700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7850101260 0x7f78501993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:00.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f7855164700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78501057c0 0x7f785019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:00.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f7855164700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019dd40 con 0x7f7850101260 2023-10-26T06:42:00.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f7855164700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019deb0 con 0x7f78501057c0 2023-10-26T06:42:00.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f7855164700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019e070 con 0x7f78501008a0 2023-10-26T06:42:00.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501008a0 0x7f7850198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:00.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501008a0 0x7f7850198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:55228/0 (socket says 172.21.15.89:55228) 2023-10-26T06:42:00.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 -- 172.21.15.89:0/2532008700 learned_addr learned my addr 172.21.15.89:0/2532008700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:42:00.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784e59c700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7850101260 0x7f78501993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:00.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784f59e700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78501057c0 0x7f785019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:00.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 -- 172.21.15.89:0/2532008700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78501057c0 msgr2=0x7f785019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:00.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78501057c0 0x7f785019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:00.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 -- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7850101260 msgr2=0x7f78501993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:00.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7850101260 0x7f78501993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:00.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.976+0000 7f784ed9d700 1 -- 172.21.15.89:0/2532008700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848009c40 con 0x7f78501008a0 2023-10-26T06:42:00.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.977+0000 7f784ed9d700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501008a0 0x7f7850198e90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7840000c00 tx=0x7f7840004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:42:00.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.977+0000 7f7837fff700 1 -- 172.21.15.89:0/2532008700 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784001d070 con 0x7f78501008a0 2023-10-26T06:42:00.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.977+0000 7f7837fff700 1 -- 172.21.15.89:0/2532008700 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7840021410 con 0x7f78501008a0 2023-10-26T06:42:00.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.977+0000 7f7837fff700 1 -- 172.21.15.89:0/2532008700 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840017650 con 0x7f78501008a0 2023-10-26T06:42:00.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.977+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78400095f0 con 0x7f78501008a0 2023-10-26T06:42:00.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.977+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785019e520 con 0x7f78501008a0 2023-10-26T06:42:00.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.978+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7850101f30 con 0x7f78501008a0 2023-10-26T06:42:00.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.980+0000 7f7837fff700 1 -- 172.21.15.89:0/2532008700 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7840021580 con 0x7f78501008a0 2023-10-26T06:42:00.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.982+0000 7f7837fff700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f78380a14d0 0x7f78380a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:00.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.982+0000 7f7837fff700 1 -- 172.21.15.89:0/2532008700 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4777+0+0 (secure 0 0 0) 0x7f78400cd950 con 0x7f78501008a0 2023-10-26T06:42:00.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.982+0000 7f784e59c700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f78380a14d0 0x7f78380a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:00.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.982+0000 7f784e59c700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f78380a14d0 0x7f78380a3990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f785019a3a0 tx=0x7f7844009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:42:00.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:00.984+0000 7f7837fff700 1 -- 172.21.15.89:0/2532008700 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7840097d70 con 0x7f78501008a0 2023-10-26T06:42:01.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:01.194+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7850110760 con 0x7f78380a14d0 2023-10-26T06:42:01.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:01 smithi089 ceph-mon[95031]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:01.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:01.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:01.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:01 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:01 smithi192 ceph-mon[131661]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:01.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:01.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:01 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:01 smithi135 ceph-mon[131403]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:01.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:01 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:02.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:02 smithi089 ceph-mon[95031]: osdmap e42: 6 total, 6 up, 6 in 2023-10-26T06:42:02.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:02 smithi089 ceph-mon[95031]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:02.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:02 smithi192 ceph-mon[131661]: osdmap e42: 6 total, 6 up, 6 in 2023-10-26T06:42:02.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:02 smithi192 ceph-mon[131661]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:02.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:02 smithi135 ceph-mon[131403]: osdmap e42: 6 total, 6 up, 6 in 2023-10-26T06:42:02.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:02 smithi135 ceph-mon[131403]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:03.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:03 smithi089 ceph-mon[95031]: osdmap e43: 6 total, 6 up, 6 in 2023-10-26T06:42:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:03 smithi192 ceph-mon[131661]: osdmap e43: 6 total, 6 up, 6 in 2023-10-26T06:42:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:03 smithi135 ceph-mon[131403]: osdmap e43: 6 total, 6 up, 6 in 2023-10-26T06:42:04.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:04 smithi192 ceph-mon[131661]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:04.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:04 smithi135 ceph-mon[131403]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:04.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:04 smithi089 ceph-mon[95031]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3441698583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984"}]: dispatch 2023-10-26T06:42:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3441698583' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984"}]': finished 2023-10-26T06:42:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:05 smithi192 ceph-mon[131661]: osdmap e44: 7 total, 6 up, 7 in 2023-10-26T06:42:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:05.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2677091237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3441698583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984"}]: dispatch 2023-10-26T06:42:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3441698583' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984"}]': finished 2023-10-26T06:42:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:05 smithi135 ceph-mon[131403]: osdmap e44: 7 total, 6 up, 7 in 2023-10-26T06:42:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2677091237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:05.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3441698583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984"}]: dispatch 2023-10-26T06:42:05.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3441698583' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984"}]': finished 2023-10-26T06:42:05.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:05 smithi089 ceph-mon[95031]: osdmap e44: 7 total, 6 up, 7 in 2023-10-26T06:42:05.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:05.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2677091237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:06.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:06 smithi192 ceph-mon[131661]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-10-26T06:42:06.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:06 smithi135 ceph-mon[131403]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-10-26T06:42:06.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:06 smithi089 ceph-mon[95031]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-10-26T06:42:08.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:08 smithi089 ceph-mon[95031]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-26T06:42:09.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:08 smithi192 ceph-mon[131661]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-26T06:42:09.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:08 smithi135 ceph-mon[131403]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-26T06:42:11.281 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:10 smithi089 ceph-mon[95031]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-26T06:42:11.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:10 smithi192 ceph-mon[131661]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-26T06:42:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:10 smithi135 ceph-mon[131403]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-26T06:42:13.263 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:13 smithi089 ceph-mon[95031]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-26T06:42:13.264 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-26T06:42:13.264 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:13.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:13 smithi192 ceph-mon[131661]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-26T06:42:13.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-26T06:42:13.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:13 smithi135 ceph-mon[131403]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-26T06:42:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-26T06:42:13.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:14.268 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:14 smithi089 ceph-mon[95031]: Deploying daemon osd.6 on smithi089 2023-10-26T06:42:14.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:14 smithi192 ceph-mon[131661]: Deploying daemon osd.6 on smithi089 2023-10-26T06:42:14.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:14 smithi135 ceph-mon[131403]: Deploying daemon osd.6 on smithi089 2023-10-26T06:42:15.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:15 smithi089 ceph-mon[95031]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-26T06:42:15.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:15 smithi192 ceph-mon[131661]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-26T06:42:15.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:15 smithi135 ceph-mon[131403]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-26T06:42:17.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:17 smithi089 ceph-mon[95031]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:17.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:17 smithi192 ceph-mon[131661]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:17.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:17 smithi135 ceph-mon[131403]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:19.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:19 smithi089 ceph-mon[95031]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:19.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:19 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:19.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:19 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:19.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:19 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:19 smithi192 ceph-mon[131661]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:19 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:19.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:19 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:19.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:19 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:19 smithi135 ceph-mon[131403]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:19 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:19.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:19 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:19.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:19 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:21.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:21 smithi089 ceph-mon[95031]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:21 smithi192 ceph-mon[131661]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:21 smithi135 ceph-mon[131403]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:22.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.203+0000 7f7837fff700 1 -- 172.21.15.89:0/2532008700 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7850110760 con 0x7f78380a14d0 2023-10-26T06:42:22.206 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 6 on host 'smithi089' 2023-10-26T06:42:22.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f78380a14d0 msgr2=0x7f78380a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:22.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f78380a14d0 0x7f78380a3990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f785019a3a0 tx=0x7f7844009260 comp rx=0 tx=0).stop 2023-10-26T06:42:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501008a0 msgr2=0x7f7850198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501008a0 0x7f7850198e90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7840000c00 tx=0x7f7840004730 comp rx=0 tx=0).stop 2023-10-26T06:42:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 shutdown_connections 2023-10-26T06:42:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f78380a14d0 0x7f78380a3990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78501057c0 0x7f785019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78501008a0 0x7f7850198e90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 --2- 172.21.15.89:0/2532008700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7850101260 0x7f78501993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 >> 172.21.15.89:0/2532008700 conn(0x7f78500779e0 msgr2=0x7f78500fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:42:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 shutdown_connections 2023-10-26T06:42:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:42:22.206+0000 7f7855164700 1 -- 172.21.15.89:0/2532008700 wait complete. 2023-10-26T06:42:22.659 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.660 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.660 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:22.660 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:22.660 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.660 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:22.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:22.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.662 DEBUG:teuthology.orchestra.run.smithi089:osd.6> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.6.service 2023-10-26T06:42:22.667 INFO:tasks.cephadm:Deploying osd.7 on smithi135 with /dev/vg_nvme/lv_2... 2023-10-26T06:42:22.668 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-26T06:42:22.743 INFO:journalctl@ceph.osd.6.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:42:22.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:22.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:22.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:22.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:22.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:22.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:22.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:22.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:22.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:22.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:23.242 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:42:23.582 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:23 smithi089 ceph-mon[95031]: from='osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-26T06:42:23.582 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:23 smithi089 ceph-mon[95031]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-26T06:42:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:23 smithi135 ceph-mon[131403]: from='osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-26T06:42:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:23 smithi135 ceph-mon[131403]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-26T06:42:23.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:23 smithi192 ceph-mon[131661]: from='osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-26T06:42:23.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:23 smithi192 ceph-mon[131661]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-26T06:42:24.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-mon[95031]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-26T06:42:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-mon[95031]: osdmap e45: 7 total, 6 up, 7 in 2023-10-26T06:42:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-mon[95031]: from='osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:42:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-mon[95031]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:42:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-mon[95031]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:24.791 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:42:24.523+0000 7feab6afa700 -1 osd.6 0 waiting for initial osdmap 2023-10-26T06:42:24.791 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:42:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:42:24.550+0000 7feaaf0e8700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:42:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-26T06:42:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-26T06:42:24.810 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2023-10-26T06:42:24.811 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2023-10-26T06:42:24.811 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109117 s, 961 MB/s 2023-10-26T06:42:24.811 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2023-10-26T06:42:24.848 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:24 smithi135 ceph-mon[131403]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-26T06:42:24.849 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:24 smithi135 ceph-mon[131403]: osdmap e45: 7 total, 6 up, 7 in 2023-10-26T06:42:24.849 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:24.849 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:24 smithi135 ceph-mon[131403]: from='osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:42:24.849 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:24 smithi135 ceph-mon[131403]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:42:24.849 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:24 smithi135 ceph-mon[131403]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:24.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:24 smithi192 ceph-mon[131661]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-26T06:42:24.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:24 smithi192 ceph-mon[131661]: osdmap e45: 7 total, 6 up, 7 in 2023-10-26T06:42:24.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:24.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:24 smithi192 ceph-mon[131661]: from='osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:42:24.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:24 smithi192 ceph-mon[131661]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:42:24.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:24 smithi192 ceph-mon[131661]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:25.186 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi135:vg_nvme/lv_2 2023-10-26T06:42:25.745 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:42:25.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:25 smithi089 ceph-mon[95031]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:42:25.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:25 smithi089 ceph-mon[95031]: osdmap e46: 7 total, 6 up, 7 in 2023-10-26T06:42:25.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:25 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:25.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:25 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:25.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:25 smithi192 ceph-mon[131661]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:42:25.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:25 smithi192 ceph-mon[131661]: osdmap e46: 7 total, 6 up, 7 in 2023-10-26T06:42:25.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:25 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:25.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:25 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:25.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:25 smithi135 ceph-mon[131403]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:42:25.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:25 smithi135 ceph-mon[131403]: osdmap e46: 7 total, 6 up, 7 in 2023-10-26T06:42:25.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:25 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:25.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:25 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:26.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:42:26.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:42:26.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] boot 2023-10-26T06:42:26.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: osdmap e47: 7 total, 7 up, 7 in 2023-10-26T06:42:26.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:26.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:26.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:26.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:26.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:42:26.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:42:26.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] boot 2023-10-26T06:42:26.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: osdmap e47: 7 total, 7 up, 7 in 2023-10-26T06:42:26.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:26.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:26.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:26.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:26.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:42:26.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:42:26.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: osd.6 [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] boot 2023-10-26T06:42:26.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: osdmap e47: 7 total, 7 up, 7 in 2023-10-26T06:42:26.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-26T06:42:26.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-26T06:42:26.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:26.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:26.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:26.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:26.940 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 -- 172.21.15.135:0/3328702911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681000a0 msgr2=0x7f4a6810c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:26.941 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3328702911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681000a0 0x7f4a6810c540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b0a0 tx=0x7f4a6000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:42:26.941 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 -- 172.21.15.135:0/3328702911 shutdown_connections 2023-10-26T06:42:26.941 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3328702911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a681045b0 0x7f4a68104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:26.941 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3328702911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681000a0 0x7f4a6810c540 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:26.941 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3328702911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a680ff6e0 0x7f4a680ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:26.942 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 -- 172.21.15.135:0/3328702911 >> 172.21.15.135:0/3328702911 conn(0x7f4a680766c0 msgr2=0x7f4a68076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:42:26.942 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 -- 172.21.15.135:0/3328702911 shutdown_connections 2023-10-26T06:42:26.942 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.939+0000 7f4a6c941700 1 -- 172.21.15.135:0/3328702911 wait complete. 2023-10-26T06:42:26.944 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 Processor -- start 2023-10-26T06:42:26.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 -- start start 2023-10-26T06:42:26.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a680ff6e0 0x7f4a68197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:26.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a681000a0 0x7f4a68198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:26.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681045b0 0x7f4a681a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:26.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68198a00 con 0x7f4a680ff6e0 2023-10-26T06:42:26.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a681a3190 con 0x7f4a681045b0 2023-10-26T06:42:26.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a6c941700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a681a3300 con 0x7f4a681000a0 2023-10-26T06:42:26.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.940+0000 7f4a66d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681045b0 0x7f4a681a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:26.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681045b0 0x7f4a681a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58702/0 (socket says 172.21.15.135:58702) 2023-10-26T06:42:26.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 -- 172.21.15.135:0/3152537904 learned_addr learned my addr 172.21.15.135:0/3152537904 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-10-26T06:42:26.947 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a6659c700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a680ff6e0 0x7f4a68197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:26.947 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a65d9b700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a681000a0 0x7f4a68198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:26.947 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 -- 172.21.15.135:0/3152537904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a681000a0 msgr2=0x7f4a68198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:26.947 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a681000a0 0x7f4a68198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:26.947 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 -- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a680ff6e0 msgr2=0x7f4a68197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:26.948 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a680ff6e0 0x7f4a68197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:26.948 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 -- 172.21.15.135:0/3152537904 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a60009d00 con 0x7f4a681045b0 2023-10-26T06:42:26.948 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a65d9b700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a681000a0 0x7f4a68198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:42:26.948 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a6659c700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a680ff6e0 0x7f4a68197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:42:26.948 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a66d9d700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681045b0 0x7f4a681a2c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b070 tx=0x7f4a60007920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:42:26.949 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a5f7fe700 1 -- 172.21.15.135:0/3152537904 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000e040 con 0x7f4a681045b0 2023-10-26T06:42:26.949 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a5f7fe700 1 -- 172.21.15.135:0/3152537904 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a60009440 con 0x7f4a681045b0 2023-10-26T06:42:26.949 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a681a3580 con 0x7f4a681045b0 2023-10-26T06:42:26.949 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a5f7fe700 1 -- 172.21.15.135:0/3152537904 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6001b6b0 con 0x7f4a681045b0 2023-10-26T06:42:26.949 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.941+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a681a3a30 con 0x7f4a681045b0 2023-10-26T06:42:26.950 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.942+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a68107c20 con 0x7f4a681045b0 2023-10-26T06:42:26.950 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.943+0000 7f4a5f7fe700 1 -- 172.21.15.135:0/3152537904 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4a6001d480 con 0x7f4a681045b0 2023-10-26T06:42:26.950 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.944+0000 7f4a5f7fe700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4a540a14d0 0x7f4a540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:26.950 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.944+0000 7f4a5f7fe700 1 -- 172.21.15.135:0/3152537904 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5180+0+0 (secure 0 0 0) 0x7f4a600ce250 con 0x7f4a681045b0 2023-10-26T06:42:26.950 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.944+0000 7f4a6659c700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4a540a14d0 0x7f4a540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:26.951 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.945+0000 7f4a6659c700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4a540a14d0 0x7f4a540a3990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a500099d0 tx=0x7f4a50008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:42:26.951 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:26.947+0000 7f4a5f7fe700 1 -- 172.21.15.135:0/3152537904 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4a60098580 con 0x7f4a681045b0 2023-10-26T06:42:27.146 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:27.145+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a68069020 con 0x7f4a540a14d0 2023-10-26T06:42:27.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:27 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi089 2023-10-26T06:42:27.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:27 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi089 to 5757M 2023-10-26T06:42:27.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:27 smithi089 ceph-mon[95031]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:27.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:27.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:27.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:27.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:27 smithi089 ceph-mon[95031]: osdmap e48: 7 total, 7 up, 7 in 2023-10-26T06:42:27.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:27 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi089 2023-10-26T06:42:27.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:27 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi089 to 5757M 2023-10-26T06:42:27.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:27 smithi192 ceph-mon[131661]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:27.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:27.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:27.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:27.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:27 smithi192 ceph-mon[131661]: osdmap e48: 7 total, 7 up, 7 in 2023-10-26T06:42:27.934 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:27 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi089 2023-10-26T06:42:27.934 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:27 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi089 to 5757M 2023-10-26T06:42:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:27 smithi135 ceph-mon[131403]: from='client.24470 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:27.935 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:27 smithi135 ceph-mon[131403]: osdmap e48: 7 total, 7 up, 7 in 2023-10-26T06:42:28.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:28 smithi089 ceph-mon[95031]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:28.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:28 smithi192 ceph-mon[131661]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:28.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:28 smithi135 ceph-mon[131403]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:29 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/2946118609' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]: dispatch 2023-10-26T06:42:29.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:29 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]: dispatch 2023-10-26T06:42:29.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:29 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]': finished 2023-10-26T06:42:29.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:29 smithi089 ceph-mon[95031]: osdmap e49: 8 total, 7 up, 8 in 2023-10-26T06:42:29.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:29.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:29 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/2946118609' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]: dispatch 2023-10-26T06:42:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:29 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]: dispatch 2023-10-26T06:42:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:29 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]': finished 2023-10-26T06:42:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:29 smithi192 ceph-mon[131661]: osdmap e49: 8 total, 7 up, 8 in 2023-10-26T06:42:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:29.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:29 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/2946118609' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]: dispatch 2023-10-26T06:42:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:29 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]: dispatch 2023-10-26T06:42:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:29 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9fc3b27-d54b-49e6-accb-8e02a7782a6b"}]': finished 2023-10-26T06:42:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:29 smithi135 ceph-mon[131403]: osdmap e49: 8 total, 7 up, 8 in 2023-10-26T06:42:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:31.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:31 smithi089 ceph-mon[95031]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:31 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/3568906763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:31 smithi192 ceph-mon[131661]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:31 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/3568906763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:31.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:31 smithi135 ceph-mon[131403]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:31.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:31 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/3568906763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:33.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:33 smithi089 ceph-mon[95031]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:33 smithi192 ceph-mon[131661]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:33.438 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:33 smithi135 ceph-mon[131403]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:35.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:35 smithi089 ceph-mon[95031]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:35.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:35 smithi192 ceph-mon[131661]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:35 smithi135 ceph-mon[131403]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:37.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:37 smithi135 ceph-mon[131403]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:37.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-26T06:42:37.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:37.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:37 smithi089 ceph-mon[95031]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:37.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-26T06:42:37.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:37 smithi192 ceph-mon[131661]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-26T06:42:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:38.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:38 smithi135 ceph-mon[131403]: Deploying daemon osd.7 on smithi135 2023-10-26T06:42:38.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:38 smithi089 ceph-mon[95031]: Deploying daemon osd.7 on smithi135 2023-10-26T06:42:38.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:38 smithi192 ceph-mon[131661]: Deploying daemon osd.7 on smithi135 2023-10-26T06:42:39.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:39 smithi089 ceph-mon[95031]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:39.354 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:39 smithi135 ceph-mon[131403]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:39 smithi192 ceph-mon[131661]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:41.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:41 smithi089 ceph-mon[95031]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:41.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:41 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:41.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:41 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:41.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:41 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:41.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:41 smithi192 ceph-mon[131661]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:41.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:41 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:41.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:41 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:41.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:41 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:41 smithi135 ceph-mon[131403]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:41 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:41.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:41 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:41.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:41 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:43.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:43 smithi089 ceph-mon[95031]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:43 smithi192 ceph-mon[131661]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:43.412 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.410+0000 7f4a5f7fe700 1 -- 172.21.15.135:0/3152537904 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a68069020 con 0x7f4a540a14d0 2023-10-26T06:42:43.412 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 7 on host 'smithi135' 2023-10-26T06:42:43.414 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4a540a14d0 msgr2=0x7f4a540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:43.415 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4a540a14d0 0x7f4a540a3990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a500099d0 tx=0x7f4a50008040 comp rx=0 tx=0).stop 2023-10-26T06:42:43.415 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681045b0 msgr2=0x7f4a681a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:43.415 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681045b0 0x7f4a681a2c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b070 tx=0x7f4a60007920 comp rx=0 tx=0).stop 2023-10-26T06:42:43.415 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 shutdown_connections 2023-10-26T06:42:43.415 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4a540a14d0 0x7f4a540a3990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:43.416 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a680ff6e0 0x7f4a68197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:43.416 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a681045b0 0x7f4a681a2c50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:43.416 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 --2- 172.21.15.135:0/3152537904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a681000a0 0x7f4a68198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:43.416 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 >> 172.21.15.135:0/3152537904 conn(0x7f4a680766c0 msgr2=0x7f4a680fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:42:43.416 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 shutdown_connections 2023-10-26T06:42:43.417 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:42:43.412+0000 7f4a6c941700 1 -- 172.21.15.135:0/3152537904 wait complete. 2023-10-26T06:42:43.425 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:43 smithi135 ceph-mon[131403]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:43.839 DEBUG:teuthology.orchestra.run.smithi135:osd.7> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.7.service 2023-10-26T06:42:43.841 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2023-10-26T06:42:43.842 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-26T06:42:43.871 INFO:journalctl@ceph.osd.7.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:42:44.409 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:42:44.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:44 smithi192 ceph-mon[131661]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:44.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:44 smithi135 ceph-mon[131403]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:44.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:44.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:44 smithi089 ceph-mon[95031]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:45.032 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 06:42:44 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T06:42:44.725+0000 7f644d9e9640 -1 osd.7 0 log_to_monitors true 2023-10-26T06:42:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:45 smithi192 ceph-mon[131661]: from='osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-26T06:42:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:45 smithi192 ceph-mon[131661]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-26T06:42:45.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:45 smithi135 ceph-mon[131403]: from='osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-26T06:42:45.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:45 smithi135 ceph-mon[131403]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-26T06:42:45.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:45 smithi089 ceph-mon[95031]: from='osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-26T06:42:45.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:45 smithi089 ceph-mon[95031]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-26T06:42:45.903 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-26T06:42:45.904 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-26T06:42:45.904 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-10-26T06:42:45.904 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-10-26T06:42:45.904 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110144 s, 952 MB/s 2023-10-26T06:42:45.904 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-10-26T06:42:46.365 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2023-10-26T06:42:46.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:46 smithi192 ceph-mon[131661]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-26T06:42:46.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:46 smithi192 ceph-mon[131661]: osdmap e50: 8 total, 7 up, 8 in 2023-10-26T06:42:46.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:46 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:46.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:46 smithi192 ceph-mon[131661]: from='osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:42:46.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:46 smithi192 ceph-mon[131661]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:42:46.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:46 smithi192 ceph-mon[131661]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:46.751 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-mon[131403]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-26T06:42:46.751 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-mon[131403]: osdmap e50: 8 total, 7 up, 8 in 2023-10-26T06:42:46.751 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:46.751 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-mon[131403]: from='osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:42:46.752 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-mon[131403]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:42:46.752 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-mon[131403]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:46.752 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T06:42:46.478+0000 7f644285f700 -1 osd.7 0 waiting for initial osdmap 2023-10-26T06:42:46.752 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 06:42:46 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T06:42:46.491+0000 7f643ce51700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:42:46.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:46 smithi089 ceph-mon[95031]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-26T06:42:46.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:46 smithi089 ceph-mon[95031]: osdmap e50: 8 total, 7 up, 8 in 2023-10-26T06:42:46.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:46 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:46.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:46 smithi089 ceph-mon[95031]: from='osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:42:46.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:46 smithi089 ceph-mon[95031]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:42:46.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:46 smithi089 ceph-mon[95031]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-26T06:42:46.928 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:42:47.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:42:47.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: osdmap e51: 8 total, 7 up, 8 in 2023-10-26T06:42:47.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:47.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:47.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi135 2023-10-26T06:42:47.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi135 to 5722M 2023-10-26T06:42:47.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:47.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:47.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:47.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:47.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:47.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:42:47.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: osdmap e51: 8 total, 7 up, 8 in 2023-10-26T06:42:47.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:47.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:47.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi135 2023-10-26T06:42:47.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi135 to 5722M 2023-10-26T06:42:47.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:47.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:47.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:47.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:47.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:47.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:42:47.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: osdmap e51: 8 total, 7 up, 8 in 2023-10-26T06:42:47.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:47.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:47.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi135 2023-10-26T06:42:47.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:42:47.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi135 to 5722M 2023-10-26T06:42:47.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:47.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:47.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:47.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:42:47.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:47.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:42:47.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:42:48.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 -- 172.21.15.192:0/924068930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068b30 msgr2=0x7f45f010d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:48.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 --2- 172.21.15.192:0/924068930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068b30 0x7f45f010d690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f45e800b0a0 tx=0x7f45e800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:42:48.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 -- 172.21.15.192:0/924068930 shutdown_connections 2023-10-26T06:42:48.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 --2- 172.21.15.192:0/924068930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f01067c0 0x7f45f0106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:48.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 --2- 172.21.15.192:0/924068930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068b30 0x7f45f010d690 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:48.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 --2- 172.21.15.192:0/924068930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f0068190 0x7f45f00685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:48.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 -- 172.21.15.192:0/924068930 >> 172.21.15.192:0/924068930 conn(0x7f45f0075790 msgr2=0x7f45f0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:42:48.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.138+0000 7f45f7e94700 1 -- 172.21.15.192:0/924068930 shutdown_connections 2023-10-26T06:42:48.141 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 -- 172.21.15.192:0/924068930 wait complete. 2023-10-26T06:42:48.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 Processor -- start 2023-10-26T06:42:48.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 -- start start 2023-10-26T06:42:48.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068190 0x7f45f00ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:48.144 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f0068b30 0x7f45f00ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:48.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f01067c0 0x7f45f01002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:48.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f01a50d0 con 0x7f45f01067c0 2023-10-26T06:42:48.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f01a5240 con 0x7f45f0068b30 2023-10-26T06:42:48.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f7e94700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f01a5400 con 0x7f45f0068190 2023-10-26T06:42:48.145 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f5c30700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068190 0x7f45f00ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f5c30700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068190 0x7f45f00ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54160/0 (socket says 172.21.15.192:54160) 2023-10-26T06:42:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.139+0000 7f45f5c30700 1 -- 172.21.15.192:0/2205500891 learned_addr learned my addr 172.21.15.192:0/2205500891 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-10-26T06:42:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f6431700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f01067c0 0x7f45f01002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f542f700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f0068b30 0x7f45f00ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:48.146 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f5c30700 1 -- 172.21.15.192:0/2205500891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f0068b30 msgr2=0x7f45f00ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f5c30700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f0068b30 0x7f45f00ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f5c30700 1 -- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f01067c0 msgr2=0x7f45f01002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:42:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f5c30700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f01067c0 0x7f45f01002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:42:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f5c30700 1 -- 172.21.15.192:0/2205500891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8009d00 con 0x7f45f0068190 2023-10-26T06:42:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f542f700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f0068b30 0x7f45f00ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:42:48.147 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f5c30700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068190 0x7f45f00ff820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c9b0 tx=0x7f45e000cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:42:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45e6ffd700 1 -- 172.21.15.192:0/2205500891 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0007b00 con 0x7f45f0068190 2023-10-26T06:42:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45e6ffd700 1 -- 172.21.15.192:0/2205500891 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45e0007c60 con 0x7f45f0068190 2023-10-26T06:42:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f01a56e0 con 0x7f45f0068190 2023-10-26T06:42:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45e6ffd700 1 -- 172.21.15.192:0/2205500891 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0018660 con 0x7f45f0068190 2023-10-26T06:42:48.148 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.140+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f01a5c30 con 0x7f45f0068190 2023-10-26T06:42:48.149 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.141+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f0108de0 con 0x7f45f0068190 2023-10-26T06:42:48.149 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.142+0000 7f45e6ffd700 1 -- 172.21.15.192:0/2205500891 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f45e0018880 con 0x7f45f0068190 2023-10-26T06:42:48.149 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.142+0000 7f45e6ffd700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45dc0a15d0 0x7f45dc0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:42:48.149 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.143+0000 7f45e6ffd700 1 -- 172.21.15.192:0/2205500891 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5645+0+0 (secure 0 0 0) 0x7f45e00cc5e0 con 0x7f45f0068190 2023-10-26T06:42:48.149 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.143+0000 7f45f542f700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45dc0a15d0 0x7f45dc0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:42:48.150 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.143+0000 7f45f542f700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45dc0a15d0 0x7f45dc0a3a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45f0100f00 tx=0x7f45ec005f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:42:48.150 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.146+0000 7f45e6ffd700 1 -- 172.21.15.192:0/2205500891 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f45e0096720 con 0x7f45f0068190 2023-10-26T06:42:48.343 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:42:48.342+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f45f0100cb0 con 0x7f45dc0a15d0 2023-10-26T06:42:48.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:42:48.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:42:48.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] boot 2023-10-26T06:42:48.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: osdmap e52: 8 total, 8 up, 8 in 2023-10-26T06:42:48.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:48.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:48.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:48.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:48.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:48 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:48.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:42:48.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:42:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] boot 2023-10-26T06:42:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: osdmap e52: 8 total, 8 up, 8 in 2023-10-26T06:42:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:48.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:48.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:48 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:48.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:42:48.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:42:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: osd.7 [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] boot 2023-10-26T06:42:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: osdmap e52: 8 total, 8 up, 8 in 2023-10-26T06:42:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-26T06:42:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:48.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:42:48.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:42:48.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:48 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:49.673 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:49 smithi192 ceph-mon[131661]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:49.674 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:49 smithi192 ceph-mon[131661]: osdmap e53: 8 total, 8 up, 8 in 2023-10-26T06:42:49.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:49 smithi089 ceph-mon[95031]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:49.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:49 smithi089 ceph-mon[95031]: osdmap e53: 8 total, 8 up, 8 in 2023-10-26T06:42:49.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:49 smithi135 ceph-mon[131403]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:42:49.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:49 smithi135 ceph-mon[131403]: osdmap e53: 8 total, 8 up, 8 in 2023-10-26T06:42:50.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:50 smithi089 ceph-mon[95031]: osdmap e54: 8 total, 8 up, 8 in 2023-10-26T06:42:50.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:50 smithi089 ceph-mon[95031]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:50.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:50 smithi089 ceph-mon[95031]: osdmap e55: 8 total, 8 up, 8 in 2023-10-26T06:42:50.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:50 smithi192 ceph-mon[131661]: osdmap e54: 8 total, 8 up, 8 in 2023-10-26T06:42:50.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:50 smithi192 ceph-mon[131661]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:50 smithi192 ceph-mon[131661]: osdmap e55: 8 total, 8 up, 8 in 2023-10-26T06:42:50.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:50 smithi135 ceph-mon[131403]: osdmap e54: 8 total, 8 up, 8 in 2023-10-26T06:42:50.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:50 smithi135 ceph-mon[131403]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:50 smithi135 ceph-mon[131403]: osdmap e55: 8 total, 8 up, 8 in 2023-10-26T06:42:51.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:51 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/3828606521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]: dispatch 2023-10-26T06:42:51.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:51 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]: dispatch 2023-10-26T06:42:51.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:51 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]': finished 2023-10-26T06:42:51.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:51 smithi089 ceph-mon[95031]: osdmap e56: 9 total, 8 up, 9 in 2023-10-26T06:42:51.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:42:51.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:51 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/3212169885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:51.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:51 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/3828606521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]: dispatch 2023-10-26T06:42:51.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:51 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]: dispatch 2023-10-26T06:42:51.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:51 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]': finished 2023-10-26T06:42:51.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:51 smithi192 ceph-mon[131661]: osdmap e56: 9 total, 8 up, 9 in 2023-10-26T06:42:51.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:42:51.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:51 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/3212169885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:51.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:51 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/3828606521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]: dispatch 2023-10-26T06:42:51.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:51 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]: dispatch 2023-10-26T06:42:51.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:51 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "809944cc-518e-49bc-b6fb-b56823f81148"}]': finished 2023-10-26T06:42:51.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:51 smithi135 ceph-mon[131403]: osdmap e56: 9 total, 8 up, 9 in 2023-10-26T06:42:51.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:42:51.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:51 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/3212169885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:42:52.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:52 smithi089 ceph-mon[95031]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:52.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:52 smithi192 ceph-mon[131661]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:52.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:52 smithi135 ceph-mon[131403]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:54.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:54 smithi192 ceph-mon[131661]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:55.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:54 smithi089 ceph-mon[95031]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:55.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:54 smithi135 ceph-mon[131403]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:57.270 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:57 smithi192 ceph-mon[131661]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:57.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:57 smithi089 ceph-mon[95031]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:57 smithi135 ceph-mon[131403]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:58.288 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-26T06:42:58.289 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:58.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-26T06:42:58.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:58.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-26T06:42:58.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:42:59.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:59 smithi089 ceph-mon[95031]: Deploying daemon osd.8 on smithi192 2023-10-26T06:42:59.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:42:59 smithi089 ceph-mon[95031]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:59.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:59 smithi192 ceph-mon[131661]: Deploying daemon osd.8 on smithi192 2023-10-26T06:42:59.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:42:59 smithi192 ceph-mon[131661]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:42:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:59 smithi135 ceph-mon[131403]: Deploying daemon osd.8 on smithi192 2023-10-26T06:42:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:42:59 smithi135 ceph-mon[131403]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:01.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:01 smithi089 ceph-mon[95031]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:01.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:01 smithi192 ceph-mon[131661]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:01.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:01 smithi135 ceph-mon[131403]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:02.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:02 smithi192 ceph-mon[131661]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:02.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:02.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:02.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:02.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:02 smithi135 ceph-mon[131403]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:02.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:03.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:02 smithi089 ceph-mon[95031]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:03.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:03.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:03.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:04.405 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.404+0000 7f45e6ffd700 1 -- 172.21.15.192:0/2205500891 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f45f0100cb0 con 0x7f45dc0a15d0 2023-10-26T06:43:04.405 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2023-10-26T06:43:04.407 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45dc0a15d0 msgr2=0x7f45dc0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:04.407 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45dc0a15d0 0x7f45dc0a3a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45f0100f00 tx=0x7f45ec005f50 comp rx=0 tx=0).stop 2023-10-26T06:43:04.408 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068190 msgr2=0x7f45f00ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:04.408 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068190 0x7f45f00ff820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c9b0 tx=0x7f45e000cd70 comp rx=0 tx=0).stop 2023-10-26T06:43:04.408 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 shutdown_connections 2023-10-26T06:43:04.408 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45dc0a15d0 0x7f45dc0a3a90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:04.408 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f01067c0 0x7f45f01002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:04.409 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45f0068190 0x7f45f00ff820 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:04.409 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 --2- 172.21.15.192:0/2205500891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f0068b30 0x7f45f00ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:04.409 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 >> 172.21.15.192:0/2205500891 conn(0x7f45f0075790 msgr2=0x7f45f00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:43:04.409 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 shutdown_connections 2023-10-26T06:43:04.409 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:04.406+0000 7f45f7e94700 1 -- 172.21.15.192:0/2205500891 wait complete. 2023-10-26T06:43:04.805 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.8.service 2023-10-26T06:43:04.808 INFO:tasks.cephadm:Deploying osd.9 on smithi089 with /dev/vg_nvme/lv_1... 2023-10-26T06:43:04.809 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-26T06:43:04.840 INFO:journalctl@ceph.osd.8.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:43:05.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:05.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:05 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:05.328 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:05.328 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.328 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.328 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:05.328 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:05.328 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.329 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.329 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.329 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:05.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:05.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:05.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:05 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:05.553 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:43:05.895 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:43:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:43:05.665+0000 7ff7ef31e640 -1 osd.8 0 log_to_monitors true 2023-10-26T06:43:06.125 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:06 smithi089 ceph-mon[95031]: from='osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-26T06:43:06.125 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:06 smithi089 ceph-mon[95031]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-26T06:43:06.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:06 smithi192 ceph-mon[131661]: from='osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-26T06:43:06.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:06 smithi192 ceph-mon[131661]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-26T06:43:06.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:06 smithi135 ceph-mon[131403]: from='osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-26T06:43:06.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:06 smithi135 ceph-mon[131403]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-26T06:43:07.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:07 smithi089 ceph-mon[95031]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:07 smithi089 ceph-mon[95031]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-26T06:43:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:07 smithi089 ceph-mon[95031]: osdmap e57: 9 total, 8 up, 9 in 2023-10-26T06:43:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:07 smithi089 ceph-mon[95031]: from='osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:43:07.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:07.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:07 smithi089 ceph-mon[95031]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:43:07.395 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:43:07.081+0000 7ff7e4194700 -1 osd.8 0 waiting for initial osdmap 2023-10-26T06:43:07.395 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:43:07.094+0000 7ff7ddf85700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:43:07.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-mon[131661]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:07.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-mon[131661]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-26T06:43:07.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-mon[131661]: osdmap e57: 9 total, 8 up, 9 in 2023-10-26T06:43:07.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-mon[131661]: from='osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:43:07.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:07.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:07 smithi192 ceph-mon[131661]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:43:07.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:07 smithi135 ceph-mon[131403]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:07.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:07 smithi135 ceph-mon[131403]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-26T06:43:07.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:07 smithi135 ceph-mon[131403]: osdmap e57: 9 total, 8 up, 9 in 2023-10-26T06:43:07.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:07 smithi135 ceph-mon[131403]: from='osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:43:07.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:07.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:07 smithi135 ceph-mon[131403]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:43:07.694 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-26T06:43:07.694 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-26T06:43:07.695 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-10-26T06:43:07.695 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-10-26T06:43:07.695 INFO:teuthology.orchestra.run.smithi089.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0321686 s, 326 MB/s 2023-10-26T06:43:07.695 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-10-26T06:43:08.143 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2023-10-26T06:43:08.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:43:08.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: osdmap e58: 9 total, 8 up, 9 in 2023-10-26T06:43:08.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:08.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:08.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:08.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:08.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:43:08.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: osdmap e58: 9 total, 8 up, 9 in 2023-10-26T06:43:08.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:08.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:08.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:08.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:08.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:43:08.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: osdmap e58: 9 total, 8 up, 9 in 2023-10-26T06:43:08.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:08.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:08.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:08.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:08.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:08.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:43:09.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:43:09.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:43:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi192 2023-10-26T06:43:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi192 to 5722M 2023-10-26T06:43:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: pgmap v169: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] boot 2023-10-26T06:43:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: osdmap e59: 9 total, 9 up, 9 in 2023-10-26T06:43:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:09 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:09.424 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:43:09.424 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:43:09.424 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi192 2023-10-26T06:43:09.424 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi192 to 5722M 2023-10-26T06:43:09.424 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: pgmap v169: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:09.425 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] boot 2023-10-26T06:43:09.425 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: osdmap e59: 9 total, 9 up, 9 in 2023-10-26T06:43:09.425 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:09 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:09.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:43:09.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:43:09.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi192 2023-10-26T06:43:09.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi192 to 5722M 2023-10-26T06:43:09.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: pgmap v169: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-26T06:43:09.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: osd.8 [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] boot 2023-10-26T06:43:09.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: osdmap e59: 9 total, 9 up, 9 in 2023-10-26T06:43:09.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:09 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-26T06:43:10.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 -- 172.21.15.89:0/2435930995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 msgr2=0x7f69f4105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:10.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 --2- 172.21.15.89:0/2435930995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f4105ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009990 tx=0x7f69e4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:43:10.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 -- 172.21.15.89:0/2435930995 shutdown_connections 2023-10-26T06:43:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 --2- 172.21.15.89:0/2435930995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f4105ba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 --2- 172.21.15.89:0/2435930995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f4101260 0x7f69f410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 --2- 172.21.15.89:0/2435930995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f41008a0 0x7f69f4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 -- 172.21.15.89:0/2435930995 >> 172.21.15.89:0/2435930995 conn(0x7f69f40779e0 msgr2=0x7f69f4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:43:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 -- 172.21.15.89:0/2435930995 shutdown_connections 2023-10-26T06:43:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.389+0000 7f69fa833700 1 -- 172.21.15.89:0/2435930995 wait complete. 2023-10-26T06:43:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.390+0000 7f69fa833700 1 Processor -- start 2023-10-26T06:43:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69fa833700 1 -- start start 2023-10-26T06:43:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69fa833700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f41008a0 0x7f69f4198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69fa833700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f4101260 0x7f69f4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69fa833700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69fa833700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f419de00 con 0x7f69f41057c0 2023-10-26T06:43:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69fa833700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f419df70 con 0x7f69f4101260 2023-10-26T06:43:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69fa833700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f419e130 con 0x7f69f41008a0 2023-10-26T06:43:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69f8dd0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69f8dd0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58986/0 (socket says 172.21.15.89:58986) 2023-10-26T06:43:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69f8dd0700 1 -- 172.21.15.89:0/70020033 learned_addr learned my addr 172.21.15.89:0/70020033 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:43:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69f37fe700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f4101260 0x7f69f4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.391+0000 7f69f3fff700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f41008a0 0x7f69f4198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f8dd0700 1 -- 172.21.15.89:0/70020033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f41008a0 msgr2=0x7f69f4198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f8dd0700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f41008a0 0x7f69f4198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f8dd0700 1 -- 172.21.15.89:0/70020033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f4101260 msgr2=0x7f69f4199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f8dd0700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f4101260 0x7f69f4199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f8dd0700 1 -- 172.21.15.89:0/70020033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40095f0 con 0x7f69f41057c0 2023-10-26T06:43:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f8dd0700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f419d7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00efc0 tx=0x7f69ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:43:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f17fa700 1 -- 172.21.15.89:0/70020033 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec018070 con 0x7f69f41057c0 2023-10-26T06:43:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f419e3b0 con 0x7f69f41057c0 2023-10-26T06:43:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f17fa700 1 -- 172.21.15.89:0/70020033 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69ec00f040 con 0x7f69f41057c0 2023-10-26T06:43:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69f17fa700 1 -- 172.21.15.89:0/70020033 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec00cbc0 con 0x7f69f41057c0 2023-10-26T06:43:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.392+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f41a5c70 con 0x7f69f41057c0 2023-10-26T06:43:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.393+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f4101f30 con 0x7f69f41057c0 2023-10-26T06:43:10.400 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:10 smithi192 ceph-mon[131661]: osdmap e60: 9 total, 9 up, 9 in 2023-10-26T06:43:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.395+0000 7f69f17fa700 1 -- 172.21.15.89:0/70020033 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f69ec00cd40 con 0x7f69f41057c0 2023-10-26T06:43:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.396+0000 7f69f17fa700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69dc0a14d0 0x7f69dc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.397+0000 7f69f17fa700 1 -- 172.21.15.89:0/70020033 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f69ec0cc800 con 0x7f69f41057c0 2023-10-26T06:43:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.397+0000 7f69f3fff700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69dc0a14d0 0x7f69dc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.397+0000 7f69f3fff700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69dc0a14d0 0x7f69dc0a3990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f69e4006010 tx=0x7f69e400b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:43:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.399+0000 7f69f17fa700 1 -- 172.21.15.89:0/70020033 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f69ec096710 con 0x7f69f41057c0 2023-10-26T06:43:10.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:10 smithi135 ceph-mon[131403]: osdmap e60: 9 total, 9 up, 9 in 2023-10-26T06:43:10.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:10 smithi089 ceph-mon[95031]: osdmap e60: 9 total, 9 up, 9 in 2023-10-26T06:43:10.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:10.609+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f69f4110760 con 0x7f69dc0a14d0 2023-10-26T06:43:11.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:11 smithi192 ceph-mon[131661]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:11.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:11 smithi192 ceph-mon[131661]: osdmap e61: 9 total, 9 up, 9 in 2023-10-26T06:43:11.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:11.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:11.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:11 smithi135 ceph-mon[131403]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:11 smithi135 ceph-mon[131403]: osdmap e61: 9 total, 9 up, 9 in 2023-10-26T06:43:11.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:11.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:11.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:11 smithi089 ceph-mon[95031]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:11 smithi089 ceph-mon[95031]: osdmap e61: 9 total, 9 up, 9 in 2023-10-26T06:43:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:12.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:12 smithi089 ceph-mon[95031]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:12.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:12 smithi089 ceph-mon[95031]: osdmap e62: 9 total, 9 up, 9 in 2023-10-26T06:43:12.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:12 smithi192 ceph-mon[131661]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:12.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:12 smithi192 ceph-mon[131661]: osdmap e62: 9 total, 9 up, 9 in 2023-10-26T06:43:12.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:12 smithi135 ceph-mon[131403]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:12.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:12 smithi135 ceph-mon[131403]: osdmap e62: 9 total, 9 up, 9 in 2023-10-26T06:43:13.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:13 smithi192 ceph-mon[131661]: pgmap v175: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:13.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:13 smithi192 ceph-mon[131661]: osdmap e63: 9 total, 9 up, 9 in 2023-10-26T06:43:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:13 smithi135 ceph-mon[131403]: pgmap v175: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:13 smithi135 ceph-mon[131403]: osdmap e63: 9 total, 9 up, 9 in 2023-10-26T06:43:13.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:13 smithi089 ceph-mon[95031]: pgmap v175: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:13.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:13 smithi089 ceph-mon[95031]: osdmap e63: 9 total, 9 up, 9 in 2023-10-26T06:43:14.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:14 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/914923359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38fdca-e054-402a-b88e-7abc1659d361"}]: dispatch 2023-10-26T06:43:14.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:14 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/914923359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f38fdca-e054-402a-b88e-7abc1659d361"}]': finished 2023-10-26T06:43:14.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:14 smithi192 ceph-mon[131661]: osdmap e64: 10 total, 9 up, 10 in 2023-10-26T06:43:14.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:14.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:14 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/914923359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38fdca-e054-402a-b88e-7abc1659d361"}]: dispatch 2023-10-26T06:43:14.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:14 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/914923359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f38fdca-e054-402a-b88e-7abc1659d361"}]': finished 2023-10-26T06:43:14.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:14 smithi135 ceph-mon[131403]: osdmap e64: 10 total, 9 up, 10 in 2023-10-26T06:43:14.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:14.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:14 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/914923359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38fdca-e054-402a-b88e-7abc1659d361"}]: dispatch 2023-10-26T06:43:14.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:14 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/914923359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f38fdca-e054-402a-b88e-7abc1659d361"}]': finished 2023-10-26T06:43:14.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:14 smithi089 ceph-mon[95031]: osdmap e64: 10 total, 9 up, 10 in 2023-10-26T06:43:14.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:15.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:15 smithi192 ceph-mon[131661]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-26T06:43:15.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:15 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2459999958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:43:15.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:15 smithi135 ceph-mon[131403]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-26T06:43:15.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:15 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2459999958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:43:15.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:15 smithi089 ceph-mon[95031]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-26T06:43:15.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:15 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2459999958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:43:17.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:17 smithi192 ceph-mon[131661]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-26T06:43:17.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:17 smithi135 ceph-mon[131403]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-26T06:43:17.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:17 smithi089 ceph-mon[95031]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-26T06:43:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:19 smithi192 ceph-mon[131661]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-10-26T06:43:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:19 smithi135 ceph-mon[131403]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-10-26T06:43:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:19 smithi089 ceph-mon[95031]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-10-26T06:43:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:21 smithi192 ceph-mon[131661]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-26T06:43:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:21 smithi135 ceph-mon[131403]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-26T06:43:21.527 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:21 smithi089 ceph-mon[95031]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-26T06:43:22.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-26T06:43:22.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:22.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-26T06:43:22.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:22.485 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-26T06:43:22.486 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:23 smithi192 ceph-mon[131661]: Deploying daemon osd.9 on smithi089 2023-10-26T06:43:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:23 smithi192 ceph-mon[131661]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-26T06:43:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:23 smithi135 ceph-mon[131403]: Deploying daemon osd.9 on smithi089 2023-10-26T06:43:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:23 smithi135 ceph-mon[131403]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-26T06:43:23.524 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:23 smithi089 ceph-mon[95031]: Deploying daemon osd.9 on smithi089 2023-10-26T06:43:23.524 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:23 smithi089 ceph-mon[95031]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-26T06:43:25.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:25 smithi089 ceph-mon[95031]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:25 smithi192 ceph-mon[131661]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:25 smithi135 ceph-mon[131403]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:27.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:27 smithi192 ceph-mon[131661]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:27.463 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:27 smithi089 ceph-mon[95031]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:27.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:27 smithi135 ceph-mon[131403]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:28.938 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:28.938 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:28.938 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:28.938 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:28 smithi089 ceph-mon[95031]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:28.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:28.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:28.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:28.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:28 smithi135 ceph-mon[131403]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:29.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:29.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:29.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:29.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:28 smithi192 ceph-mon[131661]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.922+0000 7f69f17fa700 1 -- 172.21.15.89:0/70020033 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f69f4110760 con 0x7f69dc0a14d0 2023-10-26T06:43:30.924 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 9 on host 'smithi089' 2023-10-26T06:43:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69dc0a14d0 msgr2=0x7f69dc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69dc0a14d0 0x7f69dc0a3990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f69e4006010 tx=0x7f69e400b260 comp rx=0 tx=0).stop 2023-10-26T06:43:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 msgr2=0x7f69f419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f419d7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00efc0 tx=0x7f69ec00c3a0 comp rx=0 tx=0).stop 2023-10-26T06:43:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 shutdown_connections 2023-10-26T06:43:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69dc0a14d0 0x7f69dc0a3990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f41057c0 0x7f69f419d7f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f41008a0 0x7f69f4198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 --2- 172.21.15.89:0/70020033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f4101260 0x7f69f4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 >> 172.21.15.89:0/70020033 conn(0x7f69f40779e0 msgr2=0x7f69f410ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:43:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.924+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 shutdown_connections 2023-10-26T06:43:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:43:30.925+0000 7f69fa833700 1 -- 172.21.15.89:0/70020033 wait complete. 2023-10-26T06:43:31.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:31 smithi089 ceph-mon[95031]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:31.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:31 smithi192 ceph-mon[131661]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.417 DEBUG:teuthology.orchestra.run.smithi089:osd.9> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.9.service 2023-10-26T06:43:31.422 INFO:tasks.cephadm:Deploying osd.10 on smithi135 with /dev/vg_nvme/lv_1... 2023-10-26T06:43:31.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-26T06:43:31.454 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:31 smithi135 ceph-mon[131403]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:31.455 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.455 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.455 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.455 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:31.492 INFO:journalctl@ceph.osd.9.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:43:31.995 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:43:32.022 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:43:31 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:43:31.693+0000 7f1d67d17640 -1 osd.9 0 log_to_monitors true 2023-10-26T06:43:32.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:32 smithi089 ceph-mon[95031]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-26T06:43:32.392 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:32 smithi135 ceph-mon[131403]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-26T06:43:32.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:32 smithi192 ceph-mon[131661]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-26T06:43:33.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:43:33 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:43:33.050+0000 7f1d5cb8d700 -1 osd.9 0 waiting for initial osdmap 2023-10-26T06:43:33.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:43:33 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:43:33.067+0000 7f1d5517b700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:43:33.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:33 smithi089 ceph-mon[95031]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:33.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:33 smithi089 ceph-mon[95031]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-26T06:43:33.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:33 smithi089 ceph-mon[95031]: osdmap e65: 10 total, 9 up, 10 in 2023-10-26T06:43:33.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:33 smithi089 ceph-mon[95031]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:43:33.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:33 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:33 smithi192 ceph-mon[131661]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:33.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:33 smithi192 ceph-mon[131661]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-26T06:43:33.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:33 smithi192 ceph-mon[131661]: osdmap e65: 10 total, 9 up, 10 in 2023-10-26T06:43:33.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:33 smithi192 ceph-mon[131661]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:43:33.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:33 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:33 smithi135 ceph-mon[131403]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:33 smithi135 ceph-mon[131403]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-26T06:43:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:33 smithi135 ceph-mon[131403]: osdmap e65: 10 total, 9 up, 10 in 2023-10-26T06:43:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:33 smithi135 ceph-mon[131403]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-10-26T06:43:33.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:33 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:33.550 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-26T06:43:33.550 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-26T06:43:33.550 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2023-10-26T06:43:33.551 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2023-10-26T06:43:33.551 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110829 s, 946 MB/s 2023-10-26T06:43:33.551 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2023-10-26T06:43:33.955 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi135:vg_nvme/lv_1 2023-10-26T06:43:34.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:34 smithi135 ceph-mon[131403]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:43:34.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:34 smithi135 ceph-mon[131403]: osdmap e66: 10 total, 9 up, 10 in 2023-10-26T06:43:34.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:34.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:34.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:34 smithi192 ceph-mon[131661]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:43:34.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:34 smithi192 ceph-mon[131661]: osdmap e66: 10 total, 9 up, 10 in 2023-10-26T06:43:34.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:34.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:34.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:34 smithi089 ceph-mon[95031]: from='osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-10-26T06:43:34.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:34 smithi089 ceph-mon[95031]: osdmap e66: 10 total, 9 up, 10 in 2023-10-26T06:43:34.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:34.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:34.557 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:43:35.323 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:35 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:43:35.324 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:35 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:43:35.324 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:35 smithi135 ceph-mon[131403]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:35.324 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:35 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:35.324 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:35 smithi135 ceph-mon[131403]: osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] boot 2023-10-26T06:43:35.325 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:35 smithi135 ceph-mon[131403]: osdmap e67: 10 total, 10 up, 10 in 2023-10-26T06:43:35.325 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:35 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:35.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:35 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:43:35.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:35 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:43:35.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:35 smithi192 ceph-mon[131661]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:35.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:35 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:35.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:35 smithi192 ceph-mon[131661]: osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] boot 2023-10-26T06:43:35.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:35 smithi192 ceph-mon[131661]: osdmap e67: 10 total, 10 up, 10 in 2023-10-26T06:43:35.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:35 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:35 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:43:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:35 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:43:35.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:35 smithi089 ceph-mon[95031]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-26T06:43:35.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:35 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:35.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:35 smithi089 ceph-mon[95031]: osd.9 [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] boot 2023-10-26T06:43:35.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:35 smithi089 ceph-mon[95031]: osdmap e67: 10 total, 10 up, 10 in 2023-10-26T06:43:35.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:35 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-26T06:43:35.777 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 -- 172.21.15.135:0/1855350664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 msgr2=0x7fe30c068570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:35.777 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 --2- 172.21.15.135:0/1855350664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c068570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe300009990 tx=0x7fe300009ca0 comp rx=0 tx=0).stop 2023-10-26T06:43:35.777 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 -- 172.21.15.135:0/1855350664 shutdown_connections 2023-10-26T06:43:35.777 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 --2- 172.21.15.135:0/1855350664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe30c10d300 0x7fe30c10f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:35.778 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 --2- 172.21.15.135:0/1855350664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c068570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:35.778 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 --2- 172.21.15.135:0/1855350664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe30c068ab0 0x7fe30c10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:35.778 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 -- 172.21.15.135:0/1855350664 >> 172.21.15.135:0/1855350664 conn(0x7fe30c075790 msgr2=0x7fe30c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:43:35.778 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 -- 172.21.15.135:0/1855350664 shutdown_connections 2023-10-26T06:43:35.779 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.773+0000 7fe310c99700 1 -- 172.21.15.135:0/1855350664 wait complete. 2023-10-26T06:43:35.779 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 Processor -- start 2023-10-26T06:43:35.779 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 -- start start 2023-10-26T06:43:35.779 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:35.779 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe30c068ab0 0x7fe30c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:35.779 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe30c10d300 0x7fe30c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:35.780 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c19dc80 con 0x7fe30c068ab0 2023-10-26T06:43:35.780 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c19ddf0 con 0x7fe30c068190 2023-10-26T06:43:35.780 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe310c99700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c19dfb0 con 0x7fe30c10d300 2023-10-26T06:43:35.780 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:35.780 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58848/0 (socket says 172.21.15.135:58848) 2023-10-26T06:43:35.781 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 -- 172.21.15.135:0/3747956386 learned_addr learned my addr 172.21.15.135:0/3747956386 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-10-26T06:43:35.781 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe309d9b700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe30c068ab0 0x7fe30c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:35.781 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30ad9d700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe30c10d300 0x7fe30c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:35.781 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 -- 172.21.15.135:0/3747956386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe30c10d300 msgr2=0x7fe30c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:35.781 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe30c10d300 0x7fe30c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:35.782 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 -- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe30c068ab0 msgr2=0x7fe30c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:35.782 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe30c068ab0 0x7fe30c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:35.782 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 -- 172.21.15.135:0/3747956386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3000095f0 con 0x7fe30c068190 2023-10-26T06:43:35.782 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe309d9b700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe30c068ab0 0x7fe30c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:43:35.782 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.774+0000 7fe30a59c700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c198d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe30000f880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:43:35.782 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.775+0000 7fe2fb7fe700 1 -- 172.21.15.135:0/3747956386 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30001d070 con 0x7fe30c068190 2023-10-26T06:43:35.783 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.775+0000 7fe2fb7fe700 1 -- 172.21.15.135:0/3747956386 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe30000fde0 con 0x7fe30c068190 2023-10-26T06:43:35.783 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.775+0000 7fe2fb7fe700 1 -- 172.21.15.135:0/3747956386 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300017770 con 0x7fe30c068190 2023-10-26T06:43:35.783 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.775+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c19e230 con 0x7fe30c068190 2023-10-26T06:43:35.783 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.775+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c1a5920 con 0x7fe30c068190 2023-10-26T06:43:35.784 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.776+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c108520 con 0x7fe30c068190 2023-10-26T06:43:35.784 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.777+0000 7fe2fb7fe700 1 -- 172.21.15.135:0/3747956386 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe300022aa0 con 0x7fe30c068190 2023-10-26T06:43:35.785 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.777+0000 7fe2fb7fe700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe2f40a14d0 0x7fe2f40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:35.785 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.777+0000 7fe2fb7fe700 1 -- 172.21.15.135:0/3747956386 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6476+0+0 (secure 0 0 0) 0x7fe3000cdd10 con 0x7fe30c068190 2023-10-26T06:43:35.785 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.777+0000 7fe309d9b700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe2f40a14d0 0x7fe2f40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:35.785 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.778+0000 7fe309d9b700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe2f40a14d0 0x7fe2f40a3990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005fd0 tx=0x7fe2fc005e30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:43:35.785 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.781+0000 7fe2fb7fe700 1 -- 172.21.15.135:0/3747956386 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe300097b30 con 0x7fe30c068190 2023-10-26T06:43:35.978 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:35.976+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe30c19a090 con 0x7fe2f40a14d0 2023-10-26T06:43:36.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: osdmap e68: 10 total, 10 up, 10 in 2023-10-26T06:43:36.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi089 2023-10-26T06:43:36.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi089 to 4317M 2023-10-26T06:43:36.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:36.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:36.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:36.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:36.399 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:36.400 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:36 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: osdmap e68: 10 total, 10 up, 10 in 2023-10-26T06:43:36.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi089 2023-10-26T06:43:36.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi089 to 4317M 2023-10-26T06:43:36.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:36.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:36.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:36.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:36.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:36.472 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:36 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: osdmap e68: 10 total, 10 up, 10 in 2023-10-26T06:43:36.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi089 2023-10-26T06:43:36.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.543 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.543 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.543 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.543 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:36.543 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi089 to 4317M 2023-10-26T06:43:36.544 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.544 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.544 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:36.544 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.544 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:36.544 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:36.545 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:36.545 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:36.545 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:36.545 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:36.545 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:36 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:37.118 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:37 smithi135 ceph-mon[131403]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:37.119 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:37 smithi135 ceph-mon[131403]: from='client.24533 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:37 smithi192 ceph-mon[131661]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:37 smithi192 ceph-mon[131661]: from='client.24533 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:37.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:37 smithi089 ceph-mon[95031]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:37.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:37 smithi089 ceph-mon[95031]: from='client.24533 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:39 smithi192 ceph-mon[131661]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:39 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/1961879002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]: dispatch 2023-10-26T06:43:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:39 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]: dispatch 2023-10-26T06:43:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:39 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]': finished 2023-10-26T06:43:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:39 smithi192 ceph-mon[131661]: osdmap e69: 11 total, 10 up, 11 in 2023-10-26T06:43:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:39 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:39.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:39 smithi192 ceph-mon[131661]: from='client.? 172.21.15.135:0/168513564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:43:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:39 smithi135 ceph-mon[131403]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:39 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/1961879002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]: dispatch 2023-10-26T06:43:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:39 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]: dispatch 2023-10-26T06:43:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:39 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]': finished 2023-10-26T06:43:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:39 smithi135 ceph-mon[131403]: osdmap e69: 11 total, 10 up, 11 in 2023-10-26T06:43:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:39 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:39.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:39 smithi135 ceph-mon[131403]: from='client.? 172.21.15.135:0/168513564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:43:39.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:39 smithi089 ceph-mon[95031]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:39.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:39 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/1961879002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]: dispatch 2023-10-26T06:43:39.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:39 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]: dispatch 2023-10-26T06:43:39.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:39 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd567e81-805b-489f-8005-fb636fa97372"}]': finished 2023-10-26T06:43:39.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:39 smithi089 ceph-mon[95031]: osdmap e69: 11 total, 10 up, 11 in 2023-10-26T06:43:39.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:39 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:39.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:39 smithi089 ceph-mon[95031]: from='client.? 172.21.15.135:0/168513564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:43:41.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:41 smithi192 ceph-mon[131661]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:41 smithi135 ceph-mon[131403]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:41.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:41 smithi089 ceph-mon[95031]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:43 smithi192 ceph-mon[131661]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:43 smithi135 ceph-mon[131403]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:43.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:43 smithi089 ceph-mon[95031]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:45.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:45 smithi135 ceph-mon[131403]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:45.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:45 smithi192 ceph-mon[131661]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:45.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:45 smithi089 ceph-mon[95031]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:46.192 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:46 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-26T06:43:46.192 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:46 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:46.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:46 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-26T06:43:46.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:46 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:46.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:46 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-26T06:43:46.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:46 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:47.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:47 smithi192 ceph-mon[131661]: Deploying daemon osd.10 on smithi135 2023-10-26T06:43:47.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:47 smithi192 ceph-mon[131661]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:47.465 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:47 smithi135 ceph-mon[131403]: Deploying daemon osd.10 on smithi135 2023-10-26T06:43:47.466 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:47 smithi135 ceph-mon[131403]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:47 smithi089 ceph-mon[95031]: Deploying daemon osd.10 on smithi135 2023-10-26T06:43:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:47 smithi089 ceph-mon[95031]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:49.338 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:49 smithi135 ceph-mon[131403]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:49.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:49 smithi192 ceph-mon[131661]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:49 smithi089 ceph-mon[95031]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:50.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:50 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:50.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:50 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:50.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:50 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:50.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:50 smithi135 ceph-mon[131403]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:50.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:50 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:50.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:50 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:50 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:50 smithi192 ceph-mon[131661]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:51.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:50 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:51.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:50 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:51.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:50 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:51.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:50 smithi089 ceph-mon[95031]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:52.286 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.285+0000 7fe2fb7fe700 1 -- 172.21.15.135:0/3747956386 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe30c19a090 con 0x7fe2f40a14d0 2023-10-26T06:43:52.286 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 10 on host 'smithi135' 2023-10-26T06:43:52.288 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.287+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe2f40a14d0 msgr2=0x7fe2f40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:52.288 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.287+0000 7fe310c99700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe2f40a14d0 0x7fe2f40a3990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc005fd0 tx=0x7fe2fc005e30 comp rx=0 tx=0).stop 2023-10-26T06:43:52.289 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.287+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 msgr2=0x7fe30c198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:52.289 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.287+0000 7fe310c99700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c198d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe30000f880 comp rx=0 tx=0).stop 2023-10-26T06:43:52.289 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 shutdown_connections 2023-10-26T06:43:52.289 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe2f40a14d0 0x7fe2f40a3990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:52.289 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe30c10d300 0x7fe30c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:52.289 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe30c068190 0x7fe30c198d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:52.290 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 --2- 172.21.15.135:0/3747956386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe30c068ab0 0x7fe30c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:52.290 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 >> 172.21.15.135:0/3747956386 conn(0x7fe30c075790 msgr2=0x7fe30c10e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:43:52.290 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 shutdown_connections 2023-10-26T06:43:52.290 INFO:teuthology.orchestra.run.smithi135.stderr:2023-10-26T06:43:52.288+0000 7fe310c99700 1 -- 172.21.15.135:0/3747956386 wait complete. 2023-10-26T06:43:52.754 DEBUG:teuthology.orchestra.run.smithi135:osd.10> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.10.service 2023-10-26T06:43:52.756 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2023-10-26T06:43:52.757 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-26T06:43:52.788 INFO:journalctl@ceph.osd.10.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:43:53.329 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:43:53.537 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:53 smithi192 ceph-mon[131661]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:53.537 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.537 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.537 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.538 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:53 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:53 smithi089 ceph-mon[95031]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:53 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.552 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:53 smithi135 ceph-mon[131403]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:53.553 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.553 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.553 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.553 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:53 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:53.880 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:43:53 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:43:53.550+0000 7f4c2d42c640 -1 osd.10 0 log_to_monitors true 2023-10-26T06:43:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:54 smithi135 ceph-mon[131403]: from='osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-26T06:43:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:54 smithi135 ceph-mon[131403]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-26T06:43:54.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:54 smithi089 ceph-mon[95031]: from='osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-26T06:43:54.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:54 smithi089 ceph-mon[95031]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-26T06:43:54.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:54 smithi192 ceph-mon[131661]: from='osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-26T06:43:54.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:54 smithi192 ceph-mon[131661]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-26T06:43:54.890 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-26T06:43:54.890 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-26T06:43:54.890 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2023-10-26T06:43:54.890 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2023-10-26T06:43:54.890 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110424 s, 950 MB/s 2023-10-26T06:43:54.891 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2023-10-26T06:43:55.293 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2023-10-26T06:43:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:55 smithi089 ceph-mon[95031]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:55 smithi089 ceph-mon[95031]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-26T06:43:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:55 smithi089 ceph-mon[95031]: osdmap e70: 11 total, 10 up, 11 in 2023-10-26T06:43:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:55 smithi089 ceph-mon[95031]: from='osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:43:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:55 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:55 smithi089 ceph-mon[95031]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:43:55.568 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:43:55.278+0000 7f4c222a2700 -1 osd.10 0 waiting for initial osdmap 2023-10-26T06:43:55.569 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:43:55.301+0000 7f4c1c894700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:43:55.569 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-mon[131403]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:55.569 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-mon[131403]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-26T06:43:55.569 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-mon[131403]: osdmap e70: 11 total, 10 up, 11 in 2023-10-26T06:43:55.569 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-mon[131403]: from='osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:43:55.570 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:55.570 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:55 smithi135 ceph-mon[131403]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:43:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:55 smithi192 ceph-mon[131661]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:55 smithi192 ceph-mon[131661]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-26T06:43:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:55 smithi192 ceph-mon[131661]: osdmap e70: 11 total, 10 up, 11 in 2023-10-26T06:43:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:55 smithi192 ceph-mon[131661]: from='osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:43:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:55 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:55.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:55 smithi192 ceph-mon[131661]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2023-10-26T06:43:55.851 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:43:56.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:43:56.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: osdmap e71: 11 total, 10 up, 11 in 2023-10-26T06:43:56.560 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.561 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:56.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:56.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:56.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:56.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:56.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:56 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:43:56.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: osdmap e71: 11 total, 10 up, 11 in 2023-10-26T06:43:56.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:56.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:56.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:56.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:56.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:56.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:56 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2023-10-26T06:43:56.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: osdmap e71: 11 total, 10 up, 11 in 2023-10-26T06:43:56.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:56.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:43:56.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:56.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:56.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:43:56.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:56.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:43:56.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:43:56.794 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:56 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:57.063 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 -- 172.21.15.192:0/349924706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 msgr2=0x7f7e1010c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:57.063 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 --2- 172.21.15.192:0/349924706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e1010c660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b0a0 tx=0x7f7e0800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:43:57.064 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 -- 172.21.15.192:0/349924706 shutdown_connections 2023-10-26T06:43:57.064 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 --2- 172.21.15.192:0/349924706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e1010c660 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:57.064 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 --2- 172.21.15.192:0/349924706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e101004c0 0x7f7e101008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:57.064 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 --2- 172.21.15.192:0/349924706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e10100e70 0x7f7e101043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:57.064 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 -- 172.21.15.192:0/349924706 >> 172.21.15.192:0/349924706 conn(0x7f7e10074aa0 msgr2=0x7f7e10074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:43:57.065 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 -- 172.21.15.192:0/349924706 shutdown_connections 2023-10-26T06:43:57.065 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 -- 172.21.15.192:0/349924706 wait complete. 2023-10-26T06:43:57.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.062+0000 7f7e15826700 1 Processor -- start 2023-10-26T06:43:57.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 -- start start 2023-10-26T06:43:57.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e101004c0 0x7f7e10197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:57.068 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e10100e70 0x7f7e10198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:57.069 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e101a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:57.069 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e10198870 con 0x7f7e101004c0 2023-10-26T06:43:57.069 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e101a30a0 con 0x7f7e10100e70 2023-10-26T06:43:57.069 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e101a31e0 con 0x7f7e10104ae0 2023-10-26T06:43:57.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e101a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:57.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e101a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54316/0 (socket says 172.21.15.192:54316) 2023-10-26T06:43:57.070 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 -- 172.21.15.192:0/1267473078 learned_addr learned my addr 172.21.15.192:0/1267473078 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-10-26T06:43:57.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0e7fc700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e10100e70 0x7f7e10198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:57.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 -- 172.21.15.192:0/1267473078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e10100e70 msgr2=0x7f7e10198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:57.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e10100e70 0x7f7e10198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:57.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 -- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e101004c0 msgr2=0x7f7e10197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:43:57.071 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e101004c0 0x7f7e10197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:43:57.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 -- 172.21.15.192:0/1267473078 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08009d00 con 0x7f7e10104ae0 2023-10-26T06:43:57.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0e7fc700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e10100e70 0x7f7e10198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:43:57.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e0f7fe700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e101a2b60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e08007be0 tx=0x7f7e08005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:43:57.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e14824700 1 -- 172.21.15.192:0/1267473078 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800e040 con 0x7f7e10104ae0 2023-10-26T06:43:57.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e101a33e0 con 0x7f7e10104ae0 2023-10-26T06:43:57.072 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e14824700 1 -- 172.21.15.192:0/1267473078 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e08012e40 con 0x7f7e10104ae0 2023-10-26T06:43:57.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.063+0000 7f7e14824700 1 -- 172.21.15.192:0/1267473078 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0801b450 con 0x7f7e10104ae0 2023-10-26T06:43:57.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.064+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e101a38a0 con 0x7f7e10104ae0 2023-10-26T06:43:57.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.065+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e10107d40 con 0x7f7e10104ae0 2023-10-26T06:43:57.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.065+0000 7f7e14824700 1 -- 172.21.15.192:0/1267473078 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7e08012980 con 0x7f7e10104ae0 2023-10-26T06:43:57.073 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.066+0000 7f7e14824700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7df80a14d0 0x7f7df80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:43:57.074 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.066+0000 7f7e14824700 1 -- 172.21.15.192:0/1267473078 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6942+0+0 (secure 0 0 0) 0x7f7e080ce610 con 0x7f7e10104ae0 2023-10-26T06:43:57.074 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.066+0000 7f7e0effd700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7df80a14d0 0x7f7df80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:43:57.074 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.067+0000 7f7e0effd700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7df80a14d0 0x7f7df80a3990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7e00005950 tx=0x7f7e000058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:43:57.075 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.070+0000 7f7e14824700 1 -- 172.21.15.192:0/1267473078 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e08098260 con 0x7f7e10104ae0 2023-10-26T06:43:57.272 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:43:57.270+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e100ff6f0 con 0x7f7df80a14d0 2023-10-26T06:43:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:43:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:43:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi135 2023-10-26T06:43:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi135 to 4292M 2023-10-26T06:43:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] boot 2023-10-26T06:43:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: osdmap e72: 11 total, 11 up, 11 in 2023-10-26T06:43:57.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:57.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:57.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:57.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:57.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:43:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:43:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi135 2023-10-26T06:43:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi135 to 4292M 2023-10-26T06:43:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] boot 2023-10-26T06:43:57.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: osdmap e72: 11 total, 11 up, 11 in 2023-10-26T06:43:57.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:57.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:57.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:57.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:57.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:57.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:43:57.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:43:57.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-26T06:43:57.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi135 2023-10-26T06:43:57.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi135 to 4292M 2023-10-26T06:43:57.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: osd.10 [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] boot 2023-10-26T06:43:57.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: osdmap e72: 11 total, 11 up, 11 in 2023-10-26T06:43:57.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-26T06:43:57.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:43:57.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-26T06:43:57.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-26T06:43:57.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:43:58.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:58 smithi192 ceph-mon[131661]: osdmap e73: 11 total, 11 up, 11 in 2023-10-26T06:43:58.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:58 smithi192 ceph-mon[131661]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:43:58.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:58 smithi135 ceph-mon[131403]: osdmap e73: 11 total, 11 up, 11 in 2023-10-26T06:43:58.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:58 smithi135 ceph-mon[131403]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:43:58.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:58 smithi089 ceph-mon[95031]: osdmap e73: 11 total, 11 up, 11 in 2023-10-26T06:43:58.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:58 smithi089 ceph-mon[95031]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:43:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:43:59 smithi192 ceph-mon[131661]: osdmap e74: 11 total, 11 up, 11 in 2023-10-26T06:43:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:43:59 smithi135 ceph-mon[131403]: osdmap e74: 11 total, 11 up, 11 in 2023-10-26T06:43:59.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:43:59 smithi089 ceph-mon[95031]: osdmap e74: 11 total, 11 up, 11 in 2023-10-26T06:44:00.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:00 smithi192 ceph-mon[131661]: osdmap e75: 11 total, 11 up, 11 in 2023-10-26T06:44:00.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:00 smithi192 ceph-mon[131661]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:00.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:00 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/2885180656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]: dispatch 2023-10-26T06:44:00.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:00 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]: dispatch 2023-10-26T06:44:00.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:00 smithi192 ceph-mon[131661]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]': finished 2023-10-26T06:44:00.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:00 smithi192 ceph-mon[131661]: osdmap e76: 12 total, 11 up, 12 in 2023-10-26T06:44:00.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:00.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:00 smithi135 ceph-mon[131403]: osdmap e75: 11 total, 11 up, 11 in 2023-10-26T06:44:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:00 smithi135 ceph-mon[131403]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:00 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/2885180656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]: dispatch 2023-10-26T06:44:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:00 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]: dispatch 2023-10-26T06:44:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:00 smithi135 ceph-mon[131403]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]': finished 2023-10-26T06:44:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:00 smithi135 ceph-mon[131403]: osdmap e76: 12 total, 11 up, 12 in 2023-10-26T06:44:00.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:00.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:00 smithi089 ceph-mon[95031]: osdmap e75: 11 total, 11 up, 11 in 2023-10-26T06:44:00.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:00 smithi089 ceph-mon[95031]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:00.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:00 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/2885180656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]: dispatch 2023-10-26T06:44:00.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:00 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]: dispatch 2023-10-26T06:44:00.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:00 smithi089 ceph-mon[95031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1551f98d-9cc1-45b1-9b7c-5a5b2f814669"}]': finished 2023-10-26T06:44:00.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:00 smithi089 ceph-mon[95031]: osdmap e76: 12 total, 11 up, 12 in 2023-10-26T06:44:00.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:01 smithi192 ceph-mon[131661]: from='client.? 172.21.15.192:0/4110616472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:44:01.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:01 smithi135 ceph-mon[131403]: from='client.? 172.21.15.192:0/4110616472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:44:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:01 smithi089 ceph-mon[95031]: from='client.? 172.21.15.192:0/4110616472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-26T06:44:02.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:02 smithi192 ceph-mon[131661]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:02.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:02 smithi135 ceph-mon[131403]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:02.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:02 smithi089 ceph-mon[95031]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:05.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:04 smithi089 ceph-mon[95031]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-26T06:44:05.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:04 smithi192 ceph-mon[131661]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-26T06:44:05.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:04 smithi135 ceph-mon[131403]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-26T06:44:07.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:07 smithi089 ceph-mon[95031]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-26T06:44:07.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-26T06:44:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:07.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:07 smithi192 ceph-mon[131661]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-26T06:44:07.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-26T06:44:07.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:07.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:07 smithi135 ceph-mon[131403]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-26T06:44:07.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-26T06:44:07.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:08.119 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:08 smithi192 ceph-mon[131661]: Deploying daemon osd.11 on smithi192 2023-10-26T06:44:08.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:08 smithi089 ceph-mon[95031]: Deploying daemon osd.11 on smithi192 2023-10-26T06:44:08.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:08 smithi135 ceph-mon[131403]: Deploying daemon osd.11 on smithi192 2023-10-26T06:44:09.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:09 smithi089 ceph-mon[95031]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-26T06:44:09.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:09 smithi192 ceph-mon[131661]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-26T06:44:09.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:09 smithi135 ceph-mon[131403]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-26T06:44:11.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:11 smithi192 ceph-mon[131661]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-26T06:44:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:11 smithi135 ceph-mon[131403]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-26T06:44:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:11 smithi089 ceph-mon[95031]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-26T06:44:12.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:12.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:12.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:12.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:12.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:12.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:12.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:12.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:12.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:13.354 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:13 smithi192 ceph-mon[131661]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-26T06:44:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:13 smithi135 ceph-mon[131403]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-26T06:44:13.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:13 smithi089 ceph-mon[95031]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-26T06:44:13.786 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2023-10-26T06:44:13.787 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.784+0000 7f7e14824700 1 -- 172.21.15.192:0/1267473078 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7e100ff6f0 con 0x7f7df80a14d0 2023-10-26T06:44:13.788 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7df80a14d0 msgr2=0x7f7df80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:13.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7df80a14d0 0x7f7df80a3990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7e00005950 tx=0x7f7e000058e0 comp rx=0 tx=0).stop 2023-10-26T06:44:13.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 msgr2=0x7f7e101a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:13.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e101a2b60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e08007be0 tx=0x7f7e08005a60 comp rx=0 tx=0).stop 2023-10-26T06:44:13.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 shutdown_connections 2023-10-26T06:44:13.789 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7df80a14d0 0x7f7df80a3990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:13.790 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e101004c0 0x7f7e10197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:13.790 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e10104ae0 0x7f7e101a2b60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:13.790 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 --2- 172.21.15.192:0/1267473078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e10100e70 0x7f7e10198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:13.790 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.786+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 >> 172.21.15.192:0/1267473078 conn(0x7f7e10074aa0 msgr2=0x7f7e10105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:13.790 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.787+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 shutdown_connections 2023-10-26T06:44:13.791 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:13.787+0000 7f7e15826700 1 -- 172.21.15.192:0/1267473078 wait complete. 2023-10-26T06:44:14.264 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.11.service 2023-10-26T06:44:14.267 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-26T06:44:14.267 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd stat -f json 2023-10-26T06:44:14.300 INFO:journalctl@ceph.osd.11.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:44:14.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:14 smithi192 ceph-mon[131661]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-26T06:44:14.991 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:14.992 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:14.992 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:14.992 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:14 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:44:15.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:14 smithi089 ceph-mon[95031]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-26T06:44:15.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:14 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:14 smithi135 ceph-mon[131403]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-26T06:44:15.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:14 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:15.257 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:44:14 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:44:14.985+0000 7fddd3ac4640 -1 osd.11 0 log_to_monitors true 2023-10-26T06:44:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:15 smithi089 ceph-mon[95031]: from='osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-26T06:44:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:15 smithi089 ceph-mon[95031]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-26T06:44:15.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:15 smithi192 ceph-mon[131661]: from='osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-26T06:44:15.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:15 smithi192 ceph-mon[131661]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-26T06:44:16.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:15 smithi135 ceph-mon[131403]: from='osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-26T06:44:16.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:15 smithi135 ceph-mon[131403]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-26T06:44:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 -- 172.21.15.89:0/3284368461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 msgr2=0x7f26ac10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/3284368461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 0x7f26ac10d650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009f90 tx=0x7f26a400b330 comp rx=0 tx=0).stop 2023-10-26T06:44:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 -- 172.21.15.89:0/3284368461 shutdown_connections 2023-10-26T06:44:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/3284368461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 0x7f26ac10d650 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/3284368461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1057c0 0x7f26ac105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/3284368461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 0x7f26ac100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 -- 172.21.15.89:0/3284368461 >> 172.21.15.89:0/3284368461 conn(0x7f26ac0779e0 msgr2=0x7f26ac077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 -- 172.21.15.89:0/3284368461 shutdown_connections 2023-10-26T06:44:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.425+0000 7f26b3ce6700 1 -- 172.21.15.89:0/3284368461 wait complete. 2023-10-26T06:44:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 Processor -- start 2023-10-26T06:44:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 -- start start 2023-10-26T06:44:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 0x7f26ac198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 0x7f26ac199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1057c0 0x7f26ac19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac19de40 con 0x7f26ac101260 2023-10-26T06:44:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac19dfb0 con 0x7f26ac1057c0 2023-10-26T06:44:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b3ce6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac19e170 con 0x7f26ac1008a0 2023-10-26T06:44:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.426+0000 7f26b1a82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 0x7f26ac198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1281700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 0x7f26ac199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 0x7f26ac198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:52210/0 (socket says 172.21.15.89:52210) 2023-10-26T06:44:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 -- 172.21.15.89:0/2656902566 learned_addr learned my addr 172.21.15.89:0/2656902566 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b2283700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1057c0 0x7f26ac19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 -- 172.21.15.89:0/2656902566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1057c0 msgr2=0x7f26ac19d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1057c0 0x7f26ac19d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 -- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 msgr2=0x7f26ac199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 0x7f26ac199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 -- 172.21.15.89:0/2656902566 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4009c40 con 0x7f26ac1008a0 2023-10-26T06:44:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1281700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 0x7f26ac199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b1a82700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 0x7f26ac198f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f26a800e9f0 tx=0x7f26a800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26a2ffd700 1 -- 172.21.15.89:0/2656902566 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8004220 con 0x7f26ac1008a0 2023-10-26T06:44:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26a2ffd700 1 -- 172.21.15.89:0/2656902566 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26a8004380 con 0x7f26ac1008a0 2023-10-26T06:44:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.427+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac19e450 con 0x7f26ac1008a0 2023-10-26T06:44:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.428+0000 7f26a2ffd700 1 -- 172.21.15.89:0/2656902566 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8003be0 con 0x7f26ac1008a0 2023-10-26T06:44:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.428+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac1a5cb0 con 0x7f26ac1008a0 2023-10-26T06:44:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.429+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac101f30 con 0x7f26ac1008a0 2023-10-26T06:44:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.430+0000 7f26a2ffd700 1 -- 172.21.15.89:0/2656902566 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f26a8003d40 con 0x7f26ac1008a0 2023-10-26T06:44:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.432+0000 7f26a2ffd700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f26980a1550 0x7f26980a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:16.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.432+0000 7f26a2ffd700 1 -- 172.21.15.89:0/2656902566 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7046+0+0 (secure 0 0 0) 0x7f26a8014070 con 0x7f26ac1008a0 2023-10-26T06:44:16.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.432+0000 7f26b1281700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f26980a1550 0x7f26980a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:16.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.432+0000 7f26b1281700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f26980a1550 0x7f26980a3a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f26ac19a4a0 tx=0x7f269c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:16.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.436+0000 7f26a2ffd700 1 -- 172.21.15.89:0/2656902566 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f26a8096360 con 0x7f26ac1008a0 2023-10-26T06:44:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.634+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f26ac04e4c0 con 0x7f26ac1008a0 2023-10-26T06:44:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.635+0000 7f26a2ffd700 1 -- 172.21.15.89:0/2656902566 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f26a8099980 con 0x7f26ac1008a0 2023-10-26T06:44:16.637 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f26980a1550 msgr2=0x7f26980a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f26980a1550 0x7f26980a3a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f26ac19a4a0 tx=0x7f269c009260 comp rx=0 tx=0).stop 2023-10-26T06:44:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 msgr2=0x7f26ac198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 0x7f26ac198f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f26a800e9f0 tx=0x7f26a800ed00 comp rx=0 tx=0).stop 2023-10-26T06:44:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 shutdown_connections 2023-10-26T06:44:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f26980a1550 0x7f26980a3a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1057c0 0x7f26ac19d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26ac1008a0 0x7f26ac198f40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 --2- 172.21.15.89:0/2656902566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26ac101260 0x7f26ac199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.637+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 >> 172.21.15.89:0/2656902566 conn(0x7f26ac0779e0 msgr2=0x7f26ac10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.638+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 shutdown_connections 2023-10-26T06:44:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:16.638+0000 7f26b3ce6700 1 -- 172.21.15.89:0/2656902566 wait complete. 2023-10-26T06:44:17.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:16 smithi089 ceph-mon[95031]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:17.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:16 smithi089 ceph-mon[95031]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-26T06:44:17.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:16 smithi089 ceph-mon[95031]: osdmap e77: 12 total, 11 up, 12 in 2023-10-26T06:44:17.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:17.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:16 smithi089 ceph-mon[95031]: from='osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:44:17.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:16 smithi089 ceph-mon[95031]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:44:17.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:16 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2656902566' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-26T06:44:17.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:44:16.766+0000 7fddc893a700 -1 osd.11 0 waiting for initial osdmap 2023-10-26T06:44:17.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:44:16.785+0000 7fddc0f28700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T06:44:17.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-mon[131661]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:17.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-mon[131661]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-26T06:44:17.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-mon[131661]: osdmap e77: 12 total, 11 up, 12 in 2023-10-26T06:44:17.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:17.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-mon[131661]: from='osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:44:17.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-mon[131661]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:44:17.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:16 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2656902566' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-26T06:44:17.205 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1698302636,"num_in_osds":12,"osd_in_since":1698302639,"num_remapped_pgs":0} 2023-10-26T06:44:17.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:16 smithi135 ceph-mon[131403]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:17.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:16 smithi135 ceph-mon[131403]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-26T06:44:17.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:16 smithi135 ceph-mon[131403]: osdmap e77: 12 total, 11 up, 12 in 2023-10-26T06:44:17.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:17.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:16 smithi135 ceph-mon[131403]: from='osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:44:17.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:16 smithi135 ceph-mon[131403]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2023-10-26T06:44:17.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:16 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2656902566' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-26T06:44:18.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:44:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: osdmap e78: 12 total, 11 up, 12 in 2023-10-26T06:44:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: Detected new or changed devices on smithi192 2023-10-26T06:44:18.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi192 to 4292M 2023-10-26T06:44:18.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:18.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:18.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:18.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:18.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:18.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:44:18.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: osdmap e78: 12 total, 11 up, 12 in 2023-10-26T06:44:18.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: Detected new or changed devices on smithi192 2023-10-26T06:44:18.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi192 to 4292M 2023-10-26T06:44:18.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:18.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:18.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:18.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:18.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:18.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.206 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd stat -f json 2023-10-26T06:44:18.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2023-10-26T06:44:18.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: osdmap e78: 12 total, 11 up, 12 in 2023-10-26T06:44:18.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: Detected new or changed devices on smithi192 2023-10-26T06:44:18.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:44:18.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi192 to 4292M 2023-10-26T06:44:18.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:18.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:18.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:18.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:18.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:18.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:18.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:44:19.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:18 smithi089 ceph-mon[95031]: purged_snaps scrub starts 2023-10-26T06:44:19.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:18 smithi089 ceph-mon[95031]: purged_snaps scrub ok 2023-10-26T06:44:19.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:18 smithi089 ceph-mon[95031]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:19.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:18 smithi089 ceph-mon[95031]: osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] boot 2023-10-26T06:44:19.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:18 smithi089 ceph-mon[95031]: osdmap e79: 12 total, 12 up, 12 in 2023-10-26T06:44:19.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:19.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:18 smithi192 ceph-mon[131661]: purged_snaps scrub starts 2023-10-26T06:44:19.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:18 smithi192 ceph-mon[131661]: purged_snaps scrub ok 2023-10-26T06:44:19.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:18 smithi192 ceph-mon[131661]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:19.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:18 smithi192 ceph-mon[131661]: osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] boot 2023-10-26T06:44:19.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:18 smithi192 ceph-mon[131661]: osdmap e79: 12 total, 12 up, 12 in 2023-10-26T06:44:19.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:19.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:18 smithi135 ceph-mon[131403]: purged_snaps scrub starts 2023-10-26T06:44:19.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:18 smithi135 ceph-mon[131403]: purged_snaps scrub ok 2023-10-26T06:44:19.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:18 smithi135 ceph-mon[131403]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-26T06:44:19.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:18 smithi135 ceph-mon[131403]: osd.11 [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] boot 2023-10-26T06:44:19.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:18 smithi135 ceph-mon[131403]: osdmap e79: 12 total, 12 up, 12 in 2023-10-26T06:44:19.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-26T06:44:20.135 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:19 smithi089 ceph-mon[95031]: osdmap e80: 12 total, 12 up, 12 in 2023-10-26T06:44:20.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:19 smithi192 ceph-mon[131661]: osdmap e80: 12 total, 12 up, 12 in 2023-10-26T06:44:20.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:19 smithi135 ceph-mon[131403]: osdmap e80: 12 total, 12 up, 12 in 2023-10-26T06:44:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 -- 172.21.15.89:0/1276053431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c101260 msgr2=0x7fb53c10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 --2- 172.21.15.89:0/1276053431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c101260 0x7fb53c10d650 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b0a0 tx=0x7fb53400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 -- 172.21.15.89:0/1276053431 shutdown_connections 2023-10-26T06:44:20.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 --2- 172.21.15.89:0/1276053431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c101260 0x7fb53c10d650 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 --2- 172.21.15.89:0/1276053431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb53c1057c0 0x7fb53c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 --2- 172.21.15.89:0/1276053431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb53c1008a0 0x7fb53c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 -- 172.21.15.89:0/1276053431 >> 172.21.15.89:0/1276053431 conn(0x7fb53c0779e0 msgr2=0x7fb53c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:20.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 -- 172.21.15.89:0/1276053431 shutdown_connections 2023-10-26T06:44:20.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.383+0000 7fb543a8b700 1 -- 172.21.15.89:0/1276053431 wait complete. 2023-10-26T06:44:20.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.384+0000 7fb543a8b700 1 Processor -- start 2023-10-26T06:44:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb543a8b700 1 -- start start 2023-10-26T06:44:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb543a8b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb53c1008a0 0x7fb53c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb543a8b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb53c101260 0x7fb53c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb543a8b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c1057c0 0x7fb53c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:20.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb543a8b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c19de40 con 0x7fb53c1057c0 2023-10-26T06:44:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb543a8b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c19dfb0 con 0x7fb53c1008a0 2023-10-26T06:44:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb543a8b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c19e170 con 0x7fb53c101260 2023-10-26T06:44:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb542028700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c1057c0 0x7fb53c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb542028700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c1057c0 0x7fb53c19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41046/0 (socket says 172.21.15.89:41046) 2023-10-26T06:44:20.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb542028700 1 -- 172.21.15.89:0/2488194007 learned_addr learned my addr 172.21.15.89:0/2488194007 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb541026700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb53c101260 0x7fb53c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.385+0000 7fb541827700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb53c1008a0 0x7fb53c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb542028700 1 -- 172.21.15.89:0/2488194007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb53c101260 msgr2=0x7fb53c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb542028700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb53c101260 0x7fb53c199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb542028700 1 -- 172.21.15.89:0/2488194007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb53c1008a0 msgr2=0x7fb53c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:20.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb542028700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb53c1008a0 0x7fb53c198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb542028700 1 -- 172.21.15.89:0/2488194007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534009d00 con 0x7fb53c1057c0 2023-10-26T06:44:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb542028700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c1057c0 0x7fb53c19d830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b070 tx=0x7fb5340078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb532ffd700 1 -- 172.21.15.89:0/2488194007 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400e040 con 0x7fb53c1057c0 2023-10-26T06:44:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb532ffd700 1 -- 172.21.15.89:0/2488194007 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb53401d470 con 0x7fb53c1057c0 2023-10-26T06:44:20.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c19e3f0 con 0x7fb53c1057c0 2023-10-26T06:44:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb532ffd700 1 -- 172.21.15.89:0/2488194007 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534005280 con 0x7fb53c1057c0 2023-10-26T06:44:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.386+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1a5cb0 con 0x7fb53c1057c0 2023-10-26T06:44:20.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.387+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c101f30 con 0x7fb53c1057c0 2023-10-26T06:44:20.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.389+0000 7fb532ffd700 1 -- 172.21.15.89:0/2488194007 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb53401d5e0 con 0x7fb53c1057c0 2023-10-26T06:44:20.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.391+0000 7fb532ffd700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5280a14d0 0x7fb5280a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.391+0000 7fb532ffd700 1 -- 172.21.15.89:0/2488194007 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fb5340cdd20 con 0x7fb53c1057c0 2023-10-26T06:44:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.391+0000 7fb541827700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5280a14d0 0x7fb5280a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.391+0000 7fb541827700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5280a14d0 0x7fb5280a3990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb53c101930 tx=0x7fb538009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:20.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.393+0000 7fb532ffd700 1 -- 172.21.15.89:0/2488194007 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb534097740 con 0x7fb53c1057c0 2023-10-26T06:44:20.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.591+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb53c04e4c0 con 0x7fb53c1057c0 2023-10-26T06:44:20.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.592+0000 7fb532ffd700 1 -- 172.21.15.89:0/2488194007 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7fb53409ad60 con 0x7fb53c1057c0 2023-10-26T06:44:20.593 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:20.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5280a14d0 msgr2=0x7fb5280a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:20.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5280a14d0 0x7fb5280a3990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb53c101930 tx=0x7fb538009260 comp rx=0 tx=0).stop 2023-10-26T06:44:20.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c1057c0 msgr2=0x7fb53c19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:20.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c1057c0 0x7fb53c19d830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b070 tx=0x7fb5340078c0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 shutdown_connections 2023-10-26T06:44:20.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb5280a14d0 0x7fb5280a3990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb53c1057c0 0x7fb53c19d830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb53c1008a0 0x7fb53c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 --2- 172.21.15.89:0/2488194007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb53c101260 0x7fb53c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 >> 172.21.15.89:0/2488194007 conn(0x7fb53c0779e0 msgr2=0x7fb53c10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 shutdown_connections 2023-10-26T06:44:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:20.594+0000 7fb543a8b700 1 -- 172.21.15.89:0/2488194007 wait complete. 2023-10-26T06:44:21.040 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1698302657,"num_in_osds":12,"osd_in_since":1698302639,"num_remapped_pgs":0} 2023-10-26T06:44:21.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd dump --format=json 2023-10-26T06:44:21.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:21 smithi089 ceph-mon[95031]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:21.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:21 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2488194007' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-26T06:44:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:21 smithi192 ceph-mon[131661]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:21 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2488194007' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-26T06:44:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:21 smithi135 ceph-mon[131403]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:21 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2488194007' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-26T06:44:21.777 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:44:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 -- 172.21.15.89:0/2995514862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 msgr2=0x7ff92810cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 --2- 172.21.15.89:0/2995514862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff92810cd40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff918009990 tx=0x7ff918009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 -- 172.21.15.89:0/2995514862 shutdown_connections 2023-10-26T06:44:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 --2- 172.21.15.89:0/2995514862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff92810d280 0x7ff92810f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 --2- 172.21.15.89:0/2995514862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff928072a20 0x7ff928072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 --2- 172.21.15.89:0/2995514862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff92810cd40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 -- 172.21.15.89:0/2995514862 >> 172.21.15.89:0/2995514862 conn(0x7ff9280778d0 msgr2=0x7ff928077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 -- 172.21.15.89:0/2995514862 shutdown_connections 2023-10-26T06:44:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.222+0000 7ff92fe81700 1 -- 172.21.15.89:0/2995514862 wait complete. 2023-10-26T06:44:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.223+0000 7ff92fe81700 1 Processor -- start 2023-10-26T06:44:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.223+0000 7ff92fe81700 1 -- start start 2023-10-26T06:44:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92fe81700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff928072a20 0x7ff928198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92fe81700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff928199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92fe81700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff92810d280 0x7ff92819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92fe81700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819dcf0 con 0x7ff928073340 2023-10-26T06:44:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92fe81700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819de60 con 0x7ff928072a20 2023-10-26T06:44:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92fe81700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819e020 con 0x7ff92810d280 2023-10-26T06:44:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff928199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff928199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41078/0 (socket says 172.21.15.89:41078) 2023-10-26T06:44:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 -- 172.21.15.89:0/4094060589 learned_addr learned my addr 172.21.15.89:0/4094060589 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92e41e700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff92810d280 0x7ff92819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92dc1d700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff928072a20 0x7ff928198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 -- 172.21.15.89:0/4094060589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff92810d280 msgr2=0x7ff92819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff92810d280 0x7ff92819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 -- 172.21.15.89:0/4094060589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff928072a20 msgr2=0x7ff928198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff928072a20 0x7ff928198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.224+0000 7ff92d41c700 1 -- 172.21.15.89:0/4094060589 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180095f0 con 0x7ff928073340 2023-10-26T06:44:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.225+0000 7ff92d41c700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff928199330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff918005770 tx=0x7ff9180057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.225+0000 7ff91effd700 1 -- 172.21.15.89:0/4094060589 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91801d070 con 0x7ff928073340 2023-10-26T06:44:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.225+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92819e2a0 con 0x7ff928073340 2023-10-26T06:44:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.225+0000 7ff91effd700 1 -- 172.21.15.89:0/4094060589 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff918021410 con 0x7ff928073340 2023-10-26T06:44:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.225+0000 7ff91effd700 1 -- 172.21.15.89:0/4094060589 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918017650 con 0x7ff928073340 2023-10-26T06:44:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.225+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9281a5b60 con 0x7ff928073340 2023-10-26T06:44:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.226+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff928108490 con 0x7ff928073340 2023-10-26T06:44:23.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.228+0000 7ff91effd700 1 -- 172.21.15.89:0/4094060589 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff9180217f0 con 0x7ff928073340 2023-10-26T06:44:23.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.229+0000 7ff91effd700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9140a1550 0x7ff9140a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:23.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.229+0000 7ff91effd700 1 -- 172.21.15.89:0/4094060589 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7ff9180cdbe0 con 0x7ff928073340 2023-10-26T06:44:23.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.230+0000 7ff92dc1d700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9140a1550 0x7ff9140a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:23.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.230+0000 7ff92dc1d700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9140a1550 0x7ff9140a3a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff924009c60 tx=0x7ff924009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:23.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.233+0000 7ff91effd700 1 -- 172.21.15.89:0/4094060589 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff918097600 con 0x7ff928073340 2023-10-26T06:44:23.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:23 smithi089 ceph-mon[95031]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:23 smithi192 ceph-mon[131661]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:23.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.432+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff92804e4c0 con 0x7ff928073340 2023-10-26T06:44:23.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.433+0000 7ff91effd700 1 -- 172.21.15.89:0/4094060589 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18710 (secure 0 0 0) 0x7ff91809ac20 con 0x7ff928073340 2023-10-26T06:44:23.436 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:23.436 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:44:18.778923+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:44:23.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9140a1550 msgr2=0x7ff9140a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:23.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9140a1550 0x7ff9140a3a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff924009c60 tx=0x7ff924009260 comp rx=0 tx=0).stop 2023-10-26T06:44:23.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 msgr2=0x7ff928199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:23.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff928199330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff918005770 tx=0x7ff9180057a0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 shutdown_connections 2023-10-26T06:44:23.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9140a1550 0x7ff9140a3a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff92810d280 0x7ff92819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff928072a20 0x7ff928198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 --2- 172.21.15.89:0/4094060589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff928073340 0x7ff928199330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:23.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 >> 172.21.15.89:0/4094060589 conn(0x7ff9280778d0 msgr2=0x7ff9280fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:23.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 shutdown_connections 2023-10-26T06:44:23.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:23.436+0000 7ff92fe81700 1 -- 172.21.15.89:0/4094060589 wait complete. 2023-10-26T06:44:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:23 smithi135 ceph-mon[131403]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:23.959 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:44:23.960 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-26T06:44:24.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:24 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4094060589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:44:24.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:24 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4094060589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:44:24.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:24 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4094060589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:44:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:44:25.242 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:25 smithi089 ceph-mon[95031]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:25 smithi192 ceph-mon[131661]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:25 smithi135 ceph-mon[131403]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 -- 172.21.15.89:0/278524551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4070101260 msgr2=0x7f407010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 --2- 172.21.15.89:0/278524551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4070101260 0x7f407010d650 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f406800b0a0 tx=0x7f406800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 -- 172.21.15.89:0/278524551 shutdown_connections 2023-10-26T06:44:26.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 --2- 172.21.15.89:0/278524551 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4070101260 0x7f407010d650 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 --2- 172.21.15.89:0/278524551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40701057c0 0x7f4070105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 --2- 172.21.15.89:0/278524551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40701008a0 0x7f4070100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 -- 172.21.15.89:0/278524551 >> 172.21.15.89:0/278524551 conn(0x7f40700779e0 msgr2=0x7f4070077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 -- 172.21.15.89:0/278524551 shutdown_connections 2023-10-26T06:44:26.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.083+0000 7f40773dc700 1 -- 172.21.15.89:0/278524551 wait complete. 2023-10-26T06:44:26.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.084+0000 7f40773dc700 1 Processor -- start 2023-10-26T06:44:26.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.084+0000 7f40773dc700 1 -- start start 2023-10-26T06:44:26.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f40773dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701008a0 0x7f4070072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:26.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f40773dc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4070101260 0x7f4070073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:26.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f40773dc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40701057c0 0x7f4070073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:26.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f40773dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701a5090 con 0x7f40701008a0 2023-10-26T06:44:26.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f40773dc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701a5200 con 0x7f40701057c0 2023-10-26T06:44:26.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f40773dc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40701a53c0 con 0x7f4070101260 2023-10-26T06:44:26.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f4075178700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701008a0 0x7f4070072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:26.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f4075178700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701008a0 0x7f4070072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41106/0 (socket says 172.21.15.89:41106) 2023-10-26T06:44:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f4075178700 1 -- 172.21.15.89:0/2881872903 learned_addr learned my addr 172.21.15.89:0/2881872903 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f4074977700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4070101260 0x7f4070073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.085+0000 7f4075979700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40701057c0 0x7f4070073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4075178700 1 -- 172.21.15.89:0/2881872903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4070101260 msgr2=0x7f4070073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4075178700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4070101260 0x7f4070073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4075178700 1 -- 172.21.15.89:0/2881872903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40701057c0 msgr2=0x7f4070073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4075178700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40701057c0 0x7f4070073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4075178700 1 -- 172.21.15.89:0/2881872903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068009d00 con 0x7f40701008a0 2023-10-26T06:44:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4074977700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4070101260 0x7f4070073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4075979700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40701057c0 0x7f4070073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:26.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f4075178700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701008a0 0x7f4070072af0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d980 tx=0x7f406c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f40667fc700 1 -- 172.21.15.89:0/2881872903 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c00f890 con 0x7f40701008a0 2023-10-26T06:44:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f40667fc700 1 -- 172.21.15.89:0/2881872903 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f406c010460 con 0x7f40701008a0 2023-10-26T06:44:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40701a56a0 con 0x7f40701008a0 2023-10-26T06:44:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f40667fc700 1 -- 172.21.15.89:0/2881872903 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c0195c0 con 0x7f40701008a0 2023-10-26T06:44:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.086+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40701a5bf0 con 0x7f40701008a0 2023-10-26T06:44:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.087+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4070108da0 con 0x7f40701008a0 2023-10-26T06:44:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.089+0000 7f40667fc700 1 -- 172.21.15.89:0/2881872903 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f406c0105d0 con 0x7f40701008a0 2023-10-26T06:44:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.090+0000 7f40667fc700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f405c0a1550 0x7f405c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.091+0000 7f40667fc700 1 -- 172.21.15.89:0/2881872903 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f406c0cc5e0 con 0x7f40701008a0 2023-10-26T06:44:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.091+0000 7f4074977700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f405c0a1550 0x7f405c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:26.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.091+0000 7f4074977700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f405c0a1550 0x7f405c0a3a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4070074220 tx=0x7f4060009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:26.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.095+0000 7f40667fc700 1 -- 172.21.15.89:0/2881872903 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f406c095f70 con 0x7f40701008a0 2023-10-26T06:44:26.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.290+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4070073fd0 con 0x7f40701008a0 2023-10-26T06:44:26.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.291+0000 7f40667fc700 1 -- 172.21.15.89:0/2881872903 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f406c099590 con 0x7f40701008a0 2023-10-26T06:44:26.294 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:44:26.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f405c0a1550 msgr2=0x7f405c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:26.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f405c0a1550 0x7f405c0a3a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4070074220 tx=0x7f4060009260 comp rx=0 tx=0).stop 2023-10-26T06:44:26.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701008a0 msgr2=0x7f4070072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:26.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701008a0 0x7f4070072af0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d980 tx=0x7f406c00dc90 comp rx=0 tx=0).stop 2023-10-26T06:44:26.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 shutdown_connections 2023-10-26T06:44:26.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f405c0a1550 0x7f405c0a3a10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701008a0 0x7f4070072af0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40701057c0 0x7f4070073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 --2- 172.21.15.89:0/2881872903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4070101260 0x7f4070073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:26.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 >> 172.21.15.89:0/2881872903 conn(0x7f40700779e0 msgr2=0x7f40700fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:26.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 shutdown_connections 2023-10-26T06:44:26.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:26.293+0000 7f40773dc700 1 -- 172.21.15.89:0/2881872903 wait complete. 2023-10-26T06:44:26.778 INFO:tasks.cephadm:Adding mds.a on smithi089 2023-10-26T06:44:26.779 INFO:tasks.cephadm:Adding mds.d on smithi089 2023-10-26T06:44:26.779 INFO:tasks.cephadm:Adding mds.g on smithi089 2023-10-26T06:44:26.779 INFO:tasks.cephadm:Adding mds.j on smithi089 2023-10-26T06:44:26.779 INFO:tasks.cephadm:Adding mds.b on smithi135 2023-10-26T06:44:26.780 INFO:tasks.cephadm:Adding mds.e on smithi135 2023-10-26T06:44:26.780 INFO:tasks.cephadm:Adding mds.h on smithi135 2023-10-26T06:44:26.780 INFO:tasks.cephadm:Adding mds.k on smithi135 2023-10-26T06:44:26.780 INFO:tasks.cephadm:Adding mds.c on smithi192 2023-10-26T06:44:26.781 INFO:tasks.cephadm:Adding mds.f on smithi192 2023-10-26T06:44:26.781 INFO:tasks.cephadm:Adding mds.i on smithi192 2023-10-26T06:44:26.781 INFO:tasks.cephadm:Adding mds.l on smithi192 2023-10-26T06:44:26.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph orch apply mds all '12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2023-10-26T06:44:27.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:27 smithi192 ceph-mon[131661]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:27.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:27 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2881872903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:44:27.357 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:44:27.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:27 smithi135 ceph-mon[131403]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:27.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:27 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2881872903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:44:27.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:27 smithi089 ceph-mon[95031]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:27.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:27 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2881872903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:44:28.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 -- 172.21.15.192:0/3151796179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 msgr2=0x7f6ca80ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:28.540 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/3151796179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca80ff5f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.541 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 -- 172.21.15.192:0/3151796179 shutdown_connections 2023-10-26T06:44:28.541 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/3151796179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca81058d0 0x7f6ca8105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.541 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/3151796179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca80ffb30 0x7f6ca810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.541 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/3151796179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca80ff5f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.542 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 -- 172.21.15.192:0/3151796179 >> 172.21.15.192:0/3151796179 conn(0x7f6ca8074a10 msgr2=0x7f6ca8074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:28.542 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 -- 172.21.15.192:0/3151796179 shutdown_connections 2023-10-26T06:44:28.542 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.537+0000 7f6cad3d2700 1 -- 172.21.15.192:0/3151796179 wait complete. 2023-10-26T06:44:28.542 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 Processor -- start 2023-10-26T06:44:28.542 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 -- start start 2023-10-26T06:44:28.543 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:28.543 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca80ffb30 0x7f6ca819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:28.543 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca81058d0 0x7f6ca8196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:28.543 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81970f0 con 0x7f6ca80ffb30 2023-10-26T06:44:28.543 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8197260 con 0x7f6ca81058d0 2023-10-26T06:44:28.543 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6cad3d2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8197420 con 0x7f6ca80ff190 2023-10-26T06:44:28.544 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6ca6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:28.544 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6ca6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca819c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54438/0 (socket says 172.21.15.192:54438) 2023-10-26T06:44:28.544 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6ca6ffd700 1 -- 172.21.15.192:0/4278409746 learned_addr learned my addr 172.21.15.192:0/4278409746 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-10-26T06:44:28.544 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6ca77fe700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca81058d0 0x7f6ca8196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:28.544 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6ca67fc700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca80ffb30 0x7f6ca819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:28.545 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6ca6ffd700 1 -- 172.21.15.192:0/4278409746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca81058d0 msgr2=0x7f6ca8196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:28.545 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.538+0000 7f6ca6ffd700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca81058d0 0x7f6ca8196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.545 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6ca6ffd700 1 -- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca80ffb30 msgr2=0x7f6ca819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:28.545 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6ca6ffd700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca80ffb30 0x7f6ca819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.545 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6ca6ffd700 1 -- 172.21.15.192:0/4278409746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980095f0 con 0x7f6ca80ff190 2023-10-26T06:44:28.546 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6ca6ffd700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca819c940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00d980 tx=0x7f6c9c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:28.546 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6c8ffff700 1 -- 172.21.15.192:0/4278409746 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c004020 con 0x7f6ca80ff190 2023-10-26T06:44:28.546 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6c8ffff700 1 -- 172.21.15.192:0/4278409746 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9c009730 con 0x7f6ca80ff190 2023-10-26T06:44:28.546 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca8197700 con 0x7f6ca80ff190 2023-10-26T06:44:28.546 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6c8ffff700 1 -- 172.21.15.192:0/4278409746 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003a80 con 0x7f6ca80ff190 2023-10-26T06:44:28.547 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.539+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca81a3930 con 0x7f6ca80ff190 2023-10-26T06:44:28.548 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.540+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca8108d30 con 0x7f6ca80ff190 2023-10-26T06:44:28.548 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.541+0000 7f6c8ffff700 1 -- 172.21.15.192:0/4278409746 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6c9c0098a0 con 0x7f6ca80ff190 2023-10-26T06:44:28.548 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.541+0000 7f6c8ffff700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6c900a14d0 0x7f6c900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:28.548 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.541+0000 7f6c8ffff700 1 -- 172.21.15.192:0/4278409746 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f6c9c0cc630 con 0x7f6ca80ff190 2023-10-26T06:44:28.548 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.542+0000 7f6ca67fc700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6c900a14d0 0x7f6c900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:28.549 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.542+0000 7f6ca67fc700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6c900a14d0 0x7f6c900a3990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c9800b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:28.549 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.546+0000 7f6c8ffff700 1 -- 172.21.15.192:0/4278409746 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6c9c096060 con 0x7f6ca80ff190 2023-10-26T06:44:28.746 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.744+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ca8110720 con 0x7f6c900a14d0 2023-10-26T06:44:28.754 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.753+0000 7f6c8ffff700 1 -- 172.21.15.192:0/4278409746 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6ca8110720 con 0x7f6c900a14d0 2023-10-26T06:44:28.754 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2023-10-26T06:44:28.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6c900a14d0 msgr2=0x7f6c900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:28.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6c900a14d0 0x7f6c900a3990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006010 tx=0x7f6c9800b260 comp rx=0 tx=0).stop 2023-10-26T06:44:28.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 msgr2=0x7f6ca819c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:28.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca819c940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00d980 tx=0x7f6c9c00dc90 comp rx=0 tx=0).stop 2023-10-26T06:44:28.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 shutdown_connections 2023-10-26T06:44:28.758 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6c900a14d0 0x7f6c900a3990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.758 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ca81058d0 0x7f6ca8196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.758 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca80ff190 0x7f6ca819c940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.758 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 --2- 172.21.15.192:0/4278409746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca80ffb30 0x7f6ca819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:28.758 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.755+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 >> 172.21.15.192:0/4278409746 conn(0x7f6ca8074a10 msgr2=0x7f6ca80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:28.759 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.756+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 shutdown_connections 2023-10-26T06:44:28.759 INFO:teuthology.orchestra.run.smithi192.stderr:2023-10-26T06:44:28.756+0000 7f6cad3d2700 1 -- 172.21.15.192:0/4278409746 wait complete. 2023-10-26T06:44:29.214 DEBUG:teuthology.orchestra.run.smithi089:mds.a> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.a.service 2023-10-26T06:44:29.220 DEBUG:teuthology.orchestra.run.smithi089:mds.d> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.d.service 2023-10-26T06:44:29.266 DEBUG:teuthology.orchestra.run.smithi089:mds.g> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.g.service 2023-10-26T06:44:29.300 INFO:journalctl@ceph.mds.a.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:44:29.307 DEBUG:teuthology.orchestra.run.smithi089:mds.j> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.j.service 2023-10-26T06:44:29.327 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:29.327 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:29.327 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:29.327 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:29.327 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:29.328 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:29.328 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:29.328 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:29.328 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:29.331 DEBUG:teuthology.orchestra.run.smithi135:mds.b> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.b.service 2023-10-26T06:44:29.333 DEBUG:teuthology.orchestra.run.smithi135:mds.e> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.e.service 2023-10-26T06:44:29.341 INFO:journalctl@ceph.mds.d.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:44:29.363 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:29.363 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:29.363 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:29.364 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:29.364 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:29.364 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:29.364 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:29.364 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:29.365 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:29.366 DEBUG:teuthology.orchestra.run.smithi135:mds.h> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.h.service 2023-10-26T06:44:29.370 INFO:journalctl@ceph.mds.b.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:44:29.371 DEBUG:teuthology.orchestra.run.smithi135:mds.k> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.k.service 2023-10-26T06:44:29.374 INFO:journalctl@ceph.mds.g.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:44:29.394 INFO:journalctl@ceph.mds.e.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:44:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:44:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:44:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:29.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:29.397 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.c.service 2023-10-26T06:44:29.399 INFO:journalctl@ceph.mds.j.smithi089.stdout:-- Logs begin at Thu 2023-10-26 06:18:10 UTC. -- 2023-10-26T06:44:29.400 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.f.service 2023-10-26T06:44:29.401 INFO:journalctl@ceph.mds.h.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:44:29.423 INFO:journalctl@ceph.mds.k.smithi135.stdout:-- Logs begin at Thu 2023-10-26 06:14:27 UTC. -- 2023-10-26T06:44:29.428 INFO:journalctl@ceph.mds.c.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:44:29.429 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.i.service 2023-10-26T06:44:29.454 INFO:journalctl@ceph.mds.f.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:44:29.455 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.l.service 2023-10-26T06:44:29.480 INFO:journalctl@ceph.mds.i.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:44:29.480 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-26T06:44:29.481 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:44:29.509 INFO:journalctl@ceph.mds.l.smithi192.stdout:-- Logs begin at Thu 2023-10-26 06:14:20 UTC. -- 2023-10-26T06:44:29.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 -- 172.21.15.89:0/3141031492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b301006a0 msgr2=0x7f7b30100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:29.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 --2- 172.21.15.89:0/3141031492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b301006a0 0x7f7b30100b20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0098f0 tx=0x7f7b1c009c00 comp rx=0 tx=0).stop 2023-10-26T06:44:29.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 -- 172.21.15.89:0/3141031492 shutdown_connections 2023-10-26T06:44:29.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 --2- 172.21.15.89:0/3141031492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b30101060 0x7f7b3010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 --2- 172.21.15.89:0/3141031492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b301066b0 0x7f7b30106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 --2- 172.21.15.89:0/3141031492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b301006a0 0x7f7b30100b20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 -- 172.21.15.89:0/3141031492 >> 172.21.15.89:0/3141031492 conn(0x7f7b300fc170 msgr2=0x7f7b300fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:29.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 -- 172.21.15.89:0/3141031492 shutdown_connections 2023-10-26T06:44:29.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.683+0000 7f7b371ff700 1 -- 172.21.15.89:0/3141031492 wait complete. 2023-10-26T06:44:29.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.684+0000 7f7b371ff700 1 Processor -- start 2023-10-26T06:44:29.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.685+0000 7f7b371ff700 1 -- start start 2023-10-26T06:44:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.685+0000 7f7b371ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b301006a0 0x7f7b3019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.685+0000 7f7b371ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b30101060 0x7f7b3019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.685+0000 7f7b371ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b301066b0 0x7f7b30196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.685+0000 7f7b371ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30196fd0 con 0x7f7b30101060 2023-10-26T06:44:29.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.685+0000 7f7b371ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30197140 con 0x7f7b301066b0 2023-10-26T06:44:29.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.685+0000 7f7b371ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30197300 con 0x7f7b301006a0 2023-10-26T06:44:29.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b30101060 0x7f7b3019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:29.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b34f9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b301006a0 0x7f7b3019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:29.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b30101060 0x7f7b3019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41114/0 (socket says 172.21.15.89:41114) 2023-10-26T06:44:29.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 -- 172.21.15.89:0/436639487 learned_addr learned my addr 172.21.15.89:0/436639487 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:29.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b3579c700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b301066b0 0x7f7b30196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:29.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 -- 172.21.15.89:0/436639487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b301006a0 msgr2=0x7f7b3019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:29.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b301006a0 0x7f7b3019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 -- 172.21.15.89:0/436639487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b301066b0 msgr2=0x7f7b30196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:29.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b301066b0 0x7f7b30196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 -- 172.21.15.89:0/436639487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c009520 con 0x7f7b30101060 2023-10-26T06:44:29.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2ffff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b30101060 0x7f7b3019cd80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c005950 tx=0x7f7b1c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:29.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2dffb700 1 -- 172.21.15.89:0/436639487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c01c070 con 0x7f7b30101060 2023-10-26T06:44:29.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2dffb700 1 -- 172.21.15.89:0/436639487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b1c00baa0 con 0x7f7b30101060 2023-10-26T06:44:29.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b2dffb700 1 -- 172.21.15.89:0/436639487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0038e0 con 0x7f7b30101060 2023-10-26T06:44:29.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.686+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b30197580 con 0x7f7b30101060 2023-10-26T06:44:29.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.687+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b301a3830 con 0x7f7b30101060 2023-10-26T06:44:29.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.688+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b30108c30 con 0x7f7b30101060 2023-10-26T06:44:29.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.690+0000 7f7b2dffb700 1 -- 172.21.15.89:0/436639487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7b1c020030 con 0x7f7b30101060 2023-10-26T06:44:29.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.691+0000 7f7b2dffb700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7b200a1550 0x7f7b200a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:29.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.691+0000 7f7b2dffb700 1 -- 172.21.15.89:0/436639487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f7b1c0cd6a0 con 0x7f7b30101060 2023-10-26T06:44:29.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.691+0000 7f7b34f9b700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7b200a1550 0x7f7b200a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:29.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.692+0000 7f7b34f9b700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7b200a1550 0x7f7b200a3a10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009be0 tx=0x7f7b24009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:29.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.694+0000 7f7b2dffb700 1 -- 172.21.15.89:0/436639487 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7b1c0970c0 con 0x7f7b30101060 2023-10-26T06:44:29.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.898+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b30101de0 con 0x7f7b30101060 2023-10-26T06:44:29.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.899+0000 7f7b2dffb700 1 -- 172.21.15.89:0/436639487 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f7b1c09a6e0 con 0x7f7b30101060 2023-10-26T06:44:29.902 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:29.902 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:44:18.778923+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:44:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.901+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7b200a1550 msgr2=0x7f7b200a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.901+0000 7f7b371ff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7b200a1550 0x7f7b200a3a10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009be0 tx=0x7f7b24009260 comp rx=0 tx=0).stop 2023-10-26T06:44:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b30101060 msgr2=0x7f7b3019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b30101060 0x7f7b3019cd80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c005950 tx=0x7f7b1c00fa00 comp rx=0 tx=0).stop 2023-10-26T06:44:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 shutdown_connections 2023-10-26T06:44:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7b200a1550 0x7f7b200a3a10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b301066b0 0x7f7b30196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b301006a0 0x7f7b3019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 --2- 172.21.15.89:0/436639487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b30101060 0x7f7b3019cd80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:29.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 >> 172.21.15.89:0/436639487 conn(0x7f7b300fc170 msgr2=0x7f7b300fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:29.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 shutdown_connections 2023-10-26T06:44:29.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:29.902+0000 7f7b371ff700 1 -- 172.21.15.89:0/436639487 wait complete. 2023-10-26T06:44:29.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:44:29.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:44:30.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 -- 172.21.15.89:0/1709098274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc1010d0 msgr2=0x7f51cc10d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 --2- 172.21.15.89:0/1709098274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc1010d0 0x7f51cc10d5e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b0a0 tx=0x7f51c400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 -- 172.21.15.89:0/1709098274 shutdown_connections 2023-10-26T06:44:30.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 --2- 172.21.15.89:0/1709098274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc1010d0 0x7f51cc10d5e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 --2- 172.21.15.89:0/1709098274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51cc106720 0x7f51cc106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 --2- 172.21.15.89:0/1709098274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51cc100710 0x7f51cc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 -- 172.21.15.89:0/1709098274 >> 172.21.15.89:0/1709098274 conn(0x7f51cc0fc200 msgr2=0x7f51cc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 -- 172.21.15.89:0/1709098274 shutdown_connections 2023-10-26T06:44:30.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.126+0000 7f51d11ff700 1 -- 172.21.15.89:0/1709098274 wait complete. 2023-10-26T06:44:30.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.127+0000 7f51d11ff700 1 Processor -- start 2023-10-26T06:44:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51d11ff700 1 -- start start 2023-10-26T06:44:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51d11ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc100710 0x7f51cc198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51d11ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51cc1010d0 0x7f51cc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51d11ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51cc106720 0x7f51cc19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51d11ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc19cb50 con 0x7f51cc100710 2023-10-26T06:44:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51d11ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc19ccc0 con 0x7f51cc106720 2023-10-26T06:44:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51d11ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc19ce80 con 0x7f51cc1010d0 2023-10-26T06:44:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51cad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc100710 0x7f51cc198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51cad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc100710 0x7f51cc198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45206/0 (socket says 172.21.15.89:45206) 2023-10-26T06:44:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51cad9d700 1 -- 172.21.15.89:0/3553366786 learned_addr learned my addr 172.21.15.89:0/3553366786 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:30.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51ca59c700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51cc1010d0 0x7f51cc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.128+0000 7f51cb59e700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51cc106720 0x7f51cc19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51cad9d700 1 -- 172.21.15.89:0/3553366786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51cc1010d0 msgr2=0x7f51cc199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51cad9d700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51cc1010d0 0x7f51cc199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51cad9d700 1 -- 172.21.15.89:0/3553366786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51cc106720 msgr2=0x7f51cc19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51cad9d700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51cc106720 0x7f51cc19c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51cad9d700 1 -- 172.21.15.89:0/3553366786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c4009d00 con 0x7f51cc100710 2023-10-26T06:44:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51cad9d700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc100710 0x7f51cc198d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00e9a0 tx=0x7f51bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51b3fff700 1 -- 172.21.15.89:0/3553366786 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc009790 con 0x7f51cc100710 2023-10-26T06:44:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51b3fff700 1 -- 172.21.15.89:0/3553366786 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51bc00cbd0 con 0x7f51cc100710 2023-10-26T06:44:30.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc19d160 con 0x7f51cc100710 2023-10-26T06:44:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51b3fff700 1 -- 172.21.15.89:0/3553366786 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc018680 con 0x7f51cc100710 2023-10-26T06:44:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.129+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc1a5ac0 con 0x7f51cc100710 2023-10-26T06:44:30.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.130+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc101e50 con 0x7f51cc100710 2023-10-26T06:44:30.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.132+0000 7f51b3fff700 1 -- 172.21.15.89:0/3553366786 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f51bc00cd40 con 0x7f51cc100710 2023-10-26T06:44:30.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.134+0000 7f51b3fff700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51b40a1550 0x7f51b40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.134+0000 7f51b3fff700 1 -- 172.21.15.89:0/3553366786 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f51bc014070 con 0x7f51cc100710 2023-10-26T06:44:30.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.134+0000 7f51ca59c700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51b40a1550 0x7f51b40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.134+0000 7f51ca59c700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51b40a1550 0x7f51b40a3a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f51cc19a230 tx=0x7f51c0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:30.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.136+0000 7f51b3fff700 1 -- 172.21.15.89:0/3553366786 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f51bc096ae0 con 0x7f51cc100710 2023-10-26T06:44:30.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:30 smithi089 ceph-mon[95031]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:44:30.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:30 smithi089 ceph-mon[95031]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-10-26T06:44:30.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:30 smithi089 ceph-mon[95031]: Deploying daemon mds.l on smithi192 2023-10-26T06:44:30.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:30 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/436639487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:44:30.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.334+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51cc0675a0 con 0x7f51cc100710 2023-10-26T06:44:30.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.335+0000 7f51b3fff700 1 -- 172.21.15.89:0/3553366786 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51bc09a100 con 0x7f51cc100710 2023-10-26T06:44:30.337 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:44:30.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51b40a1550 msgr2=0x7f51b40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51b40a1550 0x7f51b40a3a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f51cc19a230 tx=0x7f51c0009260 comp rx=0 tx=0).stop 2023-10-26T06:44:30.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc100710 msgr2=0x7f51cc198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc100710 0x7f51cc198d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00e9a0 tx=0x7f51bc00ed60 comp rx=0 tx=0).stop 2023-10-26T06:44:30.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 shutdown_connections 2023-10-26T06:44:30.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51b40a1550 0x7f51b40a3a10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51cc100710 0x7f51cc198d20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51cc106720 0x7f51cc19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 --2- 172.21.15.89:0/3553366786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51cc1010d0 0x7f51cc199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 >> 172.21.15.89:0/3553366786 conn(0x7f51cc0fc200 msgr2=0x7f51cc1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:30.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.337+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 shutdown_connections 2023-10-26T06:44:30.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.338+0000 7f51d11ff700 1 -- 172.21.15.89:0/3553366786 wait complete. 2023-10-26T06:44:30.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:30.393 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:30 smithi192 ceph-mon[131661]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:44:30.393 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:30 smithi192 ceph-mon[131661]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-10-26T06:44:30.393 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:30 smithi192 ceph-mon[131661]: Deploying daemon mds.l on smithi192 2023-10-26T06:44:30.393 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:30 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/436639487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:44:30.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:30 smithi135 ceph-mon[131403]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:44:30.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:30 smithi135 ceph-mon[131403]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-10-26T06:44:30.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:30 smithi135 ceph-mon[131403]: Deploying daemon mds.l on smithi192 2023-10-26T06:44:30.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:30 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/436639487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:44:30.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 -- 172.21.15.89:0/4242688269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41006a0 msgr2=0x7f32f4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/4242688269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41006a0 0x7f32f4100b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009990 tx=0x7f32e4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 -- 172.21.15.89:0/4242688269 shutdown_connections 2023-10-26T06:44:30.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/4242688269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f4101060 0x7f32f410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/4242688269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f41066b0 0x7f32f4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/4242688269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41006a0 0x7f32f4100b20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 -- 172.21.15.89:0/4242688269 >> 172.21.15.89:0/4242688269 conn(0x7f32f40fc170 msgr2=0x7f32f40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:30.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 -- 172.21.15.89:0/4242688269 shutdown_connections 2023-10-26T06:44:30.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.556+0000 7f32fc5ff700 1 -- 172.21.15.89:0/4242688269 wait complete. 2023-10-26T06:44:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 Processor -- start 2023-10-26T06:44:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 -- start start 2023-10-26T06:44:30.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f41006a0 0x7f32f419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f4101060 0x7f32f419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41066b0 0x7f32f4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4199190 con 0x7f32f41066b0 2023-10-26T06:44:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4199300 con 0x7f32f41006a0 2023-10-26T06:44:30.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.557+0000 7f32fc5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41994c0 con 0x7f32f4101060 2023-10-26T06:44:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32fab9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41066b0 0x7f32f4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f4101060 0x7f32f419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32fa39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f41006a0 0x7f32f419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f4101060 0x7f32f419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:59758/0 (socket says 172.21.15.89:59758) 2023-10-26T06:44:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 -- 172.21.15.89:0/3110440044 learned_addr learned my addr 172.21.15.89:0/3110440044 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:30.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32fa39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f41006a0 0x7f32f419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:57934/0 (socket says 172.21.15.89:57934) 2023-10-26T06:44:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 -- 172.21.15.89:0/3110440044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f41006a0 msgr2=0x7f32f419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f41006a0 0x7f32f419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 -- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41066b0 msgr2=0x7f32f4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41066b0 0x7f32f4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 -- 172.21.15.89:0/3110440044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40095f0 con 0x7f32f4101060 2023-10-26T06:44:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32fab9c700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41066b0 0x7f32f4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32f9b9a700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f4101060 0x7f32f419ef40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005910 tx=0x7f32e4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.558+0000 7f32eb7fe700 1 -- 172.21.15.89:0/3110440044 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e401d070 con 0x7f32f4101060 2023-10-26T06:44:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.559+0000 7f32eb7fe700 1 -- 172.21.15.89:0/3110440044 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32e4022470 con 0x7f32f4101060 2023-10-26T06:44:30.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.559+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f4199740 con 0x7f32f4101060 2023-10-26T06:44:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.559+0000 7f32eb7fe700 1 -- 172.21.15.89:0/3110440044 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4017650 con 0x7f32f4101060 2023-10-26T06:44:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.559+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f41a59f0 con 0x7f32f4101060 2023-10-26T06:44:30.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.559+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f4101de0 con 0x7f32f4101060 2023-10-26T06:44:30.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.561+0000 7f32eb7fe700 1 -- 172.21.15.89:0/3110440044 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f32e40225e0 con 0x7f32f4101060 2023-10-26T06:44:30.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.563+0000 7f32eb7fe700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32e00a14d0 0x7f32e00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:30.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.563+0000 7f32eb7fe700 1 -- 172.21.15.89:0/3110440044 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f32e40cdc10 con 0x7f32f4101060 2023-10-26T06:44:30.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.563+0000 7f32fa39b700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32e00a14d0 0x7f32e00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:30.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.563+0000 7f32fa39b700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32e00a14d0 0x7f32e00a3990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009be0 tx=0x7f32f0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:30.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.566+0000 7f32eb7fe700 1 -- 172.21.15.89:0/3110440044 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f32e4097630 con 0x7f32f4101060 2023-10-26T06:44:30.646 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 06:44:30 smithi192 systemd[1]: Starting Ceph mds.l for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:30.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.812+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32f40674e0 con 0x7f32f4101060 2023-10-26T06:44:30.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.812+0000 7f32eb7fe700 1 -- 172.21.15.89:0/3110440044 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f32e409ac50 con 0x7f32f4101060 2023-10-26T06:44:30.815 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:30.815 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-26T06:44:30.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.814+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32e00a14d0 msgr2=0x7f32e00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.814+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32e00a14d0 0x7f32e00a3990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009be0 tx=0x7f32f0009260 comp rx=0 tx=0).stop 2023-10-26T06:44:30.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.814+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f4101060 msgr2=0x7f32f419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.814+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f4101060 0x7f32f419ef40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005910 tx=0x7f32e4004730 comp rx=0 tx=0).stop 2023-10-26T06:44:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 shutdown_connections 2023-10-26T06:44:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32e00a14d0 0x7f32e00a3990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f41066b0 0x7f32f4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f41006a0 0x7f32f419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 --2- 172.21.15.89:0/3110440044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32f4101060 0x7f32f419ef40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 >> 172.21.15.89:0/3110440044 conn(0x7f32f40fc170 msgr2=0x7f32f4072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 shutdown_connections 2023-10-26T06:44:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:30.815+0000 7f32fc5ff700 1 -- 172.21.15.89:0/3110440044 wait complete. 2023-10-26T06:44:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 1 2023-10-26T06:44:31.048 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 06:44:30 smithi192 podman[170546]: 2023-10-26T06:44:31.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:31 smithi089 ceph-mon[95031]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:31 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3553366786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:44:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:31 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3110440044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:31 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.192:6832/1842410141,v1:172.21.15.192:6833/1842410141] up:boot 2023-10-26T06:44:31.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:31 smithi089 ceph-mon[95031]: fsmap 1 up:standby 2023-10-26T06:44:31.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-26T06:44:31.395 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-mds[170586]: -- 172.21.15.192:0/2428399018 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559707a51e0 con 0x55597077b000 2023-10-26T06:44:31.396 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 06:44:31 smithi192 bash[170546]: 4b1b2fc318c5cf10302a8ec1643d44a5406013c53139896aabe5d101bb76c6b5 2023-10-26T06:44:31.396 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-l[170562]: starting mds.l at 2023-10-26T06:44:31.396 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 06:44:31 smithi192 systemd[1]: Started Ceph mds.l for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:31.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-mon[131661]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:31.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3553366786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:44:31.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3110440044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:31.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.192:6832/1842410141,v1:172.21.15.192:6833/1842410141] up:boot 2023-10-26T06:44:31.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-mon[131661]: fsmap 1 up:standby 2023-10-26T06:44:31.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-26T06:44:31.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:31 smithi135 ceph-mon[131403]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:31.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:31 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3553366786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:44:31.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:31 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3110440044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:31.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:31 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.192:6832/1842410141,v1:172.21.15.192:6833/1842410141] up:boot 2023-10-26T06:44:31.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:31 smithi135 ceph-mon[131403]: fsmap 1 up:standby 2023-10-26T06:44:31.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-26T06:44:32.390 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.390 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.390 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:32.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:32.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:32.391 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:32 smithi135 ceph-mon[131403]: Deploying daemon mds.k on smithi135 2023-10-26T06:44:32.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:32.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:32.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:32.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:32 smithi192 ceph-mon[131661]: Deploying daemon mds.k on smithi135 2023-10-26T06:44:32.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:32.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:32.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:32.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:32.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:32 smithi089 ceph-mon[95031]: Deploying daemon mds.k on smithi135 2023-10-26T06:44:32.836 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:33.013 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 06:44:32 smithi135 systemd[1]: Starting Ceph mds.k for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.041+0000 7febc1dff700 1 -- 172.21.15.89:0/2303312616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 msgr2=0x7febbc100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.041+0000 7febc1dff700 1 --2- 172.21.15.89:0/2303312616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc100b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feba4009990 tx=0x7feba4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.042+0000 7febc1dff700 1 -- 172.21.15.89:0/2303312616 shutdown_connections 2023-10-26T06:44:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.042+0000 7febc1dff700 1 --2- 172.21.15.89:0/2303312616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febbc101060 0x7febbc10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.042+0000 7febc1dff700 1 --2- 172.21.15.89:0/2303312616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febbc1066b0 0x7febbc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.042+0000 7febc1dff700 1 --2- 172.21.15.89:0/2303312616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc100b20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.042+0000 7febc1dff700 1 -- 172.21.15.89:0/2303312616 >> 172.21.15.89:0/2303312616 conn(0x7febbc0fc170 msgr2=0x7febbc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.042+0000 7febc1dff700 1 -- 172.21.15.89:0/2303312616 shutdown_connections 2023-10-26T06:44:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.042+0000 7febc1dff700 1 -- 172.21.15.89:0/2303312616 wait complete. 2023-10-26T06:44:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 Processor -- start 2023-10-26T06:44:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 -- start start 2023-10-26T06:44:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febbc101060 0x7febbc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febbc1066b0 0x7febbc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc199190 con 0x7febbc1006a0 2023-10-26T06:44:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc199300 con 0x7febbc1066b0 2023-10-26T06:44:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febc1dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc1994c0 con 0x7febbc101060 2023-10-26T06:44:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febbb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febbb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc19ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45244/0 (socket says 172.21.15.89:45244) 2023-10-26T06:44:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.043+0000 7febbb7fe700 1 -- 172.21.15.89:0/2231939009 learned_addr learned my addr 172.21.15.89:0/2231939009 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:33.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febbbfff700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febbc1066b0 0x7febbc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:33.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febb2ffd700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febbc101060 0x7febbc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:33.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febbb7fe700 1 -- 172.21.15.89:0/2231939009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febbc101060 msgr2=0x7febbc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:33.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febbb7fe700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febbc101060 0x7febbc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febbb7fe700 1 -- 172.21.15.89:0/2231939009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febbc1066b0 msgr2=0x7febbc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:33.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febbb7fe700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febbc1066b0 0x7febbc198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febbb7fe700 1 -- 172.21.15.89:0/2231939009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40095f0 con 0x7febbc1006a0 2023-10-26T06:44:33.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febbb7fe700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc19ea00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7febac00ef10 tx=0x7febac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:33.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febb97fa700 1 -- 172.21.15.89:0/2231939009 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac00ca70 con 0x7febbc1006a0 2023-10-26T06:44:33.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febb97fa700 1 -- 172.21.15.89:0/2231939009 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febac00f040 con 0x7febbc1006a0 2023-10-26T06:44:33.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc199740 con 0x7febbc1006a0 2023-10-26T06:44:33.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febb97fa700 1 -- 172.21.15.89:0/2231939009 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac014480 con 0x7febbc1006a0 2023-10-26T06:44:33.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.044+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc1a59f0 con 0x7febbc1006a0 2023-10-26T06:44:33.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.045+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc101de0 con 0x7febbc1006a0 2023-10-26T06:44:33.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.047+0000 7febb97fa700 1 -- 172.21.15.89:0/2231939009 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7febac007500 con 0x7febbc1006a0 2023-10-26T06:44:33.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.048+0000 7febb97fa700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feba80a14d0 0x7feba80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:33.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.049+0000 7febb97fa700 1 -- 172.21.15.89:0/2231939009 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7febac0ccd10 con 0x7febbc1006a0 2023-10-26T06:44:33.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.049+0000 7febb2ffd700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feba80a14d0 0x7feba80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:33.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.049+0000 7febb2ffd700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feba80a14d0 0x7feba80a3990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feba400b2e0 tx=0x7feba4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:33.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.052+0000 7febb97fa700 1 -- 172.21.15.89:0/2231939009 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7febac0d2020 con 0x7febbc1006a0 2023-10-26T06:44:33.275 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:33 smithi135 ceph-mon[131403]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:33.275 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 06:44:33 smithi135 podman[169167]: 2023-10-26T06:44:33.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.301+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febbc0674e0 con 0x7febbc1006a0 2023-10-26T06:44:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.301+0000 7febb97fa700 1 -- 172.21.15.89:0/2231939009 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7febac0966a0 con 0x7febbc1006a0 2023-10-26T06:44:33.333 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:33.334 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-26T06:44:33.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.303+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feba80a14d0 msgr2=0x7feba80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:33.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.303+0000 7febc1dff700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feba80a14d0 0x7feba80a3990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feba400b2e0 tx=0x7feba4005fd0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.303+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 msgr2=0x7febbc19ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:33.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.303+0000 7febc1dff700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc19ea00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7febac00ef10 tx=0x7febac00c3a0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 shutdown_connections 2023-10-26T06:44:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feba80a14d0 0x7feba80a3990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febbc1006a0 0x7febbc19ea00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febbc1066b0 0x7febbc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 --2- 172.21.15.89:0/2231939009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febbc101060 0x7febbc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 >> 172.21.15.89:0/2231939009 conn(0x7febbc0fc170 msgr2=0x7febbc072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 shutdown_connections 2023-10-26T06:44:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:33.304+0000 7febc1dff700 1 -- 172.21.15.89:0/2231939009 wait complete. 2023-10-26T06:44:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 2 2023-10-26T06:44:33.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:33 smithi089 ceph-mon[95031]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:33.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:33 smithi192 ceph-mon[131661]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:33.717 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 06:44:33 smithi135 ceph-mds[169207]: -- 172.21.15.135:0/2338547868 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56014ccc11e0 con 0x56014cc97000 2023-10-26T06:44:33.717 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 06:44:33 smithi135 bash[169167]: 5fa4686d85359d589eb5a12080df93a906acbf18b1a60746e4a4e4fbfa66a244 2023-10-26T06:44:33.718 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 06:44:33 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-k[169183]: starting mds.k at 2023-10-26T06:44:33.718 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 06:44:33 smithi135 systemd[1]: Started Ceph mds.k for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:34.418 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:34 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2231939009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:34.418 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.418 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:34.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:34.419 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:34 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:34.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:34 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2231939009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:34.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:34.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:34.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:34 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:34.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:34 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2231939009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:34.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:34.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:34.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:34.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:34 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:35.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:35.440 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:35 smithi089 ceph-mon[95031]: Deploying daemon mds.j on smithi089 2023-10-26T06:44:35.441 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:35 smithi089 ceph-mon[95031]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:35.441 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:35 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.135:6832/3774068069,v1:172.21.15.135:6833/3774068069] up:boot 2023-10-26T06:44:35.441 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:35 smithi089 ceph-mon[95031]: fsmap 2 up:standby 2023-10-26T06:44:35.441 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:35 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-26T06:44:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:35 smithi135 ceph-mon[131403]: Deploying daemon mds.j on smithi089 2023-10-26T06:44:35.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:35 smithi135 ceph-mon[131403]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:35.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:35 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.135:6832/3774068069,v1:172.21.15.135:6833/3774068069] up:boot 2023-10-26T06:44:35.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:35 smithi135 ceph-mon[131403]: fsmap 2 up:standby 2023-10-26T06:44:35.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:35 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-26T06:44:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 -- 172.21.15.89:0/1297976054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9301066b0 msgr2=0x7fd930106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 --2- 172.21.15.89:0/1297976054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9301066b0 0x7fd930106a90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd920009990 tx=0x7fd920009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 -- 172.21.15.89:0/1297976054 shutdown_connections 2023-10-26T06:44:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 --2- 172.21.15.89:0/1297976054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd9301066b0 0x7fd930106a90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 --2- 172.21.15.89:0/1297976054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930101060 0x7fd93010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 --2- 172.21.15.89:0/1297976054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301006a0 0x7fd930100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 -- 172.21.15.89:0/1297976054 >> 172.21.15.89:0/1297976054 conn(0x7fd9300fc1d0 msgr2=0x7fd9300fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 -- 172.21.15.89:0/1297976054 shutdown_connections 2023-10-26T06:44:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.601+0000 7fd935fff700 1 -- 172.21.15.89:0/1297976054 wait complete. 2023-10-26T06:44:35.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.602+0000 7fd935fff700 1 Processor -- start 2023-10-26T06:44:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.602+0000 7fd935fff700 1 -- start start 2023-10-26T06:44:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd935fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9301006a0 0x7fd930198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd935fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd930101060 0x7fd930199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd935fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301066b0 0x7fd93019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:35.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd935fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93019cb30 con 0x7fd930101060 2023-10-26T06:44:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd935fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93019cca0 con 0x7fd9301066b0 2023-10-26T06:44:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd935fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93019ce60 con 0x7fd9301006a0 2023-10-26T06:44:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd92effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd930101060 0x7fd930199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd92f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9301006a0 0x7fd930198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:35.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd92effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd930101060 0x7fd930199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45258/0 (socket says 172.21.15.89:45258) 2023-10-26T06:44:35.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd92effd700 1 -- 172.21.15.89:0/2113040891 learned_addr learned my addr 172.21.15.89:0/2113040891 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:35.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd92f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9301006a0 0x7fd930198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:59794/0 (socket says 172.21.15.89:59794) 2023-10-26T06:44:35.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.603+0000 7fd92ffff700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301066b0 0x7fd93019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:35.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92f7fe700 1 -- 172.21.15.89:0/2113040891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301066b0 msgr2=0x7fd93019c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92f7fe700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301066b0 0x7fd93019c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92f7fe700 1 -- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd930101060 msgr2=0x7fd930199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92f7fe700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd930101060 0x7fd930199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92f7fe700 1 -- 172.21.15.89:0/2113040891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200095f0 con 0x7fd9301006a0 2023-10-26T06:44:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92effd700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd930101060 0x7fd930199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92f7fe700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9301006a0 0x7fd930198d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9200098e0 tx=0x7fd920005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92cff9700 1 -- 172.21.15.89:0/2113040891 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92001d070 con 0x7fd9301006a0 2023-10-26T06:44:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92cff9700 1 -- 172.21.15.89:0/2113040891 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd920022470 con 0x7fd9301006a0 2023-10-26T06:44:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93019d0e0 con 0x7fd9301006a0 2023-10-26T06:44:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd92cff9700 1 -- 172.21.15.89:0/2113040891 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920017650 con 0x7fd9301006a0 2023-10-26T06:44:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.604+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9301a5aa0 con 0x7fd9301006a0 2023-10-26T06:44:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.605+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd930101de0 con 0x7fd9301006a0 2023-10-26T06:44:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.608+0000 7fd92cff9700 1 -- 172.21.15.89:0/2113040891 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd9200225e0 con 0x7fd9301006a0 2023-10-26T06:44:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.609+0000 7fd92cff9700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9180a14d0 0x7fd9180a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.609+0000 7fd92cff9700 1 -- 172.21.15.89:0/2113040891 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fd9200ce470 con 0x7fd9301006a0 2023-10-26T06:44:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.609+0000 7fd92effd700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9180a14d0 0x7fd9180a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.610+0000 7fd92effd700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9180a14d0 0x7fd9180a3990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd93019a210 tx=0x7fd92400b500 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.611+0000 7fd92cff9700 1 -- 172.21.15.89:0/2113040891 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd920097e90 con 0x7fd9301006a0 2023-10-26T06:44:35.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:35 smithi192 ceph-mon[131661]: Deploying daemon mds.j on smithi089 2023-10-26T06:44:35.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:35 smithi192 ceph-mon[131661]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:35.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:35 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.135:6832/3774068069,v1:172.21.15.135:6833/3774068069] up:boot 2023-10-26T06:44:35.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:35 smithi192 ceph-mon[131661]: fsmap 2 up:standby 2023-10-26T06:44:35.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:35 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-26T06:44:35.789 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 06:44:35 smithi089 systemd[1]: Starting Ceph mds.j for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.902+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd93019a060 con 0x7fd9301006a0 2023-10-26T06:44:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.903+0000 7fd92cff9700 1 -- 172.21.15.89:0/2113040891 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fd92009b4b0 con 0x7fd9301006a0 2023-10-26T06:44:35.905 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:35.905 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-26T06:44:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9180a14d0 msgr2=0x7fd9180a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:35.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9180a14d0 0x7fd9180a3990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd93019a210 tx=0x7fd92400b500 comp rx=0 tx=0).stop 2023-10-26T06:44:35.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9301006a0 msgr2=0x7fd930198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:35.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9301006a0 0x7fd930198d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9200098e0 tx=0x7fd920005740 comp rx=0 tx=0).stop 2023-10-26T06:44:35.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 shutdown_connections 2023-10-26T06:44:35.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9180a14d0 0x7fd9180a3990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301066b0 0x7fd93019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9301006a0 0x7fd930198d00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 --2- 172.21.15.89:0/2113040891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd930101060 0x7fd930199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 >> 172.21.15.89:0/2113040891 conn(0x7fd9300fc1d0 msgr2=0x7fd93010b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 shutdown_connections 2023-10-26T06:44:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:35.905+0000 7fd935fff700 1 -- 172.21.15.89:0/2113040891 wait complete. 2023-10-26T06:44:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 3 2023-10-26T06:44:36.195 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:36 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2113040891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:36.195 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 06:44:35 smithi089 podman[149521]: 2023-10-26T06:44:36.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:36 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2113040891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:36.539 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 06:44:36 smithi089 ceph-mds[149563]: -- 172.21.15.89:0/3868864433 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fbb8cd91e0 con 0x55fbb8cb0800 2023-10-26T06:44:36.539 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 06:44:36 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-j[149539]: starting mds.j at 2023-10-26T06:44:36.540 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 06:44:36 smithi089 bash[149521]: e746f99c230bb562fc673ed58440a6eba0c6773365f7db45155c0e75bbe09f6d 2023-10-26T06:44:36.540 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 06:44:36 smithi089 systemd[1]: Started Ceph mds.j for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:36.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:36 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2113040891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:37.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:37 smithi135 ceph-mon[131403]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:37.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:37.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:37.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:37.487 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:37 smithi089 ceph-mon[95031]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:37.487 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.487 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.487 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:37.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:37.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:37.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:37 smithi192 ceph-mon[131661]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:37.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:37.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:37.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:37.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:37.927 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 -- 172.21.15.89:0/4033466187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1028c0 msgr2=0x7f6dcc102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/4033466187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc102d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009990 tx=0x7f6dbc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 -- 172.21.15.89:0/4033466187 shutdown_connections 2023-10-26T06:44:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/4033466187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc103280 0x7f6dcc10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/4033466187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc1088d0 0x7f6dcc108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/4033466187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc102d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 -- 172.21.15.89:0/4033466187 >> 172.21.15.89:0/4033466187 conn(0x7f6dcc0fe370 msgr2=0x7f6dcc100790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 -- 172.21.15.89:0/4033466187 shutdown_connections 2023-10-26T06:44:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.352+0000 7f6dd39ff700 1 -- 172.21.15.89:0/4033466187 wait complete. 2023-10-26T06:44:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 Processor -- start 2023-10-26T06:44:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 -- start start 2023-10-26T06:44:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc103280 0x7f6dcc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1088d0 0x7f6dcc198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1991b0 con 0x7f6dcc1088d0 2023-10-26T06:44:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc199320 con 0x7f6dcc1028c0 2023-10-26T06:44:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd39ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1994e0 con 0x7f6dcc103280 2023-10-26T06:44:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.353+0000 7f6dd179b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd0f9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc103280 0x7f6dcc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:57980/0 (socket says 172.21.15.89:57980) 2023-10-26T06:44:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 -- 172.21.15.89:0/3224811538 learned_addr learned my addr 172.21.15.89:0/3224811538 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd1f9c700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1088d0 0x7f6dcc198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:38.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 -- 172.21.15.89:0/3224811538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc103280 msgr2=0x7f6dcc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:38.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc103280 0x7f6dcc19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 -- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1088d0 msgr2=0x7f6dcc198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:38.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1088d0 0x7f6dcc198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 -- 172.21.15.89:0/3224811538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0095f0 con 0x7f6dcc1028c0 2023-10-26T06:44:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd0f9a700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc103280 0x7f6dcc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd179b700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc19e9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800e9a0 tx=0x7f6dc800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dc27fc700 1 -- 172.21.15.89:0/3224811538 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8009790 con 0x7f6dcc1028c0 2023-10-26T06:44:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dc27fc700 1 -- 172.21.15.89:0/3224811538 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc8010430 con 0x7f6dcc1028c0 2023-10-26T06:44:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc1997c0 con 0x7f6dcc1028c0 2023-10-26T06:44:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.354+0000 7f6dc27fc700 1 -- 172.21.15.89:0/3224811538 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8018680 con 0x7f6dcc1028c0 2023-10-26T06:44:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.355+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc1a59c0 con 0x7f6dcc1028c0 2023-10-26T06:44:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.356+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc10ae50 con 0x7f6dcc1028c0 2023-10-26T06:44:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.357+0000 7f6dc27fc700 1 -- 172.21.15.89:0/3224811538 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6dc80105a0 con 0x7f6dcc1028c0 2023-10-26T06:44:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.358+0000 7f6dc27fc700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db80a14d0 0x7f6db80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.358+0000 7f6dc27fc700 1 -- 172.21.15.89:0/3224811538 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f6dc8014070 con 0x7f6dcc1028c0 2023-10-26T06:44:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.358+0000 7f6dd0f9a700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db80a14d0 0x7f6db80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.358+0000 7f6dd0f9a700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db80a14d0 0x7f6db80a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc199cc0 tx=0x7f6dbc005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.363+0000 7f6dc27fc700 1 -- 172.21.15.89:0/3224811538 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6dc8052100 con 0x7f6dcc1028c0 2023-10-26T06:44:38.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:38 smithi135 ceph-mon[131403]: Deploying daemon mds.d on smithi089 2023-10-26T06:44:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:38 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.89:6834/229884175,v1:172.21.15.89:6835/229884175] up:boot 2023-10-26T06:44:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:38 smithi135 ceph-mon[131403]: fsmap 3 up:standby 2023-10-26T06:44:38.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:38 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-26T06:44:38.513 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:38 smithi089 ceph-mon[95031]: Deploying daemon mds.d on smithi089 2023-10-26T06:44:38.513 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:38 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.89:6834/229884175,v1:172.21.15.89:6835/229884175] up:boot 2023-10-26T06:44:38.514 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:38 smithi089 ceph-mon[95031]: fsmap 3 up:standby 2023-10-26T06:44:38.514 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:38 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-26T06:44:38.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.623+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dcc199fc0 con 0x7f6dcc1028c0 2023-10-26T06:44:38.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.624+0000 7f6dc27fc700 1 -- 172.21.15.89:0/3224811538 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f6dc8096510 con 0x7f6dcc1028c0 2023-10-26T06:44:38.625 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:38.626 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-26T06:44:38.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db80a14d0 msgr2=0x7f6db80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:38.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db80a14d0 0x7f6db80a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc199cc0 tx=0x7f6dbc005fd0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1028c0 msgr2=0x7f6dcc19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:38.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc19e9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800e9a0 tx=0x7f6dc800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 shutdown_connections 2023-10-26T06:44:38.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db80a14d0 0x7f6db80a3990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1088d0 0x7f6dcc198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1028c0 0x7f6dcc19e9d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 --2- 172.21.15.89:0/3224811538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc103280 0x7f6dcc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:38.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 >> 172.21.15.89:0/3224811538 conn(0x7f6dcc0fe370 msgr2=0x7f6dcc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:38.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 shutdown_connections 2023-10-26T06:44:38.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:38.627+0000 7f6dd39ff700 1 -- 172.21.15.89:0/3224811538 wait complete. 2023-10-26T06:44:38.632 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 4 2023-10-26T06:44:38.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:38 smithi192 ceph-mon[131661]: Deploying daemon mds.d on smithi089 2023-10-26T06:44:38.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:38 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.89:6834/229884175,v1:172.21.15.89:6835/229884175] up:boot 2023-10-26T06:44:38.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:38 smithi192 ceph-mon[131661]: fsmap 3 up:standby 2023-10-26T06:44:38.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:38 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-26T06:44:38.789 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 06:44:38 smithi089 systemd[1]: Starting Ceph mds.d for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:39.275 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:39 smithi089 ceph-mon[95031]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:39.275 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:39 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3224811538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:39.276 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 06:44:39 smithi089 podman[149907]: 2023-10-26T06:44:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:39 smithi135 ceph-mon[131403]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:39 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3224811538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:39.539 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 06:44:39 smithi089 bash[149907]: a1775dd227e09c9ca38b1f34b4059b3b118776a358f75eb01a4154ead31b90aa 2023-10-26T06:44:39.539 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 06:44:39 smithi089 ceph-mds[149947]: -- 172.21.15.89:0/1438171005 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558f9580d1e0 con 0x558f957e5000 2023-10-26T06:44:39.540 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 06:44:39 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-d[149923]: starting mds.d at 2023-10-26T06:44:39.540 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 06:44:39 smithi089 systemd[1]: Started Ceph mds.d for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:39.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:39 smithi192 ceph-mon[131661]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:39.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:39 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3224811538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:40.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:40.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:40.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:40.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: Deploying daemon mds.e on smithi135 2023-10-26T06:44:40.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:40.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.89:6836/104209914,v1:172.21.15.89:6837/104209914] up:boot 2023-10-26T06:44:40.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: fsmap 4 up:standby 2023-10-26T06:44:40.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:40 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-26T06:44:40.649 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:40.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:40.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:40.669 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:40.669 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: Deploying daemon mds.e on smithi135 2023-10-26T06:44:40.669 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:40.669 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.89:6836/104209914,v1:172.21.15.89:6837/104209914] up:boot 2023-10-26T06:44:40.670 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: fsmap 4 up:standby 2023-10-26T06:44:40.670 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:40 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-26T06:44:40.717 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:40.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:40.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:40.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:40.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: Deploying daemon mds.e on smithi135 2023-10-26T06:44:40.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:40.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.89:6836/104209914,v1:172.21.15.89:6837/104209914] up:boot 2023-10-26T06:44:40.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: fsmap 4 up:standby 2023-10-26T06:44:40.719 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:40 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-26T06:44:40.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/2709433138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb4101060 msgr2=0x7fcbb410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:40.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/2709433138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb4101060 0x7fcbb410d570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b0a0 tx=0x7fcbac00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:44:40.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/2709433138 shutdown_connections 2023-10-26T06:44:40.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/2709433138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb4101060 0x7fcbb410d570 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:40.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/2709433138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 0x7fcbb4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:40.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/2709433138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb41006a0 0x7fcbb4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:40.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/2709433138 >> 172.21.15.89:0/2709433138 conn(0x7fcbb40fc170 msgr2=0x7fcbb40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:40.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/2709433138 shutdown_connections 2023-10-26T06:44:40.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.857+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/2709433138 wait complete. 2023-10-26T06:44:40.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.858+0000 7fcbbb3ff700 1 Processor -- start 2023-10-26T06:44:40.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.858+0000 7fcbbb3ff700 1 -- start start 2023-10-26T06:44:40.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbbb3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb41006a0 0x7fcbb419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:40.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbbb3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb4101060 0x7fcbb419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbbb3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 0x7fcbb4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbbb3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb4199190 con 0x7fcbb41006a0 2023-10-26T06:44:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbbb3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb4199300 con 0x7fcbb41066b0 2023-10-26T06:44:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbbb3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb41994c0 con 0x7fcbb4101060 2023-10-26T06:44:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb919b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb41006a0 0x7fcbb419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:40.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 0x7fcbb4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 0x7fcbb4198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58372/0 (socket says 172.21.15.89:58372) 2023-10-26T06:44:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 -- 172.21.15.89:0/4112759047 learned_addr learned my addr 172.21.15.89:0/4112759047 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb899a700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb4101060 0x7fcbb419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:40.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 -- 172.21.15.89:0/4112759047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb4101060 msgr2=0x7fcbb419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb4101060 0x7fcbb419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 -- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb41006a0 msgr2=0x7fcbb419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb41006a0 0x7fcbb419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb999c700 1 -- 172.21.15.89:0/4112759047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac009d00 con 0x7fcbb41066b0 2023-10-26T06:44:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.859+0000 7fcbb919b700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb41006a0 0x7fcbb419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:40.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.860+0000 7fcbb999c700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 0x7fcbb4198b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b070 tx=0x7fcbac0077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.860+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/4112759047 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00e040 con 0x7fcbb41066b0 2023-10-26T06:44:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.860+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb4199740 con 0x7fcbb41066b0 2023-10-26T06:44:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.860+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/4112759047 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac0094c0 con 0x7fcbb41066b0 2023-10-26T06:44:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.860+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/4112759047 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0043a0 con 0x7fcbb41066b0 2023-10-26T06:44:40.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.860+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb41a5a40 con 0x7fcbb41066b0 2023-10-26T06:44:40.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.861+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb4101de0 con 0x7fcbb41066b0 2023-10-26T06:44:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.863+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/4112759047 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fcbac004500 con 0x7fcbb41066b0 2023-10-26T06:44:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.864+0000 7fcbaa7fc700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcba00a14d0 0x7fcba00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.864+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/4112759047 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fcbac0cd7f0 con 0x7fcbb41066b0 2023-10-26T06:44:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.864+0000 7fcbb919b700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcba00a14d0 0x7fcba00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:40.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.865+0000 7fcbb919b700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcba00a14d0 0x7fcba00a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009fd0 tx=0x7fcbb0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:40.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:40.866+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/4112759047 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcbac097210 con 0x7fcbb41066b0 2023-10-26T06:44:40.967 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 06:44:40 smithi135 systemd[1]: Starting Ceph mds.e for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:41.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.111+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbb4067530 con 0x7fcbb41066b0 2023-10-26T06:44:41.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.112+0000 7fcbaa7fc700 1 -- 172.21.15.89:0/4112759047 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3698 (secure 0 0 0) 0x7fcbac021e90 con 0x7fcbb41066b0 2023-10-26T06:44:41.113 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:41.113 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2023-10-26T06:44:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcba00a14d0 msgr2=0x7fcba00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcba00a14d0 0x7fcba00a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009fd0 tx=0x7fcbb0009260 comp rx=0 tx=0).stop 2023-10-26T06:44:41.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 msgr2=0x7fcbb4198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 0x7fcbb4198b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b070 tx=0x7fcbac0077f0 comp rx=0 tx=0).stop 2023-10-26T06:44:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 shutdown_connections 2023-10-26T06:44:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcba00a14d0 0x7fcba00a3990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb41006a0 0x7fcbb419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:41.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbb41066b0 0x7fcbb4198b30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 --2- 172.21.15.89:0/4112759047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb4101060 0x7fcbb419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 >> 172.21.15.89:0/4112759047 conn(0x7fcbb40fc170 msgr2=0x7fcbb40fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 shutdown_connections 2023-10-26T06:44:41.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:41.114+0000 7fcbbb3ff700 1 -- 172.21.15.89:0/4112759047 wait complete. 2023-10-26T06:44:41.118 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2023-10-26T06:44:41.503 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:41 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4112759047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:41.503 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 06:44:41 smithi135 podman[169513]: 2023-10-26T06:44:41.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:41 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4112759047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:41.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:41 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4112759047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:41.967 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 06:44:41 smithi135 ceph-mds[169553]: -- 172.21.15.135:0/4011258387 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595a95371e0 con 0x5595a950c800 2023-10-26T06:44:41.967 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 06:44:41 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-e[169529]: starting mds.e at 2023-10-26T06:44:41.968 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 06:44:41 smithi135 bash[169513]: c9cc36d145f695dbc49bda41ed2838e0a4b83ffee67fbb5c16638817cd0f0aab 2023-10-26T06:44:41.968 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 06:44:41 smithi135 systemd[1]: Started Ceph mds.e for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:42.845 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:42.845 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:42.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:42.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:42.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:42.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:42.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: Deploying daemon mds.f on smithi192 2023-10-26T06:44:42.847 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:42.847 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.135:6834/4067855863,v1:172.21.15.135:6835/4067855863] up:boot 2023-10-26T06:44:42.847 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: fsmap 5 up:standby 2023-10-26T06:44:42.847 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:42 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-26T06:44:42.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:42.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:42.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:42.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:42.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:42.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:42.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: Deploying daemon mds.f on smithi192 2023-10-26T06:44:42.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:42.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.135:6834/4067855863,v1:172.21.15.135:6835/4067855863] up:boot 2023-10-26T06:44:42.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: fsmap 5 up:standby 2023-10-26T06:44:42.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:42 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-26T06:44:43.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:43.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:43.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:43.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:43.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:43.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:43.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: Deploying daemon mds.f on smithi192 2023-10-26T06:44:43.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:43.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.135:6834/4067855863,v1:172.21.15.135:6835/4067855863] up:boot 2023-10-26T06:44:43.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: fsmap 5 up:standby 2023-10-26T06:44:43.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:42 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-26T06:44:43.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 -- 172.21.15.89:0/2156672036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 msgr2=0x7f139410c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2156672036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 0x7f139410c9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1388009990 tx=0x7f1388009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 -- 172.21.15.89:0/2156672036 shutdown_connections 2023-10-26T06:44:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2156672036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f139410cee0 0x7f139410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2156672036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1394072c20 0x7f1394073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2156672036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 0x7f139410c9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 -- 172.21.15.89:0/2156672036 >> 172.21.15.89:0/2156672036 conn(0x7f13940fc170 msgr2=0x7f13940fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:43.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 -- 172.21.15.89:0/2156672036 shutdown_connections 2023-10-26T06:44:43.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.342+0000 7f139a3ff700 1 -- 172.21.15.89:0/2156672036 wait complete. 2023-10-26T06:44:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.343+0000 7f139a3ff700 1 Processor -- start 2023-10-26T06:44:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139a3ff700 1 -- start start 2023-10-26T06:44:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139a3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1394072c20 0x7f1394198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139a3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 0x7f13941992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:43.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139a3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139410cee0 0x7f139419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139a3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139419dcd0 con 0x7f1394072c20 2023-10-26T06:44:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139a3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139419de40 con 0x7f1394073540 2023-10-26T06:44:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139a3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139419e000 con 0x7f139410cee0 2023-10-26T06:44:43.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1394072c20 0x7f1394198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1394072c20 0x7f1394198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58642/0 (socket says 172.21.15.89:58642) 2023-10-26T06:44:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 -- 172.21.15.89:0/2404296183 learned_addr learned my addr 172.21.15.89:0/2404296183 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f139899c700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139410cee0 0x7f139419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f13937fe700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 0x7f13941992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:43.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 -- 172.21.15.89:0/2404296183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139410cee0 msgr2=0x7f139419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139410cee0 0x7f139419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 -- 172.21.15.89:0/2404296183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 msgr2=0x7f13941992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 0x7f13941992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f1393fff700 1 -- 172.21.15.89:0/2404296183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880095f0 con 0x7f1394072c20 2023-10-26T06:44:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.344+0000 7f13937fe700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 0x7f13941992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:44:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.345+0000 7f1393fff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1394072c20 0x7f1394198db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f138400e9a0 tx=0x7f138400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.345+0000 7f13917fa700 1 -- 172.21.15.89:0/2404296183 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384009790 con 0x7f1394072c20 2023-10-26T06:44:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.345+0000 7f13917fa700 1 -- 172.21.15.89:0/2404296183 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f138400cbd0 con 0x7f1394072c20 2023-10-26T06:44:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.345+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139419e2e0 con 0x7f1394072c20 2023-10-26T06:44:43.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.345+0000 7f13917fa700 1 -- 172.21.15.89:0/2404296183 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384018680 con 0x7f1394072c20 2023-10-26T06:44:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.345+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941a5b20 con 0x7f1394072c20 2023-10-26T06:44:43.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.346+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13941080f0 con 0x7f1394072c20 2023-10-26T06:44:43.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.348+0000 7f13917fa700 1 -- 172.21.15.89:0/2404296183 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f138400cd40 con 0x7f1394072c20 2023-10-26T06:44:43.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.349+0000 7f13917fa700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f137c0a14d0 0x7f137c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:43.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.349+0000 7f13917fa700 1 -- 172.21.15.89:0/2404296183 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f1384014070 con 0x7f1394072c20 2023-10-26T06:44:43.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.349+0000 7f13937fe700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f137c0a14d0 0x7f137c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:43.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.350+0000 7f13937fe700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f137c0a14d0 0x7f137c0a3990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1388006010 tx=0x7f138800b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:43.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.353+0000 7f13917fa700 1 -- 172.21.15.89:0/2404296183 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f13840966e0 con 0x7f1394072c20 2023-10-26T06:44:43.433 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 06:44:43 smithi192 systemd[1]: Starting Ceph mds.f for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.601+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f139419a0b0 con 0x7f1394072c20 2023-10-26T06:44:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.602+0000 7f13917fa700 1 -- 172.21.15.89:0/2404296183 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7f1384099d00 con 0x7f1394072c20 2023-10-26T06:44:43.603 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:43.604 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-26T06:44:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f137c0a14d0 msgr2=0x7f137c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f137c0a14d0 0x7f137c0a3990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1388006010 tx=0x7f138800b260 comp rx=0 tx=0).stop 2023-10-26T06:44:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1394072c20 msgr2=0x7f1394198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1394072c20 0x7f1394198db0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f138400e9a0 tx=0x7f138400ed60 comp rx=0 tx=0).stop 2023-10-26T06:44:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 shutdown_connections 2023-10-26T06:44:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f137c0a14d0 0x7f137c0a3990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1394072c20 0x7f1394198db0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139410cee0 0x7f139419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.604+0000 7f139a3ff700 1 --2- 172.21.15.89:0/2404296183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1394073540 0x7f13941992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.605+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 >> 172.21.15.89:0/2404296183 conn(0x7f13940fc170 msgr2=0x7f13940fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.605+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 shutdown_connections 2023-10-26T06:44:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:43.605+0000 7f139a3ff700 1 -- 172.21.15.89:0/2404296183 wait complete. 2023-10-26T06:44:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 6 2023-10-26T06:44:43.704 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:43 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2404296183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:43.704 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 06:44:43 smithi192 podman[170887]: 2023-10-26T06:44:43.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:43 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2404296183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:44.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:43 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2404296183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:44.145 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 06:44:43 smithi192 ceph-mds[170927]: -- 172.21.15.192:0/3635200668 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5615fe5811e0 con 0x5615fe556800 2023-10-26T06:44:44.145 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 06:44:43 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-f[170903]: starting mds.f at 2023-10-26T06:44:44.146 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 06:44:43 smithi192 bash[170887]: 3f195a45af568d6d4bcb02f3bcd6e9aa53d972fdad65d441852d65c733c74ac0 2023-10-26T06:44:44.146 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 06:44:43 smithi192 systemd[1]: Started Ceph mds.f for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:45.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:45.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:45.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:45.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:45.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: Deploying daemon mds.c on smithi192 2023-10-26T06:44:45.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.192:6834/2073006190,v1:172.21.15.192:6835/2073006190] up:boot 2023-10-26T06:44:45.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: fsmap 6 up:standby 2023-10-26T06:44:45.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:44 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-26T06:44:45.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:45.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:45.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:45.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:45.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: Deploying daemon mds.c on smithi192 2023-10-26T06:44:45.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.192:6834/2073006190,v1:172.21.15.192:6835/2073006190] up:boot 2023-10-26T06:44:45.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: fsmap 6 up:standby 2023-10-26T06:44:45.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:44 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-26T06:44:45.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:45.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:45.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:45.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:45.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:45.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: Deploying daemon mds.c on smithi192 2023-10-26T06:44:45.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.192:6834/2073006190,v1:172.21.15.192:6835/2073006190] up:boot 2023-10-26T06:44:45.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: fsmap 6 up:standby 2023-10-26T06:44:45.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:44 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-26T06:44:45.626 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:45.645 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 06:44:45 smithi192 systemd[1]: Starting Ceph mds.c for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:45.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 -- 172.21.15.89:0/2117458690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 msgr2=0x7f6dcc106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:45.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/2117458690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc106a90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009990 tx=0x7f6dbc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:45.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 -- 172.21.15.89:0/2117458690 shutdown_connections 2023-10-26T06:44:45.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/2117458690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc106a90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:45.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/2117458690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc101060 0x7f6dcc10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:45.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/2117458690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1006a0 0x7f6dcc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:45.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 -- 172.21.15.89:0/2117458690 >> 172.21.15.89:0/2117458690 conn(0x7f6dcc0fc170 msgr2=0x7f6dcc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:45.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 -- 172.21.15.89:0/2117458690 shutdown_connections 2023-10-26T06:44:45.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.832+0000 7f6dd1fff700 1 -- 172.21.15.89:0/2117458690 wait complete. 2023-10-26T06:44:45.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.833+0000 7f6dd1fff700 1 Processor -- start 2023-10-26T06:44:45.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.833+0000 7f6dd1fff700 1 -- start start 2023-10-26T06:44:45.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.833+0000 7f6dd1fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1006a0 0x7f6dcc072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:45.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.833+0000 7f6dd1fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc101060 0x7f6dcc073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:45.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dd1fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:45.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dd1fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1a4ec0 con 0x7f6dcc1066b0 2023-10-26T06:44:45.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dd1fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1a5030 con 0x7f6dcc1006a0 2023-10-26T06:44:45.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dd1fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1a51f0 con 0x7f6dcc101060 2023-10-26T06:44:45.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:45.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58666/0 (socket says 172.21.15.89:58666) 2023-10-26T06:44:45.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 -- 172.21.15.89:0/315571671 learned_addr learned my addr 172.21.15.89:0/315571671 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:45.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc101060 0x7f6dcc073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:45.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcb7fe700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1006a0 0x7f6dcc072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:45.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 -- 172.21.15.89:0/315571671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc101060 msgr2=0x7f6dcc073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:45.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc101060 0x7f6dcc073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:45.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 -- 172.21.15.89:0/315571671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1006a0 msgr2=0x7f6dcc072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:45.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1006a0 0x7f6dcc072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:45.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 -- 172.21.15.89:0/315571671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0095f0 con 0x7f6dcc1066b0 2023-10-26T06:44:45.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dcbfff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc073570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007ee0 tx=0x7f6dc400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:45.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.834+0000 7f6dc8ff9700 1 -- 172.21.15.89:0/315571671 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4017070 con 0x7f6dcc1066b0 2023-10-26T06:44:45.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.835+0000 7f6dc8ff9700 1 -- 172.21.15.89:0/315571671 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc400f040 con 0x7f6dcc1066b0 2023-10-26T06:44:45.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.835+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc1a54d0 con 0x7f6dcc1066b0 2023-10-26T06:44:45.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.835+0000 7f6dc8ff9700 1 -- 172.21.15.89:0/315571671 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4005580 con 0x7f6dcc1066b0 2023-10-26T06:44:45.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.835+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc1a5990 con 0x7f6dcc1066b0 2023-10-26T06:44:45.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.836+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc108ca0 con 0x7f6dcc1066b0 2023-10-26T06:44:45.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.837+0000 7f6dc8ff9700 1 -- 172.21.15.89:0/315571671 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6dc400db20 con 0x7f6dcc1066b0 2023-10-26T06:44:45.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.839+0000 7f6dc8ff9700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db40a14d0 0x7f6db40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:45.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.839+0000 7f6dc8ff9700 1 -- 172.21.15.89:0/315571671 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f6dc40cc100 con 0x7f6dcc1066b0 2023-10-26T06:44:45.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.839+0000 7f6dcb7fe700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db40a14d0 0x7f6db40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:45.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.840+0000 7f6dcb7fe700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db40a14d0 0x7f6db40a3990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006010 tx=0x7f6dbc00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:45.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:45.843+0000 7f6dc8ff9700 1 -- 172.21.15.89:0/315571671 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6dc4095a90 con 0x7f6dcc1066b0 2023-10-26T06:44:45.963 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 06:44:45 smithi192 podman[171227]: 2023-10-26T06:44:46.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.096+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dcc101de0 con 0x7f6dcc1066b0 2023-10-26T06:44:46.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.097+0000 7f6dc8ff9700 1 -- 172.21.15.89:0/315571671 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7f6dc40990b0 con 0x7f6dcc1066b0 2023-10-26T06:44:46.098 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:46.098 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-26T06:44:46.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db40a14d0 msgr2=0x7f6db40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:46.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db40a14d0 0x7f6db40a3990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006010 tx=0x7f6dbc00b540 comp rx=0 tx=0).stop 2023-10-26T06:44:46.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 msgr2=0x7f6dcc073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:46.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc073570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007ee0 tx=0x7f6dc400d3b0 comp rx=0 tx=0).stop 2023-10-26T06:44:46.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 shutdown_connections 2023-10-26T06:44:46.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6db40a14d0 0x7f6db40a3990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:46.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dcc1066b0 0x7f6dcc073570 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:46.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc1006a0 0x7f6dcc072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:46.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 --2- 172.21.15.89:0/315571671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dcc101060 0x7f6dcc073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:46.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 >> 172.21.15.89:0/315571671 conn(0x7f6dcc0fc170 msgr2=0x7f6dcc0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:46.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 shutdown_connections 2023-10-26T06:44:46.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:46.099+0000 7f6dd1fff700 1 -- 172.21.15.89:0/315571671 wait complete. 2023-10-26T06:44:46.103 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-26T06:44:46.395 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 06:44:45 smithi192 ceph-mds[171267]: -- 172.21.15.192:0/3784236909 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1fa56f1e0 con 0x55d1fa544800 2023-10-26T06:44:46.395 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 06:44:45 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-c[171243]: starting mds.c at 2023-10-26T06:44:46.396 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 06:44:46 smithi192 bash[171227]: 9fa49fb886addb529359d0b6644ba2c677743ca5fddb68b2dba50ed666b91b72 2023-10-26T06:44:46.396 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 06:44:46 smithi192 systemd[1]: Started Ceph mds.c for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:47.331 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:47.331 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.192:6836/1332439881,v1:172.21.15.192:6837/1332439881] up:boot 2023-10-26T06:44:47.331 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: fsmap 7 up:standby 2023-10-26T06:44:47.331 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-26T06:44:47.332 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.332 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.332 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.332 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:47.332 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:47.333 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:47.333 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: Deploying daemon mds.h on smithi135 2023-10-26T06:44:47.333 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:47 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/315571671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:47.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:47.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.192:6836/1332439881,v1:172.21.15.192:6837/1332439881] up:boot 2023-10-26T06:44:47.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: fsmap 7 up:standby 2023-10-26T06:44:47.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-26T06:44:47.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:47.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:47.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:47.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: Deploying daemon mds.h on smithi135 2023-10-26T06:44:47.398 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:47 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/315571671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.192:6836/1332439881,v1:172.21.15.192:6837/1332439881] up:boot 2023-10-26T06:44:47.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: fsmap 7 up:standby 2023-10-26T06:44:47.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-26T06:44:47.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:47.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:47.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:47.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:47.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: Deploying daemon mds.h on smithi135 2023-10-26T06:44:47.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:47 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/315571671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:47.946 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 06:44:47 smithi135 systemd[1]: Starting Ceph mds.h for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:48.121 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:48.198 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 06:44:47 smithi135 podman[169853]: 2023-10-26T06:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 -- 172.21.15.89:0/3341930604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe000101060 msgr2=0x7fe00010d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 --2- 172.21.15.89:0/3341930604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe000101060 0x7fe00010d570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009f90 tx=0x7fdff800b330 comp rx=0 tx=0).stop 2023-10-26T06:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 -- 172.21.15.89:0/3341930604 shutdown_connections 2023-10-26T06:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 --2- 172.21.15.89:0/3341930604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe000101060 0x7fe00010d570 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 --2- 172.21.15.89:0/3341930604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0001066b0 0x7fe000106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 --2- 172.21.15.89:0/3341930604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0001006a0 0x7fe000100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 -- 172.21.15.89:0/3341930604 >> 172.21.15.89:0/3341930604 conn(0x7fe0000fc1d0 msgr2=0x7fe0000fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.332+0000 7fe006dff700 1 -- 172.21.15.89:0/3341930604 shutdown_connections 2023-10-26T06:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.333+0000 7fe006dff700 1 -- 172.21.15.89:0/3341930604 wait complete. 2023-10-26T06:44:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.333+0000 7fe006dff700 1 Processor -- start 2023-10-26T06:44:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe006dff700 1 -- start start 2023-10-26T06:44:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe006dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0001006a0 0x7fe00019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe006dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe000101060 0x7fe00019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe006dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0001066b0 0x7fe000198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe006dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0001992b0 con 0x7fe0001066b0 2023-10-26T06:44:48.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe006dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000199420 con 0x7fe0001006a0 2023-10-26T06:44:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe006dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0001995e0 con 0x7fe000101060 2023-10-26T06:44:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe00539c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0001066b0 0x7fe000198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe00539c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0001066b0 0x7fe000198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58686/0 (socket says 172.21.15.89:58686) 2023-10-26T06:44:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe00539c700 1 -- 172.21.15.89:0/2693211181 learned_addr learned my addr 172.21.15.89:0/2693211181 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:48.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fe004b9b700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0001006a0 0x7fe00019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.334+0000 7fdffffff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe000101060 0x7fe00019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe004b9b700 1 -- 172.21.15.89:0/2693211181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe000101060 msgr2=0x7fe00019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe004b9b700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe000101060 0x7fe00019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe004b9b700 1 -- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0001066b0 msgr2=0x7fe000198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe004b9b700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0001066b0 0x7fe000198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe004b9b700 1 -- 172.21.15.89:0/2693211181 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0095f0 con 0x7fe0001006a0 2023-10-26T06:44:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe00539c700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0001066b0 0x7fe000198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe004b9b700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0001006a0 0x7fe00019ea40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9f0 tx=0x7fdff400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:48.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fdffdffb700 1 -- 172.21.15.89:0/2693211181 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400c9c0 con 0x7fe0001006a0 2023-10-26T06:44:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fdffdffb700 1 -- 172.21.15.89:0/2693211181 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdff4010430 con 0x7fe0001006a0 2023-10-26T06:44:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff8009c40 con 0x7fe0001006a0 2023-10-26T06:44:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fdffdffb700 1 -- 172.21.15.89:0/2693211181 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4018680 con 0x7fe0001006a0 2023-10-26T06:44:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.335+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0001a59c0 con 0x7fe0001006a0 2023-10-26T06:44:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.336+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe000101de0 con 0x7fe0001006a0 2023-10-26T06:44:48.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.338+0000 7fdffdffb700 1 -- 172.21.15.89:0/2693211181 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fdff4010860 con 0x7fe0001006a0 2023-10-26T06:44:48.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.339+0000 7fdffdffb700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdff00a1550 0x7fdff00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:48.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.339+0000 7fdffdffb700 1 -- 172.21.15.89:0/2693211181 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fdff4014070 con 0x7fe0001006a0 2023-10-26T06:44:48.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.339+0000 7fdffffff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdff00a1550 0x7fdff00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:48.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.340+0000 7fdffffff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdff00a1550 0x7fdff00a3a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe00019a250 tx=0x7fdfec009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:48.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.343+0000 7fdffdffb700 1 -- 172.21.15.89:0/2693211181 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdff4052100 con 0x7fe0001006a0 2023-10-26T06:44:48.467 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 06:44:48 smithi135 bash[169853]: c62a16d96d25461c3522645ab128d420e53a48c398945534797260bbf0948155 2023-10-26T06:44:48.467 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 06:44:48 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-h[169869]: starting mds.h at 2023-10-26T06:44:48.468 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 06:44:48 smithi135 systemd[1]: Started Ceph mds.h for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.591+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe000110620 con 0x7fe0001006a0 2023-10-26T06:44:48.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.592+0000 7fdffdffb700 1 -- 172.21.15.89:0/2693211181 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7fdff40963e0 con 0x7fe0001006a0 2023-10-26T06:44:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:48.594 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-26T06:44:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdff00a1550 msgr2=0x7fdff00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdff00a1550 0x7fdff00a3a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe00019a250 tx=0x7fdfec009310 comp rx=0 tx=0).stop 2023-10-26T06:44:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0001006a0 msgr2=0x7fe00019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0001006a0 0x7fe00019ea40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9f0 tx=0x7fdff400ed00 comp rx=0 tx=0).stop 2023-10-26T06:44:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 shutdown_connections 2023-10-26T06:44:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdff00a1550 0x7fdff00a3a10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0001066b0 0x7fe000198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0001006a0 0x7fe00019ea40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 --2- 172.21.15.89:0/2693211181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe000101060 0x7fe00019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 >> 172.21.15.89:0/2693211181 conn(0x7fe0000fc1d0 msgr2=0x7fe00010b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 shutdown_connections 2023-10-26T06:44:48.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:48.595+0000 7fe006dff700 1 -- 172.21.15.89:0/2693211181 wait complete. 2023-10-26T06:44:48.599 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2023-10-26T06:44:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:49.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:49.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2693211181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:49.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.135:6836/1484649325,v1:172.21.15.135:6837/1484649325] up:boot 2023-10-26T06:44:49.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: fsmap 8 up:standby 2023-10-26T06:44:49.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:49 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-26T06:44:49.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:49.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:49.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:49.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:49.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2693211181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:49.544 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.135:6836/1484649325,v1:172.21.15.135:6837/1484649325] up:boot 2023-10-26T06:44:49.544 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: fsmap 8 up:standby 2023-10-26T06:44:49.544 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:49 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-26T06:44:49.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:49.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:49.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:49.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:49.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:49.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2693211181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:49.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.135:6836/1484649325,v1:172.21.15.135:6837/1484649325] up:boot 2023-10-26T06:44:49.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: fsmap 8 up:standby 2023-10-26T06:44:49.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:49 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-26T06:44:50.146 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:44:49 smithi192 systemd[1]: Starting Ceph mds.i for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:50.478 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:50 smithi192 ceph-mon[131661]: Deploying daemon mds.i on smithi192 2023-10-26T06:44:50.479 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:44:50 smithi192 podman[171565]: 2023-10-26T06:44:50.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:50.683 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:50 smithi089 ceph-mon[95031]: Deploying daemon mds.i on smithi192 2023-10-26T06:44:50.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:50 smithi135 ceph-mon[131403]: Deploying daemon mds.i on smithi192 2023-10-26T06:44:50.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 -- 172.21.15.89:0/3427740930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 msgr2=0x7f2104106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:50.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 --2- 172.21.15.89:0/3427740930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 0x7f2104106a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009990 tx=0x7f20f4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:50.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 -- 172.21.15.89:0/3427740930 shutdown_connections 2023-10-26T06:44:50.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 --2- 172.21.15.89:0/3427740930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2104101060 0x7f210410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 --2- 172.21.15.89:0/3427740930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 0x7f2104106a90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 --2- 172.21.15.89:0/3427740930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21041006a0 0x7f2104100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 -- 172.21.15.89:0/3427740930 >> 172.21.15.89:0/3427740930 conn(0x7f21040fc1d0 msgr2=0x7f21040fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.827+0000 7f2109fff700 1 -- 172.21.15.89:0/3427740930 shutdown_connections 2023-10-26T06:44:50.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.828+0000 7f2109fff700 1 -- 172.21.15.89:0/3427740930 wait complete. 2023-10-26T06:44:50.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.829+0000 7f2109fff700 1 Processor -- start 2023-10-26T06:44:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.829+0000 7f2109fff700 1 -- start start 2023-10-26T06:44:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2109fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21041006a0 0x7f210419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2109fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2104101060 0x7f210419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2109fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 0x7f2104196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:50.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2109fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2104196fc0 con 0x7f2104101060 2023-10-26T06:44:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2109fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2104197130 con 0x7f21041006a0 2023-10-26T06:44:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2109fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21041972f0 con 0x7f21041066b0 2023-10-26T06:44:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2104101060 0x7f210419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2104101060 0x7f210419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43522/0 (socket says 172.21.15.89:43522) 2023-10-26T06:44:50.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2103fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 0x7f2104196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 -- 172.21.15.89:0/2615666489 learned_addr learned my addr 172.21.15.89:0/2615666489 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f21037fe700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21041006a0 0x7f210419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 -- 172.21.15.89:0/2615666489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 msgr2=0x7f2104196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 0x7f2104196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:50.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 -- 172.21.15.89:0/2615666489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21041006a0 msgr2=0x7f210419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21041006a0 0x7f210419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2102ffd700 1 -- 172.21.15.89:0/2615666489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40095f0 con 0x7f2104101060 2023-10-26T06:44:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.830+0000 7f2103fff700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 0x7f2104196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.831+0000 7f2102ffd700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2104101060 0x7f210419cd70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20f8012db0 tx=0x7f20f800c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:50.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.831+0000 7f2100ff9700 1 -- 172.21.15.89:0/2615666489 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8009790 con 0x7f2104101060 2023-10-26T06:44:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.831+0000 7f2100ff9700 1 -- 172.21.15.89:0/2615666489 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f801a8f0 con 0x7f2104101060 2023-10-26T06:44:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.831+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21041975d0 con 0x7f2104101060 2023-10-26T06:44:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.831+0000 7f2100ff9700 1 -- 172.21.15.89:0/2615666489 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8018990 con 0x7f2104101060 2023-10-26T06:44:50.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.831+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2104072c00 con 0x7f2104101060 2023-10-26T06:44:50.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.832+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2104108c30 con 0x7f2104101060 2023-10-26T06:44:50.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.834+0000 7f2100ff9700 1 -- 172.21.15.89:0/2615666489 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f20f801aa60 con 0x7f2104101060 2023-10-26T06:44:50.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.835+0000 7f2100ff9700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f20ec0a14d0 0x7f20ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.835+0000 7f2100ff9700 1 -- 172.21.15.89:0/2615666489 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f20f80d0930 con 0x7f2104101060 2023-10-26T06:44:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.836+0000 7f21037fe700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f20ec0a14d0 0x7f20ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:50.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.836+0000 7f21037fe700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f20ec0a14d0 0x7f20ec0a3990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20f400b2e0 tx=0x7f20f4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:50.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:50.840+0000 7f2100ff9700 1 -- 172.21.15.89:0/2615666489 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f20f809a2c0 con 0x7f2104101060 2023-10-26T06:44:50.895 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:44:50 smithi192 ceph-mds[171606]: -- 172.21.15.192:0/1239257303 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559d7b3351e0 con 0x559d7b30b000 2023-10-26T06:44:50.895 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:44:50 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-i[171581]: starting mds.i at 2023-10-26T06:44:50.895 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:44:50 smithi192 bash[171565]: 135c63b907ff1ec1be135aff241c090ac1c398c3093649376ff6efff0902553a 2023-10-26T06:44:50.896 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:44:50 smithi192 systemd[1]: Started Ceph mds.i for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.114+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2104101de0 con 0x7f2104101060 2023-10-26T06:44:51.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.115+0000 7f2100ff9700 1 -- 172.21.15.89:0/2615666489 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6874 (secure 0 0 0) 0x7f20f809d8e0 con 0x7f2104101060 2023-10-26T06:44:51.116 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:51.116 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-10-26T06:44:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f20ec0a14d0 msgr2=0x7f20ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:51.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f20ec0a14d0 0x7f20ec0a3990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20f400b2e0 tx=0x7f20f4005fd0 comp rx=0 tx=0).stop 2023-10-26T06:44:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2104101060 msgr2=0x7f210419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2104101060 0x7f210419cd70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20f8012db0 tx=0x7f20f800c5b0 comp rx=0 tx=0).stop 2023-10-26T06:44:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 shutdown_connections 2023-10-26T06:44:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f20ec0a14d0 0x7f20ec0a3990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:51.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21041066b0 0x7f2104196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21041006a0 0x7f210419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 --2- 172.21.15.89:0/2615666489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2104101060 0x7f210419cd70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 >> 172.21.15.89:0/2615666489 conn(0x7f21040fc1d0 msgr2=0x7f21040fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 shutdown_connections 2023-10-26T06:44:51.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:51.117+0000 7f2109fff700 1 -- 172.21.15.89:0/2615666489 wait complete. 2023-10-26T06:44:51.121 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 9 2023-10-26T06:44:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:51.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:51 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2615666489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:51.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:51.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:51.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:51.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:51 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2615666489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:51.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:51.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:51.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:51.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:51 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2615666489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:52.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:52 smithi089 ceph-mon[95031]: Deploying daemon mds.g on smithi089 2023-10-26T06:44:52.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:52 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.192:6838/2442523019,v1:172.21.15.192:6839/2442523019] up:boot 2023-10-26T06:44:52.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:52 smithi089 ceph-mon[95031]: fsmap 9 up:standby 2023-10-26T06:44:52.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:52 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-26T06:44:52.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:52 smithi089 ceph-mon[95031]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:52.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:52 smithi192 ceph-mon[131661]: Deploying daemon mds.g on smithi089 2023-10-26T06:44:52.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:52 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.192:6838/2442523019,v1:172.21.15.192:6839/2442523019] up:boot 2023-10-26T06:44:52.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:52 smithi192 ceph-mon[131661]: fsmap 9 up:standby 2023-10-26T06:44:52.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:52 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-26T06:44:52.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:52 smithi192 ceph-mon[131661]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:52.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:52 smithi135 ceph-mon[131403]: Deploying daemon mds.g on smithi089 2023-10-26T06:44:52.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:52 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.192:6838/2442523019,v1:172.21.15.192:6839/2442523019] up:boot 2023-10-26T06:44:52.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:52 smithi135 ceph-mon[131403]: fsmap 9 up:standby 2023-10-26T06:44:52.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:52 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-26T06:44:52.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:52 smithi135 ceph-mon[131403]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:53.039 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 06:44:52 smithi089 systemd[1]: Starting Ceph mds.g for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:53.139 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.347+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/2445656297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc41066b0 msgr2=0x7ffbc4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.347+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/2445656297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc41066b0 0x7ffbc4106a90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009990 tx=0x7ffbc0009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.348+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/2445656297 shutdown_connections 2023-10-26T06:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.348+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/2445656297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc41066b0 0x7ffbc4106a90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.348+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/2445656297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc4101060 0x7ffbc410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.348+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/2445656297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbc41006a0 0x7ffbc4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.348+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/2445656297 >> 172.21.15.89:0/2445656297 conn(0x7ffbc40fc170 msgr2=0x7ffbc40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.348+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/2445656297 shutdown_connections 2023-10-26T06:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.348+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/2445656297 wait complete. 2023-10-26T06:44:53.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 Processor -- start 2023-10-26T06:44:53.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 -- start start 2023-10-26T06:44:53.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc41006a0 0x7ffbc419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:53.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc4101060 0x7ffbc419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:53.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbc41066b0 0x7ffbc4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4199200 con 0x7ffbc4101060 2023-10-26T06:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4199370 con 0x7ffbc41066b0 2023-10-26T06:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbcb7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4199530 con 0x7ffbc41006a0 2023-10-26T06:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbc8d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc4101060 0x7ffbc419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbc9d9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbc41066b0 0x7ffbc4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbc9d9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbc41066b0 0x7ffbc4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:46708/0 (socket says 172.21.15.89:46708) 2023-10-26T06:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbc8d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc4101060 0x7ffbc419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43550/0 (socket says 172.21.15.89:43550) 2023-10-26T06:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbc9d9c700 1 -- 172.21.15.89:0/4211942001 learned_addr learned my addr 172.21.15.89:0/4211942001 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.349+0000 7ffbc959b700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc41006a0 0x7ffbc419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbc8d9a700 1 -- 172.21.15.89:0/4211942001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc41006a0 msgr2=0x7ffbc419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbc8d9a700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc41006a0 0x7ffbc419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbc8d9a700 1 -- 172.21.15.89:0/4211942001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbc41066b0 msgr2=0x7ffbc4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbc8d9a700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbc41066b0 0x7ffbc4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbc8d9a700 1 -- 172.21.15.89:0/4211942001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00095f0 con 0x7ffbc4101060 2023-10-26T06:44:53.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbc8d9a700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc4101060 0x7ffbc419efb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400e9f0 tx=0x7ffbb400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:53.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbba7fc700 1 -- 172.21.15.89:0/4211942001 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400c9c0 con 0x7ffbc4101060 2023-10-26T06:44:53.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbba7fc700 1 -- 172.21.15.89:0/4211942001 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb4010430 con 0x7ffbc4101060 2023-10-26T06:44:53.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc4199810 con 0x7ffbc4101060 2023-10-26T06:44:53.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbba7fc700 1 -- 172.21.15.89:0/4211942001 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4018680 con 0x7ffbc4101060 2023-10-26T06:44:53.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.350+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc41a5a60 con 0x7ffbc4101060 2023-10-26T06:44:53.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.351+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc4101de0 con 0x7ffbc4101060 2023-10-26T06:44:53.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.353+0000 7ffbba7fc700 1 -- 172.21.15.89:0/4211942001 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ffbb4010860 con 0x7ffbc4101060 2023-10-26T06:44:53.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.354+0000 7ffbba7fc700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffbb00a1550 0x7ffbb00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:53.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.355+0000 7ffbba7fc700 1 -- 172.21.15.89:0/4211942001 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7ffbb4014070 con 0x7ffbc4101060 2023-10-26T06:44:53.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.355+0000 7ffbc959b700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffbb00a1550 0x7ffbb00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.355+0000 7ffbc959b700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffbb00a1550 0x7ffbb00a3a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0006010 tx=0x7ffbc000b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:53.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.359+0000 7ffbba7fc700 1 -- 172.21.15.89:0/4211942001 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffbb4052100 con 0x7ffbc4101060 2023-10-26T06:44:53.531 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 06:44:53 smithi089 podman[150477]: 2023-10-26T06:44:53.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.625+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc4067530 con 0x7ffbc4101060 2023-10-26T06:44:53.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.626+0000 7ffbba7fc700 1 -- 172.21.15.89:0/4211942001 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7ffbb4096460 con 0x7ffbc4101060 2023-10-26T06:44:53.628 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:53.628 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24703,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-26T06:44:53.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffbb00a1550 msgr2=0x7ffbb00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:53.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffbb00a1550 0x7ffbb00a3a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0006010 tx=0x7ffbc000b260 comp rx=0 tx=0).stop 2023-10-26T06:44:53.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc4101060 msgr2=0x7ffbc419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:53.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc4101060 0x7ffbc419efb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400e9f0 tx=0x7ffbb400ed00 comp rx=0 tx=0).stop 2023-10-26T06:44:53.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 shutdown_connections 2023-10-26T06:44:53.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffbb00a1550 0x7ffbb00a3a10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbc41066b0 0x7ffbc4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc41006a0 0x7ffbc419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 --2- 172.21.15.89:0/4211942001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffbc4101060 0x7ffbc419efb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:53.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.629+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 >> 172.21.15.89:0/4211942001 conn(0x7ffbc40fc170 msgr2=0x7ffbc410bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:53.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.630+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 shutdown_connections 2023-10-26T06:44:53.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:53.630+0000 7ffbcb7ff700 1 -- 172.21.15.89:0/4211942001 wait complete. 2023-10-26T06:44:53.633 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 11 2023-10-26T06:44:53.789 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 06:44:53 smithi089 bash[150477]: b68d08e7bce8b8013fed5c81851aadd72a909443016f0b2e2445f46b2597c8b8 2023-10-26T06:44:53.790 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 06:44:53 smithi089 ceph-mds[150560]: -- 172.21.15.89:0/2926169082 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c2732d1e0 con 0x555c27304c00 2023-10-26T06:44:53.790 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 06:44:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-g[150536]: starting mds.g at 2023-10-26T06:44:53.790 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 06:44:53 smithi089 systemd[1]: Started Ceph mds.g for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:54.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.89:6838/1053435135,v1:172.21.15.89:6839/1053435135] up:boot 2023-10-26T06:44:54.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: fsmap 10 up:standby 2023-10-26T06:44:54.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-26T06:44:54.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4211942001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:54.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:54.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:54.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:54.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: Deploying daemon mds.a on smithi089 2023-10-26T06:44:54.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:54 smithi192 ceph-mon[131661]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:54.958 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.89:6838/1053435135,v1:172.21.15.89:6839/1053435135] up:boot 2023-10-26T06:44:54.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: fsmap 10 up:standby 2023-10-26T06:44:54.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-26T06:44:54.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4211942001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:54.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.960 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.960 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:54.960 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:54.960 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:54.960 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: Deploying daemon mds.a on smithi089 2023-10-26T06:44:54.961 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:54 smithi089 ceph-mon[95031]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:54.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.89:6838/1053435135,v1:172.21.15.89:6839/1053435135] up:boot 2023-10-26T06:44:54.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: fsmap 10 up:standby 2023-10-26T06:44:54.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-26T06:44:54.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4211942001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:54.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:54.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:54.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:54.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:54.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: Deploying daemon mds.a on smithi089 2023-10-26T06:44:54.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:54 smithi135 ceph-mon[131403]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:55.654 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:55.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.889+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2862665841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7740734b0 msgr2=0x7ff774111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:55.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.889+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2862665841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7740734b0 0x7ff774111170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff768009990 tx=0x7ff768009ca0 comp rx=0 tx=0).stop 2023-10-26T06:44:55.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.889+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2862665841 shutdown_connections 2023-10-26T06:44:55.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.889+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2862665841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7741116b0 0x7ff774113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:55.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.889+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2862665841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff774072b90 0x7ff774072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:55.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.889+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2862665841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7740734b0 0x7ff774111170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:55.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.889+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2862665841 >> 172.21.15.89:0/2862665841 conn(0x7ff7740fc1d0 msgr2=0x7ff7740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:55.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.890+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2862665841 shutdown_connections 2023-10-26T06:44:55.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.890+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2862665841 wait complete. 2023-10-26T06:44:55.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.890+0000 7ff77a9ff700 1 Processor -- start 2023-10-26T06:44:55.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff77a9ff700 1 -- start start 2023-10-26T06:44:55.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff77a9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff774072b90 0x7ff77419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:55.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff77a9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7740734b0 0x7ff77419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:55.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff77a9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7741116b0 0x7ff7741a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:55.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff77a9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741a2110 con 0x7ff774072b90 2023-10-26T06:44:55.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff77a9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741a2280 con 0x7ff7741116b0 2023-10-26T06:44:55.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff77a9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741a2440 con 0x7ff7740734b0 2023-10-26T06:44:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff773fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff774072b90 0x7ff77419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff773fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff774072b90 0x7ff77419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43596/0 (socket says 172.21.15.89:43596) 2023-10-26T06:44:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff773fff700 1 -- 172.21.15.89:0/2178371782 learned_addr learned my addr 172.21.15.89:0/2178371782 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff778f9c700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7741116b0 0x7ff7741a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.891+0000 7ff7737fe700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7740734b0 0x7ff77419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:55.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff773fff700 1 -- 172.21.15.89:0/2178371782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7740734b0 msgr2=0x7ff77419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:55.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff773fff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7740734b0 0x7ff77419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:55.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff773fff700 1 -- 172.21.15.89:0/2178371782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7741116b0 msgr2=0x7ff7741a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:55.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff773fff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7741116b0 0x7ff7741a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:55.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff773fff700 1 -- 172.21.15.89:0/2178371782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680095f0 con 0x7ff774072b90 2023-10-26T06:44:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff773fff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff774072b90 0x7ff77419d1f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff76400e9a0 tx=0x7ff76400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff7717fa700 1 -- 172.21.15.89:0/2178371782 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764009790 con 0x7ff774072b90 2023-10-26T06:44:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff7717fa700 1 -- 172.21.15.89:0/2178371782 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff764010430 con 0x7ff774072b90 2023-10-26T06:44:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7741a2720 con 0x7ff774072b90 2023-10-26T06:44:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff7717fa700 1 -- 172.21.15.89:0/2178371782 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764018680 con 0x7ff774072b90 2023-10-26T06:44:55.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.892+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7741a9f60 con 0x7ff774072b90 2023-10-26T06:44:55.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.893+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff774103bd0 con 0x7ff774072b90 2023-10-26T06:44:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.895+0000 7ff7717fa700 1 -- 172.21.15.89:0/2178371782 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff7640105a0 con 0x7ff774072b90 2023-10-26T06:44:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.896+0000 7ff7717fa700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff75c0a14d0 0x7ff75c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.897+0000 7ff7717fa700 1 -- 172.21.15.89:0/2178371782 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7ff764014070 con 0x7ff774072b90 2023-10-26T06:44:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.897+0000 7ff7737fe700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff75c0a14d0 0x7ff75c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.897+0000 7ff7737fe700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff75c0a14d0 0x7ff75c0a3990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff77419e740 tx=0x7ff76800b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:55.901+0000 7ff7717fa700 1 -- 172.21.15.89:0/2178371782 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff764052100 con 0x7ff774072b90 2023-10-26T06:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.171+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7740742d0 con 0x7ff774072b90 2023-10-26T06:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.171+0000 7ff7717fa700 1 -- 172.21.15.89:0/2178371782 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7ff764096560 con 0x7ff774072b90 2023-10-26T06:44:56.173 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:56.173 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24703,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-26T06:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff75c0a14d0 msgr2=0x7ff75c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff75c0a14d0 0x7ff75c0a3990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff77419e740 tx=0x7ff76800b260 comp rx=0 tx=0).stop 2023-10-26T06:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff774072b90 msgr2=0x7ff77419d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff774072b90 0x7ff77419d1f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff76400e9a0 tx=0x7ff76400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 shutdown_connections 2023-10-26T06:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff75c0a14d0 0x7ff75c0a3990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff774072b90 0x7ff77419d1f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7741116b0 0x7ff7741a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 --2- 172.21.15.89:0/2178371782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7740734b0 0x7ff77419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 >> 172.21.15.89:0/2178371782 conn(0x7ff7740fc1d0 msgr2=0x7ff7740fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 shutdown_connections 2023-10-26T06:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:56.174+0000 7ff77a9ff700 1 -- 172.21.15.89:0/2178371782 wait complete. 2023-10-26T06:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 11 2023-10-26T06:44:56.289 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 06:44:55 smithi089 systemd[1]: Starting Ceph mds.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:56.766 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 06:44:56 smithi089 podman[150907]: 2023-10-26T06:44:56.767 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 06:44:56 smithi089 bash[150907]: 8408a6e86e9bc5b6368370244313917ef46864735fd01f3b717139d91802f7ca 2023-10-26T06:44:56.767 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 06:44:56 smithi089 ceph-mds[150947]: -- 172.21.15.89:0/1278988359 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5556653151e0 con 0x5556652ecc00 2023-10-26T06:44:56.767 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 06:44:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-a[150923]: starting mds.a at 2023-10-26T06:44:57.039 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 06:44:56 smithi089 systemd[1]: Started Ceph mds.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:57.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:57.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2178371782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:57.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:57.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:57.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:57 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2178371782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:57.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:57.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:57.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:57 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:57.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2178371782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:57.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:57.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-26T06:44:57.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-26T06:44:57.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:57 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:44:58.091 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:58 smithi135 ceph-mon[131403]: Deploying daemon mds.b on smithi135 2023-10-26T06:44:58.091 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:58 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.89:6840/3761987618,v1:172.21.15.89:6841/3761987618] up:boot 2023-10-26T06:44:58.092 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:58 smithi135 ceph-mon[131403]: fsmap 11 up:standby 2023-10-26T06:44:58.092 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:58 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-26T06:44:58.197 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:44:58.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:58 smithi192 ceph-mon[131661]: Deploying daemon mds.b on smithi135 2023-10-26T06:44:58.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:58 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.89:6840/3761987618,v1:172.21.15.89:6841/3761987618] up:boot 2023-10-26T06:44:58.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:58 smithi192 ceph-mon[131661]: fsmap 11 up:standby 2023-10-26T06:44:58.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:58 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-26T06:44:58.403 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:44:58 smithi135 systemd[1]: Starting Ceph mds.b for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:44:58.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 -- 172.21.15.89:0/1301626364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc101060 msgr2=0x7f9bcc10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:58.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/1301626364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc101060 0x7f9bcc10d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b0a0 tx=0x7f9bc400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 -- 172.21.15.89:0/1301626364 shutdown_connections 2023-10-26T06:44:58.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/1301626364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bcc1066b0 0x7f9bcc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/1301626364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc101060 0x7f9bcc10d570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/1301626364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bcc1006a0 0x7f9bcc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 -- 172.21.15.89:0/1301626364 >> 172.21.15.89:0/1301626364 conn(0x7f9bcc0fc170 msgr2=0x7f9bcc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:58.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 -- 172.21.15.89:0/1301626364 shutdown_connections 2023-10-26T06:44:58.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.409+0000 7f9bd29ff700 1 -- 172.21.15.89:0/1301626364 wait complete. 2023-10-26T06:44:58.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.410+0000 7f9bd29ff700 1 Processor -- start 2023-10-26T06:44:58.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.410+0000 7f9bd29ff700 1 -- start start 2023-10-26T06:44:58.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.410+0000 7f9bd29ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc1006a0 0x7f9bcc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.410+0000 7f9bd29ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bcc101060 0x7f9bcc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bd29ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bcc1066b0 0x7f9bcc196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bd29ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc196ee0 con 0x7f9bcc101060 2023-10-26T06:44:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bd29ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc197050 con 0x7f9bcc1006a0 2023-10-26T06:44:58.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bd29ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc197210 con 0x7f9bcc1066b0 2023-10-26T06:44:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bc3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bcc101060 0x7f9bcc19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc1006a0 0x7f9bcc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc1006a0 0x7f9bcc19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:46776/0 (socket says 172.21.15.89:46776) 2023-10-26T06:44:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 -- 172.21.15.89:0/2642880180 learned_addr learned my addr 172.21.15.89:0/2642880180 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:44:58.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bd0f9c700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bcc1066b0 0x7f9bcc196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 -- 172.21.15.89:0/2642880180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bcc1066b0 msgr2=0x7f9bcc196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bcc1066b0 0x7f9bcc196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 -- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bcc101060 msgr2=0x7f9bcc19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bcc101060 0x7f9bcc19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 -- 172.21.15.89:0/2642880180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4009d00 con 0x7f9bcc1006a0 2023-10-26T06:44:58.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bc3fff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bcc101060 0x7f9bcc19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:44:58.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bd0f9c700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bcc1066b0 0x7f9bcc196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:44:58.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.411+0000 7f9bcbfff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc1006a0 0x7f9bcc19c6f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00e9a0 tx=0x7f9bbc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:58.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.412+0000 7f9bc9ffb700 1 -- 172.21.15.89:0/2642880180 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc009790 con 0x7f9bcc1006a0 2023-10-26T06:44:58.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.412+0000 7f9bc9ffb700 1 -- 172.21.15.89:0/2642880180 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bbc00cbd0 con 0x7f9bcc1006a0 2023-10-26T06:44:58.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.412+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc1974f0 con 0x7f9bcc1006a0 2023-10-26T06:44:58.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.412+0000 7f9bc9ffb700 1 -- 172.21.15.89:0/2642880180 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc018680 con 0x7f9bcc1006a0 2023-10-26T06:44:58.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.412+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc1a3790 con 0x7f9bcc1006a0 2023-10-26T06:44:58.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.413+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc108cc0 con 0x7f9bcc1006a0 2023-10-26T06:44:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.415+0000 7f9bc9ffb700 1 -- 172.21.15.89:0/2642880180 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9bbc00cd40 con 0x7f9bcc1006a0 2023-10-26T06:44:58.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.416+0000 7f9bc9ffb700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9bb80a1550 0x7f9bb80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:44:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.416+0000 7f9bc9ffb700 1 -- 172.21.15.89:0/2642880180 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f9bbc014070 con 0x7f9bcc1006a0 2023-10-26T06:44:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.416+0000 7f9bc3fff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9bb80a1550 0x7f9bb80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:44:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.417+0000 7f9bc3fff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9bb80a1550 0x7f9bb80a3a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc197fb0 tx=0x7f9bb4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:44:58.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.418+0000 7f9bc9ffb700 1 -- 172.21.15.89:0/2642880180 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9bbc096840 con 0x7f9bcc1006a0 2023-10-26T06:44:58.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:58 smithi089 ceph-mon[95031]: Deploying daemon mds.b on smithi135 2023-10-26T06:44:58.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:58 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.89:6840/3761987618,v1:172.21.15.89:6841/3761987618] up:boot 2023-10-26T06:44:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:58 smithi089 ceph-mon[95031]: fsmap 11 up:standby 2023-10-26T06:44:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:58 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-26T06:44:58.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.666+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bcc101de0 con 0x7f9bcc1006a0 2023-10-26T06:44:58.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.667+0000 7f9bc9ffb700 1 -- 172.21.15.89:0/2642880180 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f9bbc099e60 con 0x7f9bcc1006a0 2023-10-26T06:44:58.668 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:44:58.669 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24703,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-26T06:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.669+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9bb80a1550 msgr2=0x7f9bb80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.669+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9bb80a1550 0x7f9bb80a3a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc197fb0 tx=0x7f9bb4009260 comp rx=0 tx=0).stop 2023-10-26T06:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.669+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc1006a0 msgr2=0x7f9bcc19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.669+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc1006a0 0x7f9bcc19c6f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00e9a0 tx=0x7f9bbc00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 shutdown_connections 2023-10-26T06:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9bb80a1550 0x7f9bb80a3a10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bcc1066b0 0x7f9bcc196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bcc1006a0 0x7f9bcc19c6f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 --2- 172.21.15.89:0/2642880180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bcc101060 0x7f9bcc19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 >> 172.21.15.89:0/2642880180 conn(0x7f9bcc0fc170 msgr2=0x7f9bcc0fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 shutdown_connections 2023-10-26T06:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:44:58.670+0000 7f9bd29ff700 1 -- 172.21.15.89:0/2642880180 wait complete. 2023-10-26T06:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 12 2023-10-26T06:44:58.968 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:44:58 smithi135 podman[170196]: 2023-10-26T06:44:58.968 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:44:58 smithi135 bash[170196]: 4088f71f66d321b67bf707312d5f6564898bf688d72583825d747a3bdcd77b7f 2023-10-26T06:44:58.968 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:44:58 smithi135 ceph-mds[170236]: -- 172.21.15.135:0/1575988362 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56427fd591e0 con 0x56427fd2e800 2023-10-26T06:44:58.968 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:44:58 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b[170212]: starting mds.b at 2023-10-26T06:44:59.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:59 smithi192 ceph-mon[131661]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:59.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:59 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2642880180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:59.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:59 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:59.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:44:59 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:59.467 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:44:58 smithi135 systemd[1]: Started Ceph mds.b for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:44:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:59 smithi135 ceph-mon[131403]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:59.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:59 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2642880180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:59.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:59 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:59.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:44:59 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:59 smithi089 ceph-mon[95031]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:44:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:59 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2642880180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:44:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:59 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:44:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:44:59 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:00 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] up:boot 2023-10-26T06:45:00.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:00 smithi192 ceph-mon[131661]: fsmap 12 up:standby 2023-10-26T06:45:00.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-26T06:45:00.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:00 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:45:00.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:00 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] up:boot 2023-10-26T06:45:00.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:00 smithi135 ceph-mon[131403]: fsmap 12 up:standby 2023-10-26T06:45:00.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-26T06:45:00.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:00 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:45:00.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:00 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] up:boot 2023-10-26T06:45:00.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:00 smithi089 ceph-mon[95031]: fsmap 12 up:standby 2023-10-26T06:45:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-26T06:45:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:00 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:45:00.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:45:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 -- 172.21.15.89:0/4153163041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8100e60 msgr2=0x7fefc810d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 --2- 172.21.15.89:0/4153163041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8100e60 0x7fefc810d320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fefc000b0d0 tx=0x7fefc000b3e0 comp rx=0 tx=0).stop 2023-10-26T06:45:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 -- 172.21.15.89:0/4153163041 shutdown_connections 2023-10-26T06:45:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 --2- 172.21.15.89:0/4153163041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8100e60 0x7fefc810d320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 --2- 172.21.15.89:0/4153163041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc8106560 0x7fefc8106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 --2- 172.21.15.89:0/4153163041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81004a0 0x7fefc8100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 -- 172.21.15.89:0/4153163041 >> 172.21.15.89:0/4153163041 conn(0x7fefc80fbfb0 msgr2=0x7fefc80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 -- 172.21.15.89:0/4153163041 shutdown_connections 2023-10-26T06:45:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.901+0000 7fefcd39d700 1 -- 172.21.15.89:0/4153163041 wait complete. 2023-10-26T06:45:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.902+0000 7fefcd39d700 1 Processor -- start 2023-10-26T06:45:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.902+0000 7fefcd39d700 1 -- start start 2023-10-26T06:45:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefcd39d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81004a0 0x7fefc819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:00.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefcd39d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc8100e60 0x7fefc819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:00.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefcd39d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8106560 0x7fefc8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:00.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefcd39d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc81990f0 con 0x7fefc8106560 2023-10-26T06:45:00.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefcd39d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc8199260 con 0x7fefc81004a0 2023-10-26T06:45:00.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefcd39d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc8199420 con 0x7fefc8100e60 2023-10-26T06:45:00.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8106560 0x7fefc8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:00.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8106560 0x7fefc8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33590/0 (socket says 172.21.15.89:33590) 2023-10-26T06:45:00.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 -- 172.21.15.89:0/980311312 learned_addr learned my addr 172.21.15.89:0/980311312 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:00.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefc77fe700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc8100e60 0x7fefc819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:00.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefc7fff700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81004a0 0x7fefc819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:00.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 -- 172.21.15.89:0/980311312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc8100e60 msgr2=0x7fefc819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc8100e60 0x7fefc819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 -- 172.21.15.89:0/980311312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81004a0 msgr2=0x7fefc819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81004a0 0x7fefc819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:00.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefccb9c700 1 -- 172.21.15.89:0/980311312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc0009d00 con 0x7fefc8106560 2023-10-26T06:45:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.903+0000 7fefc7fff700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81004a0 0x7fefc819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.904+0000 7fefccb9c700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8106560 0x7fefc8198a90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefc000bcd0 tx=0x7fefc000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.904+0000 7fefc57fa700 1 -- 172.21.15.89:0/980311312 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000e040 con 0x7fefc8106560 2023-10-26T06:45:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.904+0000 7fefc57fa700 1 -- 172.21.15.89:0/980311312 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefc0005070 con 0x7fefc8106560 2023-10-26T06:45:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.904+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc81996a0 con 0x7fefc8106560 2023-10-26T06:45:00.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.904+0000 7fefc57fa700 1 -- 172.21.15.89:0/980311312 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc001b680 con 0x7fefc8106560 2023-10-26T06:45:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.904+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc81a5900 con 0x7fefc8106560 2023-10-26T06:45:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.905+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc8101bc0 con 0x7fefc8106560 2023-10-26T06:45:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.907+0000 7fefc57fa700 1 -- 172.21.15.89:0/980311312 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fefc0003a00 con 0x7fefc8106560 2023-10-26T06:45:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.909+0000 7fefc57fa700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fefb00a14d0 0x7fefb00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.909+0000 7fefc57fa700 1 -- 172.21.15.89:0/980311312 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fefc00cd560 con 0x7fefc8106560 2023-10-26T06:45:00.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.909+0000 7fefc7fff700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fefb00a14d0 0x7fefb00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:00.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.910+0000 7fefc7fff700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fefb00a14d0 0x7fefb00a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009cb0 tx=0x7fefb8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:00.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:00.914+0000 7fefc57fa700 1 -- 172.21.15.89:0/980311312 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fefc0096f80 con 0x7fefc8106560 2023-10-26T06:45:01.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.177+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefc80673e0 con 0x7fefc8106560 2023-10-26T06:45:01.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.178+0000 7fefc57fa700 1 -- 172.21.15.89:0/980311312 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7fefc0003cb0 con 0x7fefc8106560 2023-10-26T06:45:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:01.179 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24703,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24743,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-26T06:45:01.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fefb00a14d0 msgr2=0x7fefb00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fefb00a14d0 0x7fefb00a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009cb0 tx=0x7fefb8009260 comp rx=0 tx=0).stop 2023-10-26T06:45:01.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8106560 msgr2=0x7fefc8198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8106560 0x7fefc8198a90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefc000bcd0 tx=0x7fefc000bd00 comp rx=0 tx=0).stop 2023-10-26T06:45:01.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 shutdown_connections 2023-10-26T06:45:01.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fefb00a14d0 0x7fefb00a3990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fefc8106560 0x7fefc8198a90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81004a0 0x7fefc819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 --2- 172.21.15.89:0/980311312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc8100e60 0x7fefc819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 >> 172.21.15.89:0/980311312 conn(0x7fefc80fbfb0 msgr2=0x7fefc80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:01.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 shutdown_connections 2023-10-26T06:45:01.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.181+0000 7fefcd39d700 1 -- 172.21.15.89:0/980311312 wait complete. 2023-10-26T06:45:01.185 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 13 2023-10-26T06:45:01.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:45:01.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:01 smithi192 ceph-mon[131661]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:01.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 -- 172.21.15.89:0/3486719789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 msgr2=0x7fabc4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 --2- 172.21.15.89:0/3486719789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4100b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009990 tx=0x7fabb8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 -- 172.21.15.89:0/3486719789 shutdown_connections 2023-10-26T06:45:01.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 --2- 172.21.15.89:0/3486719789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc41066b0 0x7fabc4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 --2- 172.21.15.89:0/3486719789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabc4101060 0x7fabc410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 --2- 172.21.15.89:0/3486719789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4100b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 -- 172.21.15.89:0/3486719789 >> 172.21.15.89:0/3486719789 conn(0x7fabc40fc170 msgr2=0x7fabc40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:01.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 -- 172.21.15.89:0/3486719789 shutdown_connections 2023-10-26T06:45:01.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.410+0000 7fabc9bff700 1 -- 172.21.15.89:0/3486719789 wait complete. 2023-10-26T06:45:01.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.411+0000 7fabc9bff700 1 Processor -- start 2023-10-26T06:45:01.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.411+0000 7fabc9bff700 1 -- start start 2023-10-26T06:45:01.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc9bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc9bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc4101060 0x7fabc4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc9bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabc41066b0 0x7fabc419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc9bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc419dc80 con 0x7fabc4101060 2023-10-26T06:45:01.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc9bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc419ddf0 con 0x7fabc41066b0 2023-10-26T06:45:01.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc9bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc419dfb0 con 0x7fabc41006a0 2023-10-26T06:45:01.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:55118/0 (socket says 172.21.15.89:55118) 2023-10-26T06:45:01.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc37fe700 1 -- 172.21.15.89:0/2704722857 learned_addr learned my addr 172.21.15.89:0/2704722857 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:01.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc2ffd700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc4101060 0x7fabc4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.412+0000 7fabc3fff700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabc41066b0 0x7fabc419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc37fe700 1 -- 172.21.15.89:0/2704722857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabc41066b0 msgr2=0x7fabc419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc37fe700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabc41066b0 0x7fabc419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc37fe700 1 -- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc4101060 msgr2=0x7fabc4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc37fe700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc4101060 0x7fabc4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc37fe700 1 -- 172.21.15.89:0/2704722857 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80095f0 con 0x7fabc41006a0 2023-10-26T06:45:01.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc37fe700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4198e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e9a0 tx=0x7fabb400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:01.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc0ff9700 1 -- 172.21.15.89:0/2704722857 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4009790 con 0x7fabc41006a0 2023-10-26T06:45:01.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc0ff9700 1 -- 172.21.15.89:0/2704722857 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabb4010430 con 0x7fabc41006a0 2023-10-26T06:45:01.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc0ff9700 1 -- 172.21.15.89:0/2704722857 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4018680 con 0x7fabc41006a0 2023-10-26T06:45:01.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc419e290 con 0x7fabc41006a0 2023-10-26T06:45:01.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.413+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41a5ad0 con 0x7fabc41006a0 2023-10-26T06:45:01.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.414+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4101de0 con 0x7fabc41006a0 2023-10-26T06:45:01.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.416+0000 7fabc0ff9700 1 -- 172.21.15.89:0/2704722857 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fabb40105a0 con 0x7fabc41006a0 2023-10-26T06:45:01.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.417+0000 7fabc0ff9700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fabac0a1550 0x7fabac0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.418+0000 7fabc0ff9700 1 -- 172.21.15.89:0/2704722857 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7fabb4014070 con 0x7fabc41006a0 2023-10-26T06:45:01.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.418+0000 7fabc2ffd700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fabac0a1550 0x7fabac0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.418+0000 7fabc2ffd700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fabac0a1550 0x7fabac0a3a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fabc419a350 tx=0x7fabb800b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:01.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.420+0000 7fabc0ff9700 1 -- 172.21.15.89:0/2704722857 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fabb4096700 con 0x7fabc41006a0 2023-10-26T06:45:01.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:01 smithi135 ceph-mon[131403]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:01.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:01 smithi089 ceph-mon[95031]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:01.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.641+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabc4067530 con 0x7fabc41006a0 2023-10-26T06:45:01.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.642+0000 7fabc0ff9700 1 -- 172.21.15.89:0/2704722857 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18710 (secure 0 0 0) 0x7fabb4099d20 con 0x7fabc41006a0 2023-10-26T06:45:01.644 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:01.644 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:44:18.778923+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:45:01.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fabac0a1550 msgr2=0x7fabac0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fabac0a1550 0x7fabac0a3a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fabc419a350 tx=0x7fabb800b260 comp rx=0 tx=0).stop 2023-10-26T06:45:01.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 msgr2=0x7fabc4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4198e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e9a0 tx=0x7fabb400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 shutdown_connections 2023-10-26T06:45:01.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fabac0a1550 0x7fabac0a3a10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabc41066b0 0x7fabc419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc41006a0 0x7fabc4198e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 --2- 172.21.15.89:0/2704722857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fabc4101060 0x7fabc4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 >> 172.21.15.89:0/2704722857 conn(0x7fabc40fc170 msgr2=0x7fabc410b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:01.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 shutdown_connections 2023-10-26T06:45:01.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.645+0000 7fabc9bff700 1 -- 172.21.15.89:0/2704722857 wait complete. 2023-10-26T06:45:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:45:01.667 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:45:01.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 -- 172.21.15.89:0/3742134333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c067d40 msgr2=0x7f461c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 --2- 172.21.15.89:0/3742134333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c067d40 0x7f461c068120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46180099c0 tx=0x7f4618009cd0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 -- 172.21.15.89:0/3742134333 shutdown_connections 2023-10-26T06:45:01.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 --2- 172.21.15.89:0/3742134333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c067d40 0x7f461c068120 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 --2- 172.21.15.89:0/3742134333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c105b30 0x7f461c10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 --2- 172.21.15.89:0/3742134333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c105170 0x7f461c1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 -- 172.21.15.89:0/3742134333 >> 172.21.15.89:0/3742134333 conn(0x7f461c0767f0 msgr2=0x7f461c076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:01.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 -- 172.21.15.89:0/3742134333 shutdown_connections 2023-10-26T06:45:01.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.877+0000 7f46237ff700 1 -- 172.21.15.89:0/3742134333 wait complete. 2023-10-26T06:45:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.878+0000 7f46237ff700 1 Processor -- start 2023-10-26T06:45:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.878+0000 7f46237ff700 1 -- start start 2023-10-26T06:45:01.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f46237ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c067d40 0x7f461c0ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f46237ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c105170 0x7f461c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f46237ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c105b30 0x7f461c100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f46237ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c1a5010 con 0x7f461c105b30 2023-10-26T06:45:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f46237ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c1a5180 con 0x7f461c067d40 2023-10-26T06:45:01.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f46237ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c1a5340 con 0x7f461c105170 2023-10-26T06:45:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f462159b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c067d40 0x7f461c0ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f4621d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c105b30 0x7f461c100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f4620d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c105170 0x7f461c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f462159b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c067d40 0x7f461c0ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:46376/0 (socket says 172.21.15.89:46376) 2023-10-26T06:45:01.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.879+0000 7f462159b700 1 -- 172.21.15.89:0/481967628 learned_addr learned my addr 172.21.15.89:0/481967628 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f462159b700 1 -- 172.21.15.89:0/481967628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c105170 msgr2=0x7f461c0ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f462159b700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c105170 0x7f461c0ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f462159b700 1 -- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c105b30 msgr2=0x7f461c100150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f462159b700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c105b30 0x7f461c100150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:01.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f462159b700 1 -- 172.21.15.89:0/481967628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180095f0 con 0x7f461c067d40 2023-10-26T06:45:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f462159b700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c067d40 0x7f461c0ff6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4618005b40 tx=0x7f461800b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f46127fc700 1 -- 172.21.15.89:0/481967628 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461801d070 con 0x7f461c067d40 2023-10-26T06:45:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f46127fc700 1 -- 172.21.15.89:0/481967628 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f461800be00 con 0x7f461c067d40 2023-10-26T06:45:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c1a55c0 con 0x7f461c067d40 2023-10-26T06:45:01.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f46127fc700 1 -- 172.21.15.89:0/481967628 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800fb00 con 0x7f461c067d40 2023-10-26T06:45:01.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.880+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c1a59e0 con 0x7f461c067d40 2023-10-26T06:45:01.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.881+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c108c20 con 0x7f461c067d40 2023-10-26T06:45:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.883+0000 7f46127fc700 1 -- 172.21.15.89:0/481967628 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4618003680 con 0x7f461c067d40 2023-10-26T06:45:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.884+0000 7f46127fc700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46080a14d0 0x7f46080a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.884+0000 7f46127fc700 1 -- 172.21.15.89:0/481967628 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f46180cd940 con 0x7f461c067d40 2023-10-26T06:45:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.885+0000 7f4620d9a700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46080a14d0 0x7f46080a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:01.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.885+0000 7f4620d9a700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46080a14d0 0x7f46080a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f461c100de0 tx=0x7f460c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:01.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:01.887+0000 7f46127fc700 1 -- 172.21.15.89:0/481967628 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4618097360 con 0x7f461c067d40 2023-10-26T06:45:02.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.142+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f461c100b90 con 0x7f461c067d40 2023-10-26T06:45:02.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.143+0000 7f46127fc700 1 -- 172.21.15.89:0/481967628 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f461809a980 con 0x7f461c067d40 2023-10-26T06:45:02.144 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:45:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46080a14d0 msgr2=0x7f46080a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46080a14d0 0x7f46080a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f461c100de0 tx=0x7f460c009260 comp rx=0 tx=0).stop 2023-10-26T06:45:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c067d40 msgr2=0x7f461c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c067d40 0x7f461c0ff6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4618005b40 tx=0x7f461800b9c0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 shutdown_connections 2023-10-26T06:45:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46080a14d0 0x7f46080a3990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461c105b30 0x7f461c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f461c067d40 0x7f461c0ff6b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 --2- 172.21.15.89:0/481967628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f461c105170 0x7f461c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.145+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 >> 172.21.15.89:0/481967628 conn(0x7f461c0767f0 msgr2=0x7f461c0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:02.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.146+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 shutdown_connections 2023-10-26T06:45:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.146+0000 7f46237ff700 1 -- 172.21.15.89:0/481967628 wait complete. 2023-10-26T06:45:02.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T06:45:02.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1235718869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4106560 msgr2=0x7f62a4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1235718869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4106560 0x7f62a4106940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f62980099c0 tx=0x7f6298009cd0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1235718869 shutdown_connections 2023-10-26T06:45:02.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1235718869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4106560 0x7f62a4106940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1235718869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a4100e60 0x7f62a410d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1235718869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a41004a0 0x7f62a4100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1235718869 >> 172.21.15.89:0/1235718869 conn(0x7f62a40fbfb0 msgr2=0x7f62a40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:02.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1235718869 shutdown_connections 2023-10-26T06:45:02.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.372+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1235718869 wait complete. 2023-10-26T06:45:02.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.373+0000 7f62a9f9d700 1 Processor -- start 2023-10-26T06:45:02.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.373+0000 7f62a9f9d700 1 -- start start 2023-10-26T06:45:02.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a9f9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a41004a0 0x7f62a419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a9f9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a4100e60 0x7f62a419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a9f9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a4106560 0x7f62a4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a9f9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4198f90 con 0x7f62a41004a0 2023-10-26T06:45:02.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a9f9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4199100 con 0x7f62a4100e60 2023-10-26T06:45:02.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a9f9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41992c0 con 0x7f62a4106560 2023-10-26T06:45:02.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a41004a0 0x7f62a419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a979c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a4106560 0x7f62a4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a41004a0 0x7f62a419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33640/0 (socket says 172.21.15.89:33640) 2023-10-26T06:45:02.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 -- 172.21.15.89:0/1234271703 learned_addr learned my addr 172.21.15.89:0/1234271703 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:02.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a3fff700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a4100e60 0x7f62a419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 -- 172.21.15.89:0/1234271703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a4106560 msgr2=0x7f62a4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a4106560 0x7f62a4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 -- 172.21.15.89:0/1234271703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a4100e60 msgr2=0x7f62a419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a4100e60 0x7f62a419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 -- 172.21.15.89:0/1234271703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980095f0 con 0x7f62a41004a0 2023-10-26T06:45:02.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.374+0000 7f62a8f9b700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a41004a0 0x7f62a419e800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6298005b40 tx=0x7f629800b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:02.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.375+0000 7f62a1ffb700 1 -- 172.21.15.89:0/1234271703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629801d070 con 0x7f62a41004a0 2023-10-26T06:45:02.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.375+0000 7f62a1ffb700 1 -- 172.21.15.89:0/1234271703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f629800be00 con 0x7f62a41004a0 2023-10-26T06:45:02.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.375+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a4199540 con 0x7f62a41004a0 2023-10-26T06:45:02.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.375+0000 7f62a1ffb700 1 -- 172.21.15.89:0/1234271703 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800fb00 con 0x7f62a41004a0 2023-10-26T06:45:02.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.375+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a41a57f0 con 0x7f62a41004a0 2023-10-26T06:45:02.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.376+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a4101bc0 con 0x7f62a41004a0 2023-10-26T06:45:02.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.378+0000 7f62a1ffb700 1 -- 172.21.15.89:0/1234271703 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f629802b440 con 0x7f62a41004a0 2023-10-26T06:45:02.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.379+0000 7f62a1ffb700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f62940a1550 0x7f62940a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.379+0000 7f62a1ffb700 1 -- 172.21.15.89:0/1234271703 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f62980cd5e0 con 0x7f62a41004a0 2023-10-26T06:45:02.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.379+0000 7f62a3fff700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f62940a1550 0x7f62940a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.380+0000 7f62a3fff700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f62940a1550 0x7f62940a3a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62a4199af0 tx=0x7f6290009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:02.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.383+0000 7f62a1ffb700 1 -- 172.21.15.89:0/1234271703 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6298097000 con 0x7f62a41004a0 2023-10-26T06:45:02.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:02 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/980311312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:45:02.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:02 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2704722857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:02.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:02 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/980311312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:45:02.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:02 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2704722857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:02 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/980311312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:45:02.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:02 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2704722857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:02.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:02.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.604+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f62a40673e0 con 0x7f62a41004a0 2023-10-26T06:45:02.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.605+0000 7f62a1ffb700 1 -- 172.21.15.89:0/1234271703 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f629809a620 con 0x7f62a41004a0 2023-10-26T06:45:02.606 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:02.607 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2023-10-26T06:45:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:45:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2023-10-26T06:45:02.607 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2023-10-26T06:45:02.608 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:45:02.608 INFO:teuthology.orchestra.run.smithi089.stdout:] 2023-10-26T06:45:02.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f62940a1550 msgr2=0x7f62940a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f62940a1550 0x7f62940a3a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62a4199af0 tx=0x7f6290009260 comp rx=0 tx=0).stop 2023-10-26T06:45:02.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a41004a0 msgr2=0x7f62a419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a41004a0 0x7f62a419e800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6298005b40 tx=0x7f629800b9c0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 shutdown_connections 2023-10-26T06:45:02.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f62940a1550 0x7f62940a3a10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a41004a0 0x7f62a419e800 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62a4106560 0x7f62a4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 --2- 172.21.15.89:0/1234271703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a4100e60 0x7f62a419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 >> 172.21.15.89:0/1234271703 conn(0x7f62a40fbfb0 msgr2=0x7f62a40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:02.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 shutdown_connections 2023-10-26T06:45:02.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.607+0000 7f62a9f9d700 1 -- 172.21.15.89:0/1234271703 wait complete. 2023-10-26T06:45:02.631 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T06:45:02.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T06:45:02.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 -- 172.21.15.89:0/1309323267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac106720 msgr2=0x7f6aac106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/1309323267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac106720 0x7f6aac106b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009990 tx=0x7f6aa8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 -- 172.21.15.89:0/1309323267 shutdown_connections 2023-10-26T06:45:02.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/1309323267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac106720 0x7f6aac106b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/1309323267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6aac1010d0 0x7f6aac10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/1309323267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aac100710 0x7f6aac100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 -- 172.21.15.89:0/1309323267 >> 172.21.15.89:0/1309323267 conn(0x7f6aac0fc200 msgr2=0x7f6aac0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:02.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 -- 172.21.15.89:0/1309323267 shutdown_connections 2023-10-26T06:45:02.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.837+0000 7f6ab37ff700 1 -- 172.21.15.89:0/1309323267 wait complete. 2023-10-26T06:45:02.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.838+0000 7f6ab37ff700 1 Processor -- start 2023-10-26T06:45:02.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.839+0000 7f6ab37ff700 1 -- start start 2023-10-26T06:45:02.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.839+0000 7f6ab37ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac100710 0x7f6aac19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.839+0000 7f6ab37ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6aac1010d0 0x7f6aac19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.839+0000 7f6ab37ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aac106720 0x7f6aac196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.839+0000 7f6ab37ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac196fa0 con 0x7f6aac100710 2023-10-26T06:45:02.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.839+0000 7f6ab37ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac197110 con 0x7f6aac106720 2023-10-26T06:45:02.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.839+0000 7f6ab37ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1972d0 con 0x7f6aac1010d0 2023-10-26T06:45:02.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac100710 0x7f6aac19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab1d9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aac106720 0x7f6aac196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab0d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6aac1010d0 0x7f6aac19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac100710 0x7f6aac19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33670/0 (socket says 172.21.15.89:33670) 2023-10-26T06:45:02.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 -- 172.21.15.89:0/2713094760 learned_addr learned my addr 172.21.15.89:0/2713094760 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:02.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 -- 172.21.15.89:0/2713094760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6aac1010d0 msgr2=0x7f6aac19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6aac1010d0 0x7f6aac19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 -- 172.21.15.89:0/2713094760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aac106720 msgr2=0x7f6aac196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aac106720 0x7f6aac196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 -- 172.21.15.89:0/2713094760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80095f0 con 0x7f6aac100710 2023-10-26T06:45:02.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab0d9a700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6aac1010d0 0x7f6aac19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6ab159b700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac100710 0x7f6aac19c810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80098e0 tx=0x7f6aa800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.840+0000 7f6aa27fc700 1 -- 172.21.15.89:0/2713094760 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa801d070 con 0x7f6aac100710 2023-10-26T06:45:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.841+0000 7f6aa27fc700 1 -- 172.21.15.89:0/2713094760 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa8022470 con 0x7f6aac100710 2023-10-26T06:45:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.841+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac197550 con 0x7f6aac100710 2023-10-26T06:45:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.841+0000 7f6aa27fc700 1 -- 172.21.15.89:0/2713094760 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8017650 con 0x7f6aac100710 2023-10-26T06:45:02.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.841+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac072c00 con 0x7f6aac100710 2023-10-26T06:45:02.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.842+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac108d10 con 0x7f6aac100710 2023-10-26T06:45:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.844+0000 7f6aa27fc700 1 -- 172.21.15.89:0/2713094760 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6aa80225e0 con 0x7f6aac100710 2023-10-26T06:45:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.845+0000 7f6aa27fc700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a980a14d0 0x7f6a980a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.845+0000 7f6aa27fc700 1 -- 172.21.15.89:0/2713094760 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7342+0+0 (secure 0 0 0) 0x7f6aa80cdca0 con 0x7f6aac100710 2023-10-26T06:45:02.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.845+0000 7f6ab0d9a700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a980a14d0 0x7f6a980a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:02.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.846+0000 7f6ab0d9a700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a980a14d0 0x7f6a980a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ffd0 tx=0x7f6a9c00f820 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:02.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:02.847+0000 7f6aa27fc700 1 -- 172.21.15.89:0/2713094760 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6aa8097760 con 0x7f6aac100710 2023-10-26T06:45:03.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.076+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6aac101e50 con 0x7f6aac100710 2023-10-26T06:45:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.891+0000 7f6aa27fc700 1 -- 172.21.15.89:0/2713094760 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6aa809ad80 con 0x7f6aac100710 2023-10-26T06:45:03.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.893+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a980a14d0 msgr2=0x7f6a980a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:03.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.893+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a980a14d0 0x7f6a980a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00ffd0 tx=0x7f6a9c00f820 comp rx=0 tx=0).stop 2023-10-26T06:45:03.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.893+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac100710 msgr2=0x7f6aac19c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:03.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.893+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac100710 0x7f6aac19c810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80098e0 tx=0x7f6aa800f670 comp rx=0 tx=0).stop 2023-10-26T06:45:03.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:03 smithi192 ceph-mon[131661]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:03.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:03 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/481967628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:45:03.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:03 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1234271703' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-26T06:45:03.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:03.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:03.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 shutdown_connections 2023-10-26T06:45:03.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a980a14d0 0x7f6a980a3990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:03.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6aac100710 0x7f6aac19c810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:03.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6aac106720 0x7f6aac196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:03.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 --2- 172.21.15.89:0/2713094760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6aac1010d0 0x7f6aac19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:03.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 >> 172.21.15.89:0/2713094760 conn(0x7f6aac0fc200 msgr2=0x7f6aac0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:03.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 shutdown_connections 2023-10-26T06:45:03.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:03.894+0000 7f6ab37ff700 1 -- 172.21.15.89:0/2713094760 wait complete. 2023-10-26T06:45:03.899 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2023-10-26T06:45:03.915 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:03 smithi089 ceph-mon[95031]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:03.916 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:03 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/481967628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:45:03.916 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:03 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1234271703' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-26T06:45:03.916 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:03.916 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:03.918 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T06:45:03.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:03 smithi135 ceph-mon[131403]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:03 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/481967628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:45:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:03 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1234271703' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-26T06:45:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:03.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:04.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 -- 172.21.15.89:0/625439106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0073540 msgr2=0x7fbda010ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:04.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 --2- 172.21.15.89:0/625439106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0073540 0x7fbda010ca10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009990 tx=0x7fbd90009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 -- 172.21.15.89:0/625439106 shutdown_connections 2023-10-26T06:45:04.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 --2- 172.21.15.89:0/625439106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda0072c20 0x7fbda0073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 --2- 172.21.15.89:0/625439106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda010cf50 0x7fbda010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 --2- 172.21.15.89:0/625439106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0073540 0x7fbda010ca10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 -- 172.21.15.89:0/625439106 >> 172.21.15.89:0/625439106 conn(0x7fbda00fc200 msgr2=0x7fbda00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:04.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 -- 172.21.15.89:0/625439106 shutdown_connections 2023-10-26T06:45:04.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.140+0000 7fbda6bff700 1 -- 172.21.15.89:0/625439106 wait complete. 2023-10-26T06:45:04.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.141+0000 7fbda6bff700 1 Processor -- start 2023-10-26T06:45:04.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda6bff700 1 -- start start 2023-10-26T06:45:04.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda6bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0072c20 0x7fbda0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:04.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda6bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda0073540 0x7fbda0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:04.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda6bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda010cf50 0x7fbda019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:04.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda6bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019dce0 con 0x7fbda0073540 2023-10-26T06:45:04.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda6bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019de50 con 0x7fbda0072c20 2023-10-26T06:45:04.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda6bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019e010 con 0x7fbda010cf50 2023-10-26T06:45:04.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda499b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0072c20 0x7fbda0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:04.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda499b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0072c20 0x7fbda0198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:46444/0 (socket says 172.21.15.89:46444) 2023-10-26T06:45:04.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.142+0000 7fbda499b700 1 -- 172.21.15.89:0/1484540202 learned_addr learned my addr 172.21.15.89:0/1484540202 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:04.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbd97fff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda0073540 0x7fbda0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:04.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda519c700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda010cf50 0x7fbda019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:04.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda499b700 1 -- 172.21.15.89:0/1484540202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda010cf50 msgr2=0x7fbda019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:04.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda499b700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda010cf50 0x7fbda019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda499b700 1 -- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda0073540 msgr2=0x7fbda0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:04.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda499b700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda0073540 0x7fbda0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda499b700 1 -- 172.21.15.89:0/1484540202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900095f0 con 0x7fbda0072c20 2023-10-26T06:45:04.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda519c700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda010cf50 0x7fbda019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:04.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda499b700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0072c20 0x7fbda0198dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00e9a0 tx=0x7fbd9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:04.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbd95ffb700 1 -- 172.21.15.89:0/1484540202 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c009790 con 0x7fbda0072c20 2023-10-26T06:45:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda019e2f0 con 0x7fbda0072c20 2023-10-26T06:45:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbd95ffb700 1 -- 172.21.15.89:0/1484540202 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9c00cbd0 con 0x7fbda0072c20 2023-10-26T06:45:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbd95ffb700 1 -- 172.21.15.89:0/1484540202 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c018680 con 0x7fbda0072c20 2023-10-26T06:45:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.143+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda01a5b30 con 0x7fbda0072c20 2023-10-26T06:45:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.144+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda0108160 con 0x7fbda0072c20 2023-10-26T06:45:04.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.147+0000 7fbd95ffb700 1 -- 172.21.15.89:0/1484540202 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fbd9c00cd40 con 0x7fbda0072c20 2023-10-26T06:45:04.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.149+0000 7fbd95ffb700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbd880a14d0 0x7fbd880a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:04.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.149+0000 7fbd95ffb700 1 -- 172.21.15.89:0/1484540202 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7712+0+0 (secure 0 0 0) 0x7fbd9c014070 con 0x7fbda0072c20 2023-10-26T06:45:04.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.150+0000 7fbd97fff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbd880a14d0 0x7fbd880a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:04.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.150+0000 7fbd97fff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbd880a14d0 0x7fbd880a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd900098e0 tx=0x7fbd90005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:04.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.153+0000 7fbd95ffb700 1 -- 172.21.15.89:0/1484540202 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbd9c052100 con 0x7fbda0072c20 2023-10-26T06:45:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.362+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbda00742d0 con 0x7fbda0072c20 2023-10-26T06:45:04.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.899+0000 7fbd95ffb700 1 -- 172.21.15.89:0/1484540202 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbd9c096660 con 0x7fbda0072c20 2023-10-26T06:45:04.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.908+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbd880a14d0 msgr2=0x7fbd880a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:04.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.908+0000 7fbda6bff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbd880a14d0 0x7fbd880a3990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd900098e0 tx=0x7fbd90005960 comp rx=0 tx=0).stop 2023-10-26T06:45:04.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.908+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0072c20 msgr2=0x7fbda0198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:04.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.908+0000 7fbda6bff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0072c20 0x7fbda0198dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00e9a0 tx=0x7fbd9c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.909+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 shutdown_connections 2023-10-26T06:45:04.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.909+0000 7fbda6bff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbd880a14d0 0x7fbd880a3990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.909+0000 7fbda6bff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda010cf50 0x7fbda019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.909+0000 7fbda6bff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbda0072c20 0x7fbda0198dc0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00e9a0 tx=0x7fbd9c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.909+0000 7fbda6bff700 1 --2- 172.21.15.89:0/1484540202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbda0073540 0x7fbda0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:04.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.909+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 >> 172.21.15.89:0/1484540202 conn(0x7fbda00fc200 msgr2=0x7fbda00fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:04.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.910+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 shutdown_connections 2023-10-26T06:45:04.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:04.910+0000 7fbda6bff700 1 -- 172.21.15.89:0/1484540202 wait complete. 2023-10-26T06:45:04.917 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2023-10-26T06:45:04.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T06:45:05.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2713094760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-26T06:45:05.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:05.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:45:05.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:45:05.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:05.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2713094760' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-26T06:45:05.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: osdmap e81: 12 total, 12 up, 12 in 2023-10-26T06:45:05.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:04 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:45:05.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 -- 172.21.15.89:0/1643196832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 msgr2=0x7f1ecc10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/1643196832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 0x7f1ecc10d5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009f90 tx=0x7f1ec400b330 comp rx=0 tx=0).stop 2023-10-26T06:45:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 -- 172.21.15.89:0/1643196832 shutdown_connections 2023-10-26T06:45:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/1643196832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 0x7f1ecc10d5f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:05.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/1643196832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc106730 0x7f1ecc106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/1643196832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc100720 0x7f1ecc100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 -- 172.21.15.89:0/1643196832 >> 172.21.15.89:0/1643196832 conn(0x7f1ecc0fc1b0 msgr2=0x7f1ecc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 -- 172.21.15.89:0/1643196832 shutdown_connections 2023-10-26T06:45:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.154+0000 7f1ed33ff700 1 -- 172.21.15.89:0/1643196832 wait complete. 2023-10-26T06:45:05.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.155+0000 7f1ed33ff700 1 Processor -- start 2023-10-26T06:45:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.155+0000 7f1ed33ff700 1 -- start start 2023-10-26T06:45:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed33ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc100720 0x7f1ecc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed33ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 0x7f1ecc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed33ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc106730 0x7f1ecc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:05.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed33ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc19cb60 con 0x7f1ecc106730 2023-10-26T06:45:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed33ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc19ccd0 con 0x7f1ecc100720 2023-10-26T06:45:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed33ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc19ce90 con 0x7f1ecc1010e0 2023-10-26T06:45:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed199c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc106730 0x7f1ecc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed099a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 0x7f1ecc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:05.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed199c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc106730 0x7f1ecc19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33722/0 (socket says 172.21.15.89:33722) 2023-10-26T06:45:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed199c700 1 -- 172.21.15.89:0/2103884760 learned_addr learned my addr 172.21.15.89:0/2103884760 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed119b700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc100720 0x7f1ecc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed099a700 1 -- 172.21.15.89:0/2103884760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc100720 msgr2=0x7f1ecc198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed099a700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc100720 0x7f1ecc198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:05.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed099a700 1 -- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc106730 msgr2=0x7f1ecc19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:05.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed099a700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc106730 0x7f1ecc19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:05.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed099a700 1 -- 172.21.15.89:0/2103884760 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc009520 con 0x7f1ecc1010e0 2023-10-26T06:45:05.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.156+0000 7f1ed199c700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc106730 0x7f1ecc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:05.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.157+0000 7f1ed099a700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 0x7f1ecc199270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9f0 tx=0x7f1ebc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:05.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.157+0000 7f1ec27fc700 1 -- 172.21.15.89:0/2103884760 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00cb70 con 0x7f1ecc1010e0 2023-10-26T06:45:05.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.157+0000 7f1ec27fc700 1 -- 172.21.15.89:0/2103884760 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ebc00ee80 con 0x7f1ecc1010e0 2023-10-26T06:45:05.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.157+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec4009c40 con 0x7f1ecc1010e0 2023-10-26T06:45:05.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.157+0000 7f1ec27fc700 1 -- 172.21.15.89:0/2103884760 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003d30 con 0x7f1ecc1010e0 2023-10-26T06:45:05.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.157+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc1a58f0 con 0x7f1ecc1010e0 2023-10-26T06:45:05.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.158+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc101e60 con 0x7f1ecc1010e0 2023-10-26T06:45:05.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.160+0000 7f1ec27fc700 1 -- 172.21.15.89:0/2103884760 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f1ebc00ccd0 con 0x7f1ecc1010e0 2023-10-26T06:45:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.161+0000 7f1ec27fc700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1eb80a1550 0x7f1eb80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.161+0000 7f1ec27fc700 1 -- 172.21.15.89:0/2103884760 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8094+0+0 (secure 0 0 0) 0x7f1ebc014070 con 0x7f1ecc1010e0 2023-10-26T06:45:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.161+0000 7f1ed119b700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1eb80a1550 0x7f1eb80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:05.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.162+0000 7f1ed119b700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1eb80a1550 0x7f1eb80a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005fd0 tx=0x7f1ec8009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:05.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.163+0000 7f1ec27fc700 1 -- 172.21.15.89:0/2103884760 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ebc0965a0 con 0x7f1ecc1010e0 2023-10-26T06:45:05.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2713094760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-26T06:45:05.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:05.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:45:05.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:45:05.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:05.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2713094760' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-26T06:45:05.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: osdmap e81: 12 total, 12 up, 12 in 2023-10-26T06:45:05.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:04 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:45:05.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2713094760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-26T06:45:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:45:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:45:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:05.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2713094760' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-26T06:45:05.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: osdmap e81: 12 total, 12 up, 12 in 2023-10-26T06:45:05.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:04 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:45:05.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:05.442+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1ecc0675a0 con 0x7f1ecc1010e0 2023-10-26T06:45:06.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.053+0000 7f1ec27fc700 1 -- 172.21.15.89:0/2103884760 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1ebc099bc0 con 0x7f1ecc1010e0 2023-10-26T06:45:06.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.055+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1eb80a1550 msgr2=0x7f1eb80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.055+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1eb80a1550 0x7f1eb80a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005fd0 tx=0x7f1ec8009310 comp rx=0 tx=0).stop 2023-10-26T06:45:06.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.055+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 msgr2=0x7f1ecc199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.055+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 0x7f1ecc199270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9f0 tx=0x7f1ebc00ed00 comp rx=0 tx=0).stop 2023-10-26T06:45:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 shutdown_connections 2023-10-26T06:45:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1eb80a1550 0x7f1eb80a3a10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ecc106730 0x7f1ecc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ecc100720 0x7f1ecc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 --2- 172.21.15.89:0/2103884760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc1010e0 0x7f1ecc199270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 >> 172.21.15.89:0/2103884760 conn(0x7f1ecc0fc1b0 msgr2=0x7f1ecc072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:06.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 shutdown_connections 2023-10-26T06:45:06.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.056+0000 7f1ed33ff700 1 -- 172.21.15.89:0/2103884760 wait complete. 2023-10-26T06:45:06.060 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 2 and data pool 3 2023-10-26T06:45:06.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:45:06.140 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1484540202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-26T06:45:06.141 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:05 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-26T06:45:06.141 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:05 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-26T06:45:06.141 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:05 smithi089 ceph-mon[95031]: osdmap e82: 12 total, 12 up, 12 in 2023-10-26T06:45:06.141 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2103884760' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-26T06:45:06.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:05 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-26T06:45:06.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:05 smithi089 ceph-mon[95031]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-26T06:45:06.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1484540202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-26T06:45:06.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:05 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-26T06:45:06.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:05 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-26T06:45:06.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:05 smithi192 ceph-mon[131661]: osdmap e82: 12 total, 12 up, 12 in 2023-10-26T06:45:06.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2103884760' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-26T06:45:06.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:05 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-26T06:45:06.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:05 smithi192 ceph-mon[131661]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-26T06:45:06.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1484540202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-26T06:45:06.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:05 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-26T06:45:06.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:05 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-26T06:45:06.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:05 smithi135 ceph-mon[131403]: osdmap e82: 12 total, 12 up, 12 in 2023-10-26T06:45:06.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2103884760' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-26T06:45:06.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:05 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-26T06:45:06.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:05 smithi135 ceph-mon[131403]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-26T06:45:06.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.285+0000 7f905f5ff700 1 -- 172.21.15.89:0/4002642723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058073540 msgr2=0x7f905810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.285+0000 7f905f5ff700 1 --2- 172.21.15.89:0/4002642723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058073540 0x7f905810c9a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90480098f0 tx=0x7f9048009c00 comp rx=0 tx=0).stop 2023-10-26T06:45:06.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.285+0000 7f905f5ff700 1 -- 172.21.15.89:0/4002642723 shutdown_connections 2023-10-26T06:45:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.285+0000 7f905f5ff700 1 --2- 172.21.15.89:0/4002642723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f905810cee0 0x7f905810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.285+0000 7f905f5ff700 1 --2- 172.21.15.89:0/4002642723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058072c20 0x7f9058073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.285+0000 7f905f5ff700 1 --2- 172.21.15.89:0/4002642723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058073540 0x7f905810c9a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.285+0000 7f905f5ff700 1 -- 172.21.15.89:0/4002642723 >> 172.21.15.89:0/4002642723 conn(0x7f90580fc170 msgr2=0x7f90580fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.286+0000 7f905f5ff700 1 -- 172.21.15.89:0/4002642723 shutdown_connections 2023-10-26T06:45:06.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.286+0000 7f905f5ff700 1 -- 172.21.15.89:0/4002642723 wait complete. 2023-10-26T06:45:06.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.286+0000 7f905f5ff700 1 Processor -- start 2023-10-26T06:45:06.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905f5ff700 1 -- start start 2023-10-26T06:45:06.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905f5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058072c20 0x7f9058198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905f5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058073540 0x7f9058199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905f5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f905810cee0 0x7f905819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905f5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905819dd30 con 0x7f9058072c20 2023-10-26T06:45:06.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905f5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905819dea0 con 0x7f905810cee0 2023-10-26T06:45:06.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905f5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905819e060 con 0x7f9058073540 2023-10-26T06:45:06.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905d39b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058072c20 0x7f9058198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905cb9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058073540 0x7f9058199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905db9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f905810cee0 0x7f905819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905cb9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058073540 0x7f9058199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:55218/0 (socket says 172.21.15.89:55218) 2023-10-26T06:45:06.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905db9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f905810cee0 0x7f905819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:46470/0 (socket says 172.21.15.89:46470) 2023-10-26T06:45:06.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.287+0000 7f905cb9a700 1 -- 172.21.15.89:0/2693663870 learned_addr learned my addr 172.21.15.89:0/2693663870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:06.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905cb9a700 1 -- 172.21.15.89:0/2693663870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f905810cee0 msgr2=0x7f905819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905cb9a700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f905810cee0 0x7f905819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905cb9a700 1 -- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058072c20 msgr2=0x7f9058198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905cb9a700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058072c20 0x7f9058198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905cb9a700 1 -- 172.21.15.89:0/2693663870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540095f0 con 0x7f9058073540 2023-10-26T06:45:06.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905cb9a700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058073540 0x7f9058199350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9048005950 tx=0x7f904800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:06.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f904e7fc700 1 -- 172.21.15.89:0/2693663870 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904801c070 con 0x7f9058073540 2023-10-26T06:45:06.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f904e7fc700 1 -- 172.21.15.89:0/2693663870 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904800baa0 con 0x7f9058073540 2023-10-26T06:45:06.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9048009520 con 0x7f9058073540 2023-10-26T06:45:06.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f904e7fc700 1 -- 172.21.15.89:0/2693663870 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90480038e0 con 0x7f9058073540 2023-10-26T06:45:06.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.288+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90581a59a0 con 0x7f9058073540 2023-10-26T06:45:06.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.289+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90581080f0 con 0x7f9058073540 2023-10-26T06:45:06.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.291+0000 7f904e7fc700 1 -- 172.21.15.89:0/2693663870 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9048020030 con 0x7f9058073540 2023-10-26T06:45:06.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.292+0000 7f904e7fc700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90440a14d0 0x7f90440a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.292+0000 7f904e7fc700 1 -- 172.21.15.89:0/2693663870 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f90480cdae0 con 0x7f9058073540 2023-10-26T06:45:06.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.292+0000 7f905d39b700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90440a14d0 0x7f90440a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.293+0000 7f905d39b700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90440a14d0 0x7f90440a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9054005fd0 tx=0x7f9054009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:06.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.294+0000 7f904e7fc700 1 -- 172.21.15.89:0/2693663870 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9048097190 con 0x7f9058073540 2023-10-26T06:45:06.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.507+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f905819a110 con 0x7f9058073540 2023-10-26T06:45:06.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.509+0000 7f904e7fc700 1 -- 172.21.15.89:0/2693663870 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22420 (secure 0 0 0) 0x7f904809a7b0 con 0x7f9058073540 2023-10-26T06:45:06.511 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":83,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:45:05.444457+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:45:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90440a14d0 msgr2=0x7f90440a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90440a14d0 0x7f90440a3990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9054005fd0 tx=0x7f9054009310 comp rx=0 tx=0).stop 2023-10-26T06:45:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058073540 msgr2=0x7f9058199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058073540 0x7f9058199350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9048005950 tx=0x7f904800fa00 comp rx=0 tx=0).stop 2023-10-26T06:45:06.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 shutdown_connections 2023-10-26T06:45:06.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90440a14d0 0x7f90440a3990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9058072c20 0x7f9058198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f905810cee0 0x7f905819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 --2- 172.21.15.89:0/2693663870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9058073540 0x7f9058199350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 >> 172.21.15.89:0/2693663870 conn(0x7f90580fc170 msgr2=0x7f90580fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:06.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.511+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 shutdown_connections 2023-10-26T06:45:06.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.512+0000 7f905f5ff700 1 -- 172.21.15.89:0/2693663870 wait complete. 2023-10-26T06:45:06.532 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:45:06.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 -- 172.21.15.89:0/1409847017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101060 msgr2=0x7f728010d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 --2- 172.21.15.89:0/1409847017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101060 0x7f728010d570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7278009990 tx=0x7f7278009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 -- 172.21.15.89:0/1409847017 shutdown_connections 2023-10-26T06:45:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 --2- 172.21.15.89:0/1409847017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101060 0x7f728010d570 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 --2- 172.21.15.89:0/1409847017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72801066b0 0x7f7280106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 --2- 172.21.15.89:0/1409847017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72801006a0 0x7f7280100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 -- 172.21.15.89:0/1409847017 >> 172.21.15.89:0/1409847017 conn(0x7f72800fc1d0 msgr2=0x7f72800fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:06.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 -- 172.21.15.89:0/1409847017 shutdown_connections 2023-10-26T06:45:06.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.755+0000 7f72885ff700 1 -- 172.21.15.89:0/1409847017 wait complete. 2023-10-26T06:45:06.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.756+0000 7f72885ff700 1 Processor -- start 2023-10-26T06:45:06.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.756+0000 7f72885ff700 1 -- start start 2023-10-26T06:45:06.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f72885ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72801006a0 0x7f728019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f72885ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7280101060 0x7f728019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f72885ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72801066b0 0x7f7280198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f72885ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72801992d0 con 0x7f72801006a0 2023-10-26T06:45:06.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f72885ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280199440 con 0x7f72801066b0 2023-10-26T06:45:06.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f72885ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280199600 con 0x7f7280101060 2023-10-26T06:45:06.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72801006a0 0x7f728019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72801006a0 0x7f728019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33774/0 (socket says 172.21.15.89:33774) 2023-10-26T06:45:06.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f7285b9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7280101060 0x7f728019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 -- 172.21.15.89:0/1509943650 learned_addr learned my addr 172.21.15.89:0/1509943650 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:06.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f7286b9c700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72801066b0 0x7f7280198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 -- 172.21.15.89:0/1509943650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7280101060 msgr2=0x7f728019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7280101060 0x7f728019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 -- 172.21.15.89:0/1509943650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72801066b0 msgr2=0x7f7280198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72801066b0 0x7f7280198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 -- 172.21.15.89:0/1509943650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c009c40 con 0x7f72801006a0 2023-10-26T06:45:06.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.757+0000 7f728639b700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72801006a0 0x7f728019ea60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f727c00c370 tx=0x7f727c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:06.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.758+0000 7f72777fe700 1 -- 172.21.15.89:0/1509943650 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c017070 con 0x7f72801006a0 2023-10-26T06:45:06.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.758+0000 7f72777fe700 1 -- 172.21.15.89:0/1509943650 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f727c00f040 con 0x7f72801006a0 2023-10-26T06:45:06.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.758+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72780095f0 con 0x7f72801006a0 2023-10-26T06:45:06.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.758+0000 7f72777fe700 1 -- 172.21.15.89:0/1509943650 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c00e050 con 0x7f72801006a0 2023-10-26T06:45:06.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.758+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72801a59e0 con 0x7f72801006a0 2023-10-26T06:45:06.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.759+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7280101de0 con 0x7f72801006a0 2023-10-26T06:45:06.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.761+0000 7f72777fe700 1 -- 172.21.15.89:0/1509943650 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f727c004440 con 0x7f72801006a0 2023-10-26T06:45:06.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.763+0000 7f72777fe700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f726c0a1550 0x7f726c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:06.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.763+0000 7f72777fe700 1 -- 172.21.15.89:0/1509943650 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f727c0ccca0 con 0x7f72801006a0 2023-10-26T06:45:06.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.764+0000 7f7285b9a700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f726c0a1550 0x7f726c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:06.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.764+0000 7f7285b9a700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f726c0a1550 0x7f726c0a3a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f728019a270 tx=0x7f727000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:06.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.766+0000 7f72777fe700 1 -- 172.21.15.89:0/1509943650 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f727c0962c0 con 0x7f72801006a0 2023-10-26T06:45:06.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.981+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7280199fe0 con 0x7f72801006a0 2023-10-26T06:45:06.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.984+0000 7f72777fe700 1 -- 172.21.15.89:0/1509943650 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22407 (secure 0 0 0) 0x7f727c00cb80 con 0x7f72801006a0 2023-10-26T06:45:06.986 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:06.986 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:45:06.881272+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:45:06.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.986+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f726c0a1550 msgr2=0x7f726c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.986+0000 7f72885ff700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f726c0a1550 0x7f726c0a3a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f728019a270 tx=0x7f727000b410 comp rx=0 tx=0).stop 2023-10-26T06:45:06.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.986+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72801006a0 msgr2=0x7f728019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.986+0000 7f72885ff700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72801006a0 0x7f728019ea60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f727c00c370 tx=0x7f727c00c680 comp rx=0 tx=0).stop 2023-10-26T06:45:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 shutdown_connections 2023-10-26T06:45:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f726c0a1550 0x7f726c0a3a10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72801006a0 0x7f728019ea60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72801066b0 0x7f7280198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 --2- 172.21.15.89:0/1509943650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7280101060 0x7f728019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:06.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 >> 172.21.15.89:0/1509943650 conn(0x7f72800fc1d0 msgr2=0x7f72800fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:06.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 shutdown_connections 2023-10-26T06:45:06.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:06.987+0000 7f72885ff700 1 -- 172.21.15.89:0/1509943650 wait complete. 2023-10-26T06:45:07.008 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T06:45:07.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: pgmap v252: 97 pgs: 21 creating+peering, 7 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:07.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: osdmap e83: 12 total, 12 up, 12 in 2023-10-26T06:45:07.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-26T06:45:07.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-26T06:45:07.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-26T06:45:07.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-26T06:45:07.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2693663870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:07.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:06 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: pgmap v252: 97 pgs: 21 creating+peering, 7 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:07.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: osdmap e83: 12 total, 12 up, 12 in 2023-10-26T06:45:07.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-26T06:45:07.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-26T06:45:07.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-26T06:45:07.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-26T06:45:07.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2693663870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:07.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:06 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 -- 172.21.15.89:0/3826663736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0072c20 msgr2=0x7f45a0073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 --2- 172.21.15.89:0/3826663736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0072c20 0x7f45a0073000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45900098f0 tx=0x7f4590009c00 comp rx=0 tx=0).stop 2023-10-26T06:45:07.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 -- 172.21.15.89:0/3826663736 shutdown_connections 2023-10-26T06:45:07.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 --2- 172.21.15.89:0/3826663736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a010ce20 0x7f45a010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:07.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 --2- 172.21.15.89:0/3826663736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0072c20 0x7f45a0073000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:07.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 --2- 172.21.15.89:0/3826663736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a0073540 0x7f45a010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:07.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 -- 172.21.15.89:0/3826663736 >> 172.21.15.89:0/3826663736 conn(0x7f45a00fc0d0 msgr2=0x7f45a00fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:07.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 -- 172.21.15.89:0/3826663736 shutdown_connections 2023-10-26T06:45:07.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.219+0000 7f459f59e700 1 -- 172.21.15.89:0/3826663736 wait complete. 2023-10-26T06:45:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.220+0000 7f459f59e700 1 Processor -- start 2023-10-26T06:45:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459f59e700 1 -- start start 2023-10-26T06:45:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459f59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a0072c20 0x7f45a019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:07.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459f59e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0073540 0x7f45a019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459f59e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a010ce20 0x7f45a0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459f59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01991e0 con 0x7f45a0072c20 2023-10-26T06:45:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459f59e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0199350 con 0x7f45a0073540 2023-10-26T06:45:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459f59e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0199510 con 0x7f45a010ce20 2023-10-26T06:45:07.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a0072c20 0x7f45a019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f4597fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0073540 0x7f45a019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a0072c20 0x7f45a019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33792/0 (socket says 172.21.15.89:33792) 2023-10-26T06:45:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f4597fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0073540 0x7f45a019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:46500/0 (socket says 172.21.15.89:46500) 2023-10-26T06:45:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459e59c700 1 -- 172.21.15.89:0/1848276963 learned_addr learned my addr 172.21.15.89:0/1848276963 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:07.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.221+0000 7f459ed9d700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a010ce20 0x7f45a0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f4597fff700 1 -- 172.21.15.89:0/1848276963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a010ce20 msgr2=0x7f45a0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f4597fff700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a010ce20 0x7f45a0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f4597fff700 1 -- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a0072c20 msgr2=0x7f45a019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f4597fff700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a0072c20 0x7f45a019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f4597fff700 1 -- 172.21.15.89:0/1848276963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590009520 con 0x7f45a0073540 2023-10-26T06:45:07.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f459e59c700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a0072c20 0x7f45a019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:07.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f459ed9d700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a010ce20 0x7f45a0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:07.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f4597fff700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0073540 0x7f45a019ef90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f458800e9f0 tx=0x7f458800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:07.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f45977fe700 1 -- 172.21.15.89:0/1848276963 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458800c9c0 con 0x7f45a0073540 2023-10-26T06:45:07.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f45977fe700 1 -- 172.21.15.89:0/1848276963 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4588010430 con 0x7f45a0073540 2023-10-26T06:45:07.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a01997f0 con 0x7f45a0073540 2023-10-26T06:45:07.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.222+0000 7f45977fe700 1 -- 172.21.15.89:0/1848276963 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588018680 con 0x7f45a0073540 2023-10-26T06:45:07.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.223+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a01a5a40 con 0x7f45a0073540 2023-10-26T06:45:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.224+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a0108030 con 0x7f45a0073540 2023-10-26T06:45:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.225+0000 7f45977fe700 1 -- 172.21.15.89:0/1848276963 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f4588010860 con 0x7f45a0073540 2023-10-26T06:45:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.227+0000 7f45977fe700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f458c0a1550 0x7f458c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:07.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.227+0000 7f45977fe700 1 -- 172.21.15.89:0/1848276963 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f4588014070 con 0x7f45a0073540 2023-10-26T06:45:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.227+0000 7f459e59c700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f458c0a1550 0x7f458c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.227+0000 7f459e59c700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f458c0a1550 0x7f458c0a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4590009fd0 tx=0x7f4590019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:07.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.230+0000 7f45977fe700 1 -- 172.21.15.89:0/1848276963 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4588052100 con 0x7f45a0073540 2023-10-26T06:45:07.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: pgmap v252: 97 pgs: 21 creating+peering, 7 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-26T06:45:07.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: osdmap e83: 12 total, 12 up, 12 in 2023-10-26T06:45:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-26T06:45:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-26T06:45:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-26T06:45:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-26T06:45:07.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2693663870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:07.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:06 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:07.551+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f45a0199fd0 con 0x7f45a0073540 2023-10-26T06:45:08.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.061+0000 7f45977fe700 1 -- 172.21.15.89:0/1848276963 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4588096460 con 0x7f45a0073540 2023-10-26T06:45:08.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f458c0a1550 msgr2=0x7f458c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:08.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f458c0a1550 0x7f458c0a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4590009fd0 tx=0x7f4590019040 comp rx=0 tx=0).stop 2023-10-26T06:45:08.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0073540 msgr2=0x7f45a019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:08.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0073540 0x7f45a019ef90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f458800e9f0 tx=0x7f458800ed00 comp rx=0 tx=0).stop 2023-10-26T06:45:08.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 shutdown_connections 2023-10-26T06:45:08.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f458c0a1550 0x7f458c0a3a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45a0072c20 0x7f45a019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45a010ce20 0x7f45a0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 --2- 172.21.15.89:0/1848276963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45a0073540 0x7f45a019ef90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 >> 172.21.15.89:0/1848276963 conn(0x7f45a00fc0d0 msgr2=0x7f45a00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 shutdown_connections 2023-10-26T06:45:08.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.064+0000 7f459f59e700 1 -- 172.21.15.89:0/1848276963 wait complete. 2023-10-26T06:45:08.084 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-10-26T06:45:08.085 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-26T06:45:08.144 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: osdmap e84: 12 total, 12 up, 12 in 2023-10-26T06:45:08.145 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1509943650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:08.145 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] up:active 2023-10-26T06:45:08.145 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:08.145 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:08.145 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1848276963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-26T06:45:08.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-26T06:45:08.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:45:08.146 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:45:08.147 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: osdmap e84: 12 total, 12 up, 12 in 2023-10-26T06:45:08.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1509943650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:08.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] up:active 2023-10-26T06:45:08.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:08.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:08.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1848276963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-26T06:45:08.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-26T06:45:08.148 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:45:08.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:45:08.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: osdmap e84: 12 total, 12 up, 12 in 2023-10-26T06:45:08.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1509943650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:08.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] up:active 2023-10-26T06:45:08.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:08.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:08.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1848276963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-26T06:45:08.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-26T06:45:08.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:45:08.220 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:45:08.221 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:45:08.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 -- 172.21.15.89:0/574568231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084106720 msgr2=0x7f9084106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:08.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 --2- 172.21.15.89:0/574568231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084106720 0x7f9084106b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9080009990 tx=0x7f9080009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 -- 172.21.15.89:0/574568231 shutdown_connections 2023-10-26T06:45:08.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 --2- 172.21.15.89:0/574568231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 0x7f908410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 --2- 172.21.15.89:0/574568231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084106720 0x7f9084106b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 --2- 172.21.15.89:0/574568231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9084100710 0x7f9084100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 -- 172.21.15.89:0/574568231 >> 172.21.15.89:0/574568231 conn(0x7f90840fc200 msgr2=0x7f90840fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:08.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 -- 172.21.15.89:0/574568231 shutdown_connections 2023-10-26T06:45:08.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.283+0000 7f908b9ff700 1 -- 172.21.15.89:0/574568231 wait complete. 2023-10-26T06:45:08.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.284+0000 7f908b9ff700 1 Processor -- start 2023-10-26T06:45:08.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.284+0000 7f908b9ff700 1 -- start start 2023-10-26T06:45:08.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f908b9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084100710 0x7f9084198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:08.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f908b9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 0x7f90841992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:08.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f908b9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9084106720 0x7f908419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:08.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f908b9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908419dca0 con 0x7f9084106720 2023-10-26T06:45:08.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f908b9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908419de10 con 0x7f9084100710 2023-10-26T06:45:08.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f908b9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908419dfd0 con 0x7f90841010d0 2023-10-26T06:45:08.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 0x7f90841992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9089f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9084106720 0x7f908419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 0x7f90841992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:55278/0 (socket says 172.21.15.89:55278) 2023-10-26T06:45:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 -- 172.21.15.89:0/3761815960 learned_addr learned my addr 172.21.15.89:0/3761815960 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f908979b700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084100710 0x7f9084198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:08.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 -- 172.21.15.89:0/3761815960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084100710 msgr2=0x7f9084198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:08.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084100710 0x7f9084198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 -- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9084106720 msgr2=0x7f908419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:08.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9084106720 0x7f908419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:08.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9088f9a700 1 -- 172.21.15.89:0/3761815960 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90800095f0 con 0x7f90841010d0 2023-10-26T06:45:08.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.285+0000 7f9089f9c700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9084106720 0x7f908419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:08.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.286+0000 7f9088f9a700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 0x7f90841992c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9a0 tx=0x7f907400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:08.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.286+0000 7f907a7fc700 1 -- 172.21.15.89:0/3761815960 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074009790 con 0x7f90841010d0 2023-10-26T06:45:08.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.286+0000 7f907a7fc700 1 -- 172.21.15.89:0/3761815960 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9074010430 con 0x7f90841010d0 2023-10-26T06:45:08.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.286+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908419e2b0 con 0x7f90841010d0 2023-10-26T06:45:08.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.286+0000 7f907a7fc700 1 -- 172.21.15.89:0/3761815960 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074018680 con 0x7f90841010d0 2023-10-26T06:45:08.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.286+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90841a5af0 con 0x7f90841010d0 2023-10-26T06:45:08.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.287+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9084101e50 con 0x7f90841010d0 2023-10-26T06:45:08.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.289+0000 7f907a7fc700 1 -- 172.21.15.89:0/3761815960 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f90740105a0 con 0x7f90841010d0 2023-10-26T06:45:08.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.290+0000 7f907a7fc700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90700a1550 0x7f90700a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:08.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.290+0000 7f907a7fc700 1 -- 172.21.15.89:0/3761815960 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f9074014070 con 0x7f90841010d0 2023-10-26T06:45:08.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.290+0000 7f908979b700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90700a1550 0x7f90700a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:08.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.291+0000 7f908979b700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90700a1550 0x7f90700a3a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9080006010 tx=0x7f908000b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:08.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.292+0000 7f907a7fc700 1 -- 172.21.15.89:0/3761815960 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9074096570 con 0x7f90841010d0 2023-10-26T06:45:08.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:08.535+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f90840675a0 con 0x7f90841010d0 2023-10-26T06:45:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.069+0000 7f907a7fc700 1 -- 172.21.15.89:0/3761815960 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9074099b90 con 0x7f90841010d0 2023-10-26T06:45:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90700a1550 msgr2=0x7f90700a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:09.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90700a1550 0x7f90700a3a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9080006010 tx=0x7f908000b260 comp rx=0 tx=0).stop 2023-10-26T06:45:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 msgr2=0x7f90841992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 0x7f90841992c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9a0 tx=0x7f907400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 shutdown_connections 2023-10-26T06:45:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90700a1550 0x7f90700a3a10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9084106720 0x7f908419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9084100710 0x7f9084198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 --2- 172.21.15.89:0/3761815960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90841010d0 0x7f90841992c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 >> 172.21.15.89:0/3761815960 conn(0x7f90840fc200 msgr2=0x7f908410be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 shutdown_connections 2023-10-26T06:45:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.072+0000 7f908b9ff700 1 -- 172.21.15.89:0/3761815960 wait complete. 2023-10-26T06:45:09.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-26T06:45:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 -- 172.21.15.89:0/906390230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 msgr2=0x7f9f3010d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:09.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 --2- 172.21.15.89:0/906390230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3010d5e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800b0a0 tx=0x7f9f2800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 -- 172.21.15.89:0/906390230 shutdown_connections 2023-10-26T06:45:09.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 --2- 172.21.15.89:0/906390230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3010d5e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 --2- 172.21.15.89:0/906390230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f30106720 0x7f9f30106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 --2- 172.21.15.89:0/906390230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f30100710 0x7f9f30100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 -- 172.21.15.89:0/906390230 >> 172.21.15.89:0/906390230 conn(0x7f9f300fc200 msgr2=0x7f9f300fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:09.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 -- 172.21.15.89:0/906390230 shutdown_connections 2023-10-26T06:45:09.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.292+0000 7f9f36fff700 1 -- 172.21.15.89:0/906390230 wait complete. 2023-10-26T06:45:09.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 Processor -- start 2023-10-26T06:45:09.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 -- start start 2023-10-26T06:45:09.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f30100710 0x7f9f3019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:09.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:09.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f30106720 0x7f9f30198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:09.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f301991e0 con 0x7f9f301010d0 2023-10-26T06:45:09.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f30199350 con 0x7f9f30100710 2023-10-26T06:45:09.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.293+0000 7f9f36fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f30199510 con 0x7f9f30106720 2023-10-26T06:45:09.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:09.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33820/0 (socket says 172.21.15.89:33820) 2023-10-26T06:45:09.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 -- 172.21.15.89:0/515285147 learned_addr learned my addr 172.21.15.89:0/515285147 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:09.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f3559c700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f30106720 0x7f9f30198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:09.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f34d9b700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f30100710 0x7f9f3019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:09.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 -- 172.21.15.89:0/515285147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f30106720 msgr2=0x7f9f30198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:09.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f30106720 0x7f9f30198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 -- 172.21.15.89:0/515285147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f30100710 msgr2=0x7f9f3019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:09.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f30100710 0x7f9f3019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:09.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 -- 172.21.15.89:0/515285147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28009d00 con 0x7f9f301010d0 2023-10-26T06:45:09.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f34d9b700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f30100710 0x7f9f3019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:09.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.294+0000 7f9f2ffff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3019ef90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d950 tx=0x7f9f1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:09.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.295+0000 7f9f2dffb700 1 -- 172.21.15.89:0/515285147 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c009940 con 0x7f9f301010d0 2023-10-26T06:45:09.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.295+0000 7f9f2dffb700 1 -- 172.21.15.89:0/515285147 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f1c010460 con 0x7f9f301010d0 2023-10-26T06:45:09.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.295+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f301997f0 con 0x7f9f301010d0 2023-10-26T06:45:09.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.295+0000 7f9f2dffb700 1 -- 172.21.15.89:0/515285147 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00f620 con 0x7f9f301010d0 2023-10-26T06:45:09.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.295+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f301a5a40 con 0x7f9f301010d0 2023-10-26T06:45:09.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.296+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f30101e50 con 0x7f9f301010d0 2023-10-26T06:45:09.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.298+0000 7f9f2dffb700 1 -- 172.21.15.89:0/515285147 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f9f1c0108d0 con 0x7f9f301010d0 2023-10-26T06:45:09.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.299+0000 7f9f2dffb700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f200a1550 0x7f9f200a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:09.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.299+0000 7f9f2dffb700 1 -- 172.21.15.89:0/515285147 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f9f1c0ccb70 con 0x7f9f301010d0 2023-10-26T06:45:09.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.299+0000 7f9f34d9b700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f200a1550 0x7f9f200a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:09.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.300+0000 7f9f34d9b700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f200a1550 0x7f9f200a3a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009c90 tx=0x7f9f24009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:09.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.303+0000 7f9f2dffb700 1 -- 172.21.15.89:0/515285147 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9f1c096190 con 0x7f9f301010d0 2023-10-26T06:45:09.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:09 smithi192 ceph-mon[131661]: pgmap v255: 97 pgs: 21 creating+peering, 52 active+clean, 24 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-10-26T06:45:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:09 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-26T06:45:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:09 smithi192 ceph-mon[131661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:09 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3761815960' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-26T06:45:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:09 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-26T06:45:09.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:09 smithi192 ceph-mon[131661]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-26T06:45:09.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:09 smithi192 ceph-mon[131661]: Cluster is now healthy 2023-10-26T06:45:09.467 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b[170212]: 2023-10-26T06:45:09.076+0000 7f3b4a6df700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T06:45:09.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-mon[131403]: pgmap v255: 97 pgs: 21 creating+peering, 52 active+clean, 24 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-10-26T06:45:09.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-26T06:45:09.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-mon[131403]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:09.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3761815960' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-26T06:45:09.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-26T06:45:09.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-mon[131403]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-26T06:45:09.471 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:09 smithi135 ceph-mon[131403]: Cluster is now healthy 2023-10-26T06:45:09.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:09 smithi089 ceph-mon[95031]: pgmap v255: 97 pgs: 21 creating+peering, 52 active+clean, 24 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-10-26T06:45:09.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:09 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-26T06:45:09.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:09 smithi089 ceph-mon[95031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:09.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:09 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3761815960' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-26T06:45:09.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:09 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-26T06:45:09.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:09 smithi089 ceph-mon[95031]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-26T06:45:09.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:09 smithi089 ceph-mon[95031]: Cluster is now healthy 2023-10-26T06:45:09.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:09.556+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f9f300675a0 con 0x7f9f301010d0 2023-10-26T06:45:10.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.080+0000 7f9f2dffb700 1 -- 172.21.15.89:0/515285147 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9f1c0160b0 con 0x7f9f301010d0 2023-10-26T06:45:10.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.082+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f200a1550 msgr2=0x7f9f200a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.082+0000 7f9f36fff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f200a1550 0x7f9f200a3a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009c90 tx=0x7f9f24009260 comp rx=0 tx=0).stop 2023-10-26T06:45:10.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 msgr2=0x7f9f3019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3019ef90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d950 tx=0x7f9f1c00dc60 comp rx=0 tx=0).stop 2023-10-26T06:45:10.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 shutdown_connections 2023-10-26T06:45:10.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f200a1550 0x7f9f200a3a10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f30106720 0x7f9f30198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f30100710 0x7f9f3019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 --2- 172.21.15.89:0/515285147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f301010d0 0x7f9f3019ef90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 >> 172.21.15.89:0/515285147 conn(0x7f9f300fc200 msgr2=0x7f9f3010c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:10.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 shutdown_connections 2023-10-26T06:45:10.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.083+0000 7f9f36fff700 1 -- 172.21.15.89:0/515285147 wait complete. 2023-10-26T06:45:10.104 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-10-26T06:45:10.105 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-26T06:45:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 -- 172.21.15.89:0/1985974747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec106730 msgr2=0x7f12ec106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 --2- 172.21.15.89:0/1985974747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec106730 0x7f12ec106b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009990 tx=0x7f12dc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 -- 172.21.15.89:0/1985974747 shutdown_connections 2023-10-26T06:45:10.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 --2- 172.21.15.89:0/1985974747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec106730 0x7f12ec106b10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 --2- 172.21.15.89:0/1985974747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12ec1010e0 0x7f12ec10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 --2- 172.21.15.89:0/1985974747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12ec100720 0x7f12ec100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 -- 172.21.15.89:0/1985974747 >> 172.21.15.89:0/1985974747 conn(0x7f12ec0fc1b0 msgr2=0x7f12ec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 -- 172.21.15.89:0/1985974747 shutdown_connections 2023-10-26T06:45:10.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.304+0000 7f12f0fff700 1 -- 172.21.15.89:0/1985974747 wait complete. 2023-10-26T06:45:10.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 Processor -- start 2023-10-26T06:45:10.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 -- start start 2023-10-26T06:45:10.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec100720 0x7f12ec19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12ec1010e0 0x7f12ec19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12ec106730 0x7f12ec1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec197010 con 0x7f12ec100720 2023-10-26T06:45:10.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec197180 con 0x7f12ec1010e0 2023-10-26T06:45:10.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.306+0000 7f12f0fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec197340 con 0x7f12ec106730 2023-10-26T06:45:10.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec100720 0x7f12ec19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec100720 0x7f12ec19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60148/0 (socket says 172.21.15.89:60148) 2023-10-26T06:45:10.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 -- 172.21.15.89:0/714936572 learned_addr learned my addr 172.21.15.89:0/714936572 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:10.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12e3fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12ec1010e0 0x7f12ec19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ead9d700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12ec106730 0x7f12ec1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 -- 172.21.15.89:0/714936572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12ec106730 msgr2=0x7f12ec1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12ec106730 0x7f12ec1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 -- 172.21.15.89:0/714936572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12ec1010e0 msgr2=0x7f12ec19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12ec1010e0 0x7f12ec19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 -- 172.21.15.89:0/714936572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc0095f0 con 0x7f12ec100720 2023-10-26T06:45:10.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12ea59c700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec100720 0x7f12ec19c880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f12dc000c00 tx=0x7f12dc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:10.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.307+0000 7f12e37fe700 1 -- 172.21.15.89:0/714936572 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc01d070 con 0x7f12ec100720 2023-10-26T06:45:10.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.308+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec1975c0 con 0x7f12ec100720 2023-10-26T06:45:10.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.308+0000 7f12e37fe700 1 -- 172.21.15.89:0/714936572 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12dc022470 con 0x7f12ec100720 2023-10-26T06:45:10.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.308+0000 7f12e37fe700 1 -- 172.21.15.89:0/714936572 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0175f0 con 0x7f12ec100720 2023-10-26T06:45:10.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.308+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec1a3870 con 0x7f12ec100720 2023-10-26T06:45:10.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.309+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec108d20 con 0x7f12ec100720 2023-10-26T06:45:10.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.311+0000 7f12e37fe700 1 -- 172.21.15.89:0/714936572 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f12dc0225e0 con 0x7f12ec100720 2023-10-26T06:45:10.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.312+0000 7f12e37fe700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f12d80a14d0 0x7f12d80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.312+0000 7f12e37fe700 1 -- 172.21.15.89:0/714936572 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f12dc0cdfa0 con 0x7f12ec100720 2023-10-26T06:45:10.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.312+0000 7f12e3fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f12d80a14d0 0x7f12d80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.313+0000 7f12e3fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f12d80a14d0 0x7f12d80a3990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12ec197b70 tx=0x7f12d4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:10.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.316+0000 7f12e37fe700 1 -- 172.21.15.89:0/714936572 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f12dc097650 con 0x7f12ec100720 2023-10-26T06:45:10.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:10 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-26T06:45:10.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:10 smithi192 ceph-mon[131661]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:10.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:10 smithi192 ceph-mon[131661]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-26T06:45:10.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:10 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/515285147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-26T06:45:10.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:10 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-26T06:45:10.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:10 smithi135 ceph-mon[131403]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:10.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:10 smithi135 ceph-mon[131403]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-26T06:45:10.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:10 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/515285147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-26T06:45:10.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-26T06:45:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-mon[95031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-26T06:45:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-mon[95031]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-26T06:45:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/515285147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-26T06:45:10.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.551+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f12ec197e70 con 0x7f12d80a14d0 2023-10-26T06:45:10.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.591+0000 7f12e37fe700 1 -- 172.21.15.89:0/714936572 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f12ec197e70 con 0x7f12d80a14d0 2023-10-26T06:45:10.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f12d80a14d0 msgr2=0x7f12d80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f12d80a14d0 0x7f12d80a3990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12ec197b70 tx=0x7f12d4009260 comp rx=0 tx=0).stop 2023-10-26T06:45:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec100720 msgr2=0x7f12ec19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec100720 0x7f12ec19c880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f12dc000c00 tx=0x7f12dc005740 comp rx=0 tx=0).stop 2023-10-26T06:45:10.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 shutdown_connections 2023-10-26T06:45:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f12d80a14d0 0x7f12d80a3990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12ec100720 0x7f12ec19c880 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12ec106730 0x7f12ec1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 --2- 172.21.15.89:0/714936572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12ec1010e0 0x7f12ec19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 >> 172.21.15.89:0/714936572 conn(0x7f12ec0fc1b0 msgr2=0x7f12ec0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:10.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.593+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 shutdown_connections 2023-10-26T06:45:10.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.594+0000 7f12f0fff700 1 -- 172.21.15.89:0/714936572 wait complete. 2023-10-26T06:45:10.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-26T06:45:10.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.814+0000 7f15623ff700 1 -- 172.21.15.89:0/3268079715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c1010e0 msgr2=0x7f155c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.814+0000 7f15623ff700 1 --2- 172.21.15.89:0/3268079715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c1010e0 0x7f155c10d5f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f155400b0a0 tx=0x7f155400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.815+0000 7f15623ff700 1 -- 172.21.15.89:0/3268079715 shutdown_connections 2023-10-26T06:45:10.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.815+0000 7f15623ff700 1 --2- 172.21.15.89:0/3268079715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c1010e0 0x7f155c10d5f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.815+0000 7f15623ff700 1 --2- 172.21.15.89:0/3268079715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f155c106730 0x7f155c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.815+0000 7f15623ff700 1 --2- 172.21.15.89:0/3268079715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f155c100720 0x7f155c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.815+0000 7f15623ff700 1 -- 172.21.15.89:0/3268079715 >> 172.21.15.89:0/3268079715 conn(0x7f155c0fc1b0 msgr2=0x7f155c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:10.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.815+0000 7f15623ff700 1 -- 172.21.15.89:0/3268079715 shutdown_connections 2023-10-26T06:45:10.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.815+0000 7f15623ff700 1 -- 172.21.15.89:0/3268079715 wait complete. 2023-10-26T06:45:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.816+0000 7f15623ff700 1 Processor -- start 2023-10-26T06:45:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.816+0000 7f15623ff700 1 -- start start 2023-10-26T06:45:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f15623ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c100720 0x7f155c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f15623ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f155c1010e0 0x7f155c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f15623ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f155c106730 0x7f155c196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f15623ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c196fd0 con 0x7f155c100720 2023-10-26T06:45:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f15623ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c197140 con 0x7f155c106730 2023-10-26T06:45:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f15623ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c197300 con 0x7f155c1010e0 2023-10-26T06:45:10.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c100720 0x7f155c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c100720 0x7f155c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60168/0 (socket says 172.21.15.89:60168) 2023-10-26T06:45:10.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 -- 172.21.15.89:0/3876202857 learned_addr learned my addr 172.21.15.89:0/3876202857 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:10.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155b7fe700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f155c1010e0 0x7f155c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f156099c700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f155c106730 0x7f155c196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 -- 172.21.15.89:0/3876202857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f155c1010e0 msgr2=0x7f155c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f155c1010e0 0x7f155c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 -- 172.21.15.89:0/3876202857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f155c106730 msgr2=0x7f155c196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f155c106730 0x7f155c196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.817+0000 7f155bfff700 1 -- 172.21.15.89:0/3876202857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1554009d00 con 0x7f155c100720 2023-10-26T06:45:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.818+0000 7f155bfff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c100720 0x7f155c19c7f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9a0 tx=0x7f154c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:10.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.818+0000 7f15597fa700 1 -- 172.21.15.89:0/3876202857 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c009790 con 0x7f155c100720 2023-10-26T06:45:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.818+0000 7f15597fa700 1 -- 172.21.15.89:0/3876202857 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154c00cbd0 con 0x7f155c100720 2023-10-26T06:45:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.818+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c1975e0 con 0x7f155c100720 2023-10-26T06:45:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.818+0000 7f15597fa700 1 -- 172.21.15.89:0/3876202857 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c018680 con 0x7f155c100720 2023-10-26T06:45:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.818+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c1a37e0 con 0x7f155c100720 2023-10-26T06:45:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.819+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155c108d20 con 0x7f155c100720 2023-10-26T06:45:10.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.821+0000 7f15597fa700 1 -- 172.21.15.89:0/3876202857 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f154c00cd40 con 0x7f155c100720 2023-10-26T06:45:10.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.822+0000 7f15597fa700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15440a14d0 0x7f15440a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.822+0000 7f15597fa700 1 -- 172.21.15.89:0/3876202857 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f154c014070 con 0x7f155c100720 2023-10-26T06:45:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.822+0000 7f155b7fe700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15440a14d0 0x7f15440a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.823+0000 7f155b7fe700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15440a14d0 0x7f15440a3990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f155c197ae0 tx=0x7f15500073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:10.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:10.826+0000 7f15597fa700 1 -- 172.21.15.89:0/3876202857 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f154c052100 con 0x7f155c100720 2023-10-26T06:45:11.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:45:10.557+0000 7f5789e0a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:45:11.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:45:10.557+0000 7f5789e0a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:45:11.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:45:10.557+0000 7f5789e0a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:45:11.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:45:10.557+0000 7f5789e0a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:45:11.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:45:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:45:10.557+0000 7f5789e0a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:45:11.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.063+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f155c197de0 con 0x7f15440a14d0 2023-10-26T06:45:11.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.068+0000 7f15597fa700 1 -- 172.21.15.89:0/3876202857 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f155c197de0 con 0x7f15440a14d0 2023-10-26T06:45:11.070 INFO:teuthology.orchestra.run.smithi089.stdout:/volumes/_nogroup/sv_0/9e680bf3-3e3f-439f-816c-cd8e1cca8532 2023-10-26T06:45:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.070+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15440a14d0 msgr2=0x7f15440a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.070+0000 7f15623ff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15440a14d0 0x7f15440a3990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f155c197ae0 tx=0x7f15500073d0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.070+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c100720 msgr2=0x7f155c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.070+0000 7f15623ff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c100720 0x7f155c19c7f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9a0 tx=0x7f154c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 shutdown_connections 2023-10-26T06:45:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15440a14d0 0x7f15440a3990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f155c100720 0x7f155c19c7f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f155c106730 0x7f155c196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 --2- 172.21.15.89:0/3876202857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f155c1010e0 0x7f155c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 >> 172.21.15.89:0/3876202857 conn(0x7f155c0fc1b0 msgr2=0x7f155c0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 shutdown_connections 2023-10-26T06:45:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.071+0000 7f15623ff700 1 -- 172.21.15.89:0/3876202857 wait complete. 2023-10-26T06:45:11.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-26T06:45:11.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 -- 172.21.15.89:0/1496790029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 msgr2=0x7feaa410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 --2- 172.21.15.89:0/1496790029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa410f3d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00b0a0 tx=0x7fea9c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 -- 172.21.15.89:0/1496790029 shutdown_connections 2023-10-26T06:45:11.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 --2- 172.21.15.89:0/1496790029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa410f3d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 --2- 172.21.15.89:0/1496790029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaa4072c20 0x7feaa4073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 --2- 172.21.15.89:0/1496790029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa4073540 0x7feaa410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 -- 172.21.15.89:0/1496790029 >> 172.21.15.89:0/1496790029 conn(0x7feaa40fc1b0 msgr2=0x7feaa40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:11.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 -- 172.21.15.89:0/1496790029 shutdown_connections 2023-10-26T06:45:11.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.294+0000 7feaab7ff700 1 -- 172.21.15.89:0/1496790029 wait complete. 2023-10-26T06:45:11.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.295+0000 7feaab7ff700 1 Processor -- start 2023-10-26T06:45:11.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaab7ff700 1 -- start start 2023-10-26T06:45:11.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaab7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaa4072c20 0x7feaa4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaab7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa4073540 0x7feaa4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaab7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaab7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa419de30 con 0x7feaa410cf60 2023-10-26T06:45:11.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaab7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa419dfa0 con 0x7feaa4072c20 2023-10-26T06:45:11.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaab7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa419e160 con 0x7feaa4073540 2023-10-26T06:45:11.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaa9d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaa9d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60188/0 (socket says 172.21.15.89:60188) 2023-10-26T06:45:11.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaa9d9c700 1 -- 172.21.15.89:0/3885226528 learned_addr learned my addr 172.21.15.89:0/3885226528 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:11.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaa959b700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaa4072c20 0x7feaa4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.296+0000 7feaa8d9a700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa4073540 0x7feaa4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaa9d9c700 1 -- 172.21.15.89:0/3885226528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa4073540 msgr2=0x7feaa4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaa9d9c700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa4073540 0x7feaa4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaa9d9c700 1 -- 172.21.15.89:0/3885226528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaa4072c20 msgr2=0x7feaa4198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaa9d9c700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaa4072c20 0x7feaa4198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaa9d9c700 1 -- 172.21.15.89:0/3885226528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c009d00 con 0x7feaa410cf60 2023-10-26T06:45:11.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaa8d9a700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa4073540 0x7feaa4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:11.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaa9d9c700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa419d7f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00b070 tx=0x7fea9c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:11.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7fea9a7fc700 1 -- 172.21.15.89:0/3885226528 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00e040 con 0x7feaa410cf60 2023-10-26T06:45:11.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7fea9a7fc700 1 -- 172.21.15.89:0/3885226528 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea9c01d470 con 0x7feaa410cf60 2023-10-26T06:45:11.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7fea9a7fc700 1 -- 172.21.15.89:0/3885226528 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0052a0 con 0x7feaa410cf60 2023-10-26T06:45:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa419e3e0 con 0x7feaa410cf60 2023-10-26T06:45:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.297+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa41a5c80 con 0x7feaa410cf60 2023-10-26T06:45:11.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.298+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa4108170 con 0x7feaa410cf60 2023-10-26T06:45:11.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.300+0000 7fea9a7fc700 1 -- 172.21.15.89:0/3885226528 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fea9c01d850 con 0x7feaa410cf60 2023-10-26T06:45:11.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.301+0000 7fea9a7fc700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea900a1550 0x7fea900a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.302+0000 7fea9a7fc700 1 -- 172.21.15.89:0/3885226528 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fea9c0cdda0 con 0x7feaa410cf60 2023-10-26T06:45:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.302+0000 7feaa959b700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea900a1550 0x7fea900a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.302+0000 7feaa959b700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea900a1550 0x7fea900a3a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009c60 tx=0x7feaa0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:11.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.305+0000 7fea9a7fc700 1 -- 172.21.15.89:0/3885226528 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fea9c097450 con 0x7feaa410cf60 2023-10-26T06:45:11.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:11 smithi192 ceph-mon[131661]: pgmap v256: 97 pgs: 6 creating+peering, 91 active+clean; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 703 B/s rd, 1.0 KiB/s wr, 7 op/s 2023-10-26T06:45:11.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:11 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/515285147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-26T06:45:11.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:11 smithi192 ceph-mon[131661]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-26T06:45:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:11 smithi135 ceph-mon[131403]: pgmap v256: 97 pgs: 6 creating+peering, 91 active+clean; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 703 B/s rd, 1.0 KiB/s wr, 7 op/s 2023-10-26T06:45:11.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:11 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/515285147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-26T06:45:11.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:11 smithi135 ceph-mon[131403]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-26T06:45:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:11 smithi089 ceph-mon[95031]: pgmap v256: 97 pgs: 6 creating+peering, 91 active+clean; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 703 B/s rd, 1.0 KiB/s wr, 7 op/s 2023-10-26T06:45:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:11 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/515285147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-26T06:45:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:11 smithi089 ceph-mon[95031]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2023-10-26T06:45:11.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.538+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7feaa419a280 con 0x7fea900a1550 2023-10-26T06:45:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.563+0000 7fea9a7fc700 1 -- 172.21.15.89:0/3885226528 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feaa419a280 con 0x7fea900a1550 2023-10-26T06:45:11.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea900a1550 msgr2=0x7fea900a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea900a1550 0x7fea900a3a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009c60 tx=0x7feaa0009260 comp rx=0 tx=0).stop 2023-10-26T06:45:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 msgr2=0x7feaa419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa419d7f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00b070 tx=0x7fea9c0078c0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 shutdown_connections 2023-10-26T06:45:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea900a1550 0x7fea900a3a10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa410cf60 0x7feaa419d7f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaa4072c20 0x7feaa4198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 --2- 172.21.15.89:0/3885226528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa4073540 0x7feaa4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 >> 172.21.15.89:0/3885226528 conn(0x7feaa40fc1b0 msgr2=0x7feaa40fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 shutdown_connections 2023-10-26T06:45:11.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.565+0000 7feaab7ff700 1 -- 172.21.15.89:0/3885226528 wait complete. 2023-10-26T06:45:11.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-26T06:45:11.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/3733818616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 msgr2=0x7f6a24100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/3733818616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24100ba0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009990 tx=0x7f6a14009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/3733818616 shutdown_connections 2023-10-26T06:45:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/3733818616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 0x7f6a2410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/3733818616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24106730 0x7f6a24106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/3733818616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24100ba0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/3733818616 >> 172.21.15.89:0/3733818616 conn(0x7f6a240fc1b0 msgr2=0x7f6a240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:11.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/3733818616 shutdown_connections 2023-10-26T06:45:11.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.786+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/3733818616 wait complete. 2023-10-26T06:45:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.788+0000 7f6a2c3ff700 1 Processor -- start 2023-10-26T06:45:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.788+0000 7f6a2c3ff700 1 -- start start 2023-10-26T06:45:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.788+0000 7f6a2c3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.788+0000 7f6a2c3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 0x7f6a24073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.788+0000 7f6a2c3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24106730 0x7f6a24073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2c3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241a4f40 con 0x7f6a24100720 2023-10-26T06:45:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2c3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241a50b0 con 0x7f6a24106730 2023-10-26T06:45:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2c3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241a5270 con 0x7f6a241010e0 2023-10-26T06:45:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2a19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 0x7f6a24073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 0x7f6a24073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:32858/0 (socket says 172.21.15.89:32858) 2023-10-26T06:45:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2a19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60208/0 (socket says 172.21.15.89:60208) 2023-10-26T06:45:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 -- 172.21.15.89:0/104871308 learned_addr learned my addr 172.21.15.89:0/104871308 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2a99c700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24106730 0x7f6a24073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 -- 172.21.15.89:0/104871308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24106730 msgr2=0x7f6a24073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24106730 0x7f6a24073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 -- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 msgr2=0x7f6a24072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:11.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 -- 172.21.15.89:0/104871308 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140095f0 con 0x7f6a241010e0 2023-10-26T06:45:11.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2a99c700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24106730 0x7f6a24073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:11.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2a19b700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:11.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.789+0000 7f6a2999a700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 0x7f6a24073030 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005910 tx=0x7f6a14004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:11.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.790+0000 7f6a1b7fe700 1 -- 172.21.15.89:0/104871308 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1401d070 con 0x7f6a241010e0 2023-10-26T06:45:11.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.790+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a241a54f0 con 0x7f6a241010e0 2023-10-26T06:45:11.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.790+0000 7f6a1b7fe700 1 -- 172.21.15.89:0/104871308 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a14022470 con 0x7f6a241010e0 2023-10-26T06:45:11.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.790+0000 7f6a1b7fe700 1 -- 172.21.15.89:0/104871308 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14017650 con 0x7f6a241010e0 2023-10-26T06:45:11.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.790+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a241a59d0 con 0x7f6a241010e0 2023-10-26T06:45:11.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.791+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a24108d40 con 0x7f6a241010e0 2023-10-26T06:45:11.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.793+0000 7f6a1b7fe700 1 -- 172.21.15.89:0/104871308 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f6a140225e0 con 0x7f6a241010e0 2023-10-26T06:45:11.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.794+0000 7f6a1b7fe700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a100a1550 0x7f6a100a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:11.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.794+0000 7f6a1b7fe700 1 -- 172.21.15.89:0/104871308 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f6a140ce430 con 0x7f6a241010e0 2023-10-26T06:45:11.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.795+0000 7f6a2a19b700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a100a1550 0x7f6a100a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:11.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.795+0000 7f6a2a19b700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a100a1550 0x7f6a100a3a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009fd0 tx=0x7f6a20009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:11.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:11.797+0000 7f6a1b7fe700 1 -- 172.21.15.89:0/104871308 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a14097ae0 con 0x7f6a241010e0 2023-10-26T06:45:12.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.031+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a24073f80 con 0x7f6a100a1550 2023-10-26T06:45:12.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.035+0000 7f6a1b7fe700 1 -- 172.21.15.89:0/104871308 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f6a24073f80 con 0x7f6a100a1550 2023-10-26T06:45:12.037 INFO:teuthology.orchestra.run.smithi089.stdout:/volumes/_nogroup/sv_1/4456288b-9087-4ca7-95ed-920f9d752dee 2023-10-26T06:45:12.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.037+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a100a1550 msgr2=0x7f6a100a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.037+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a100a1550 0x7f6a100a3a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009fd0 tx=0x7f6a20009260 comp rx=0 tx=0).stop 2023-10-26T06:45:12.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 msgr2=0x7f6a24073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 0x7f6a24073030 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005910 tx=0x7f6a14004730 comp rx=0 tx=0).stop 2023-10-26T06:45:12.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 shutdown_connections 2023-10-26T06:45:12.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6a100a1550 0x7f6a100a3a10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a24100720 0x7f6a24072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a24106730 0x7f6a24073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 --2- 172.21.15.89:0/104871308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a241010e0 0x7f6a24073030 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 >> 172.21.15.89:0/104871308 conn(0x7f6a240fc1b0 msgr2=0x7f6a240fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 shutdown_connections 2023-10-26T06:45:12.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.038+0000 7f6a2c3ff700 1 -- 172.21.15.89:0/104871308 wait complete. 2023-10-26T06:45:12.059 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:45:12.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 -- 172.21.15.89:0/349610649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8072c20 msgr2=0x7fa2d8073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/349610649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8072c20 0x7fa2d8073000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40098f0 tx=0x7fa2d4009c00 comp rx=0 tx=0).stop 2023-10-26T06:45:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 -- 172.21.15.89:0/349610649 shutdown_connections 2023-10-26T06:45:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/349610649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d810cee0 0x7fa2d810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/349610649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8072c20 0x7fa2d8073000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/349610649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2d8073540 0x7fa2d810c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 -- 172.21.15.89:0/349610649 >> 172.21.15.89:0/349610649 conn(0x7fa2d80fc1d0 msgr2=0x7fa2d80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:12.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 -- 172.21.15.89:0/349610649 shutdown_connections 2023-10-26T06:45:12.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.259+0000 7fa2e05ff700 1 -- 172.21.15.89:0/349610649 wait complete. 2023-10-26T06:45:12.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 Processor -- start 2023-10-26T06:45:12.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 -- start start 2023-10-26T06:45:12.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d8072c20 0x7fa2d8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8073540 0x7fa2d81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2d810cee0 0x7fa2d819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d819ddb0 con 0x7fa2d810cee0 2023-10-26T06:45:12.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d819df20 con 0x7fa2d8073540 2023-10-26T06:45:12.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.260+0000 7fa2e05ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d819e0e0 con 0x7fa2d8072c20 2023-10-26T06:45:12.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2deb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2d810cee0 0x7fa2d819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2de39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d8072c20 0x7fa2d8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2deb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2d810cee0 0x7fa2d819d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60236/0 (socket says 172.21.15.89:60236) 2023-10-26T06:45:12.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2deb9c700 1 -- 172.21.15.89:0/2845103052 learned_addr learned my addr 172.21.15.89:0/2845103052 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:12.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2ddb9a700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8073540 0x7fa2d81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2de39b700 1 -- 172.21.15.89:0/2845103052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8073540 msgr2=0x7fa2d81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2de39b700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8073540 0x7fa2d81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2de39b700 1 -- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2d810cee0 msgr2=0x7fa2d819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2de39b700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2d810cee0 0x7fa2d819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2de39b700 1 -- 172.21.15.89:0/2845103052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c80095f0 con 0x7fa2d8072c20 2023-10-26T06:45:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.261+0000 7fa2de39b700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d8072c20 0x7fa2d8198e90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009fd0 tx=0x7fa2d400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.262+0000 7fa2cf7fe700 1 -- 172.21.15.89:0/2845103052 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d401c070 con 0x7fa2d8072c20 2023-10-26T06:45:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.262+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d4009520 con 0x7fa2d8072c20 2023-10-26T06:45:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.262+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d81a5a20 con 0x7fa2d8072c20 2023-10-26T06:45:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.262+0000 7fa2cf7fe700 1 -- 172.21.15.89:0/2845103052 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2d400baa0 con 0x7fa2d8072c20 2023-10-26T06:45:12.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.262+0000 7fa2cf7fe700 1 -- 172.21.15.89:0/2845103052 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40038e0 con 0x7fa2d8072c20 2023-10-26T06:45:12.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.263+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d81080f0 con 0x7fa2d8072c20 2023-10-26T06:45:12.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.264+0000 7fa2cf7fe700 1 -- 172.21.15.89:0/2845103052 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fa2d4020030 con 0x7fa2d8072c20 2023-10-26T06:45:12.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.266+0000 7fa2cf7fe700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa2c40a1560 0x7fa2c40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.266+0000 7fa2cf7fe700 1 -- 172.21.15.89:0/2845103052 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fa2d40cdca0 con 0x7fa2d8072c20 2023-10-26T06:45:12.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.266+0000 7fa2ddb9a700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa2c40a1560 0x7fa2c40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.266+0000 7fa2ddb9a700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa2c40a1560 0x7fa2c40a3a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa2d819a3e0 tx=0x7fa2c8009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:12.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.270+0000 7fa2cf7fe700 1 -- 172.21.15.89:0/2845103052 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa2d40971f0 con 0x7fa2d8072c20 2023-10-26T06:45:12.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:12 smithi192 ceph-mon[131661]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:12.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:12 smithi192 ceph-mon[131661]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:12.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:12 smithi135 ceph-mon[131403]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:12.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:12 smithi135 ceph-mon[131403]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.513+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2d819a190 con 0x7fa2d8072c20 2023-10-26T06:45:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.515+0000 7fa2cf7fe700 1 -- 172.21.15.89:0/2845103052 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fa2d409a810 con 0x7fa2d8072c20 2023-10-26T06:45:12.516 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:12.517 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.516+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa2c40a1560 msgr2=0x7fa2c40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa2c40a1560 0x7fa2c40a3a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa2d819a3e0 tx=0x7fa2c8009310 comp rx=0 tx=0).stop 2023-10-26T06:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d8072c20 msgr2=0x7fa2d8198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d8072c20 0x7fa2d8198e90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009fd0 tx=0x7fa2d400fa00 comp rx=0 tx=0).stop 2023-10-26T06:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 shutdown_connections 2023-10-26T06:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa2c40a1560 0x7fa2c40a3a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2d810cee0 0x7fa2d819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d8072c20 0x7fa2d8198e90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 --2- 172.21.15.89:0/2845103052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2d8073540 0x7fa2d81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 >> 172.21.15.89:0/2845103052 conn(0x7fa2d80fc1d0 msgr2=0x7fa2d80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 shutdown_connections 2023-10-26T06:45:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.517+0000 7fa2e05ff700 1 -- 172.21.15.89:0/2845103052 wait complete. 2023-10-26T06:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:45:12.535 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:12 smithi089 ceph-mon[95031]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:12.536 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:12 smithi089 ceph-mon[95031]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:12.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:45:12.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 -- 172.21.15.89:0/583236252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441006a0 msgr2=0x7f9e44100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 --2- 172.21.15.89:0/583236252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441006a0 0x7f9e44100b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009990 tx=0x7f9e38009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 -- 172.21.15.89:0/583236252 shutdown_connections 2023-10-26T06:45:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 --2- 172.21.15.89:0/583236252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e44101060 0x7f9e4410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 --2- 172.21.15.89:0/583236252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e441066b0 0x7f9e44106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 --2- 172.21.15.89:0/583236252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441006a0 0x7f9e44100b20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 -- 172.21.15.89:0/583236252 >> 172.21.15.89:0/583236252 conn(0x7f9e440fc170 msgr2=0x7f9e440fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:12.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 -- 172.21.15.89:0/583236252 shutdown_connections 2023-10-26T06:45:12.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.733+0000 7f9e493ff700 1 -- 172.21.15.89:0/583236252 wait complete. 2023-10-26T06:45:12.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 Processor -- start 2023-10-26T06:45:12.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 -- start start 2023-10-26T06:45:12.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e441006a0 0x7f9e4419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e44101060 0x7f9e4419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441066b0 0x7f9e44196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44197150 con 0x7f9e441066b0 2023-10-26T06:45:12.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e441972c0 con 0x7f9e44101060 2023-10-26T06:45:12.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.734+0000 7f9e493ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44197480 con 0x7f9e441006a0 2023-10-26T06:45:12.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441066b0 0x7f9e44196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441066b0 0x7f9e44196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60252/0 (socket says 172.21.15.89:60252) 2023-10-26T06:45:12.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e427fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e44101060 0x7f9e4419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 -- 172.21.15.89:0/1234055047 learned_addr learned my addr 172.21.15.89:0/1234055047 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e42ffd700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e441006a0 0x7f9e4419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 -- 172.21.15.89:0/1234055047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e441006a0 msgr2=0x7f9e4419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e441006a0 0x7f9e4419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 -- 172.21.15.89:0/1234055047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e44101060 msgr2=0x7f9e4419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e44101060 0x7f9e4419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 -- 172.21.15.89:0/1234055047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34009520 con 0x7f9e441066b0 2023-10-26T06:45:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e427fc700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e44101060 0x7f9e4419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.735+0000 7f9e437fe700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441066b0 0x7f9e44196980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007ee0 tx=0x7f9e3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.736+0000 7f9e2bfff700 1 -- 172.21.15.89:0/1234055047 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00dd20 con 0x7f9e441066b0 2023-10-26T06:45:12.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.736+0000 7f9e2bfff700 1 -- 172.21.15.89:0/1234055047 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e3c00f040 con 0x7f9e441066b0 2023-10-26T06:45:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.736+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e380095f0 con 0x7f9e441066b0 2023-10-26T06:45:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.736+0000 7f9e2bfff700 1 -- 172.21.15.89:0/1234055047 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0048a0 con 0x7f9e441066b0 2023-10-26T06:45:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.736+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e441a3860 con 0x7f9e441066b0 2023-10-26T06:45:12.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.737+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e44108cc0 con 0x7f9e441066b0 2023-10-26T06:45:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.739+0000 7f9e2bfff700 1 -- 172.21.15.89:0/1234055047 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9e3c003a00 con 0x7f9e441066b0 2023-10-26T06:45:12.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.740+0000 7f9e2bfff700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9e2c0a1560 0x7f9e2c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:12.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.740+0000 7f9e2bfff700 1 -- 172.21.15.89:0/1234055047 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f9e3c0cc770 con 0x7f9e441066b0 2023-10-26T06:45:12.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.740+0000 7f9e42ffd700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9e2c0a1560 0x7f9e2c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:12.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.741+0000 7f9e42ffd700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9e2c0a1560 0x7f9e2c0a3a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009d70 tx=0x7f9e34009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:12.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.744+0000 7f9e2bfff700 1 -- 172.21.15.89:0/1234055047 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9e3c04f100 con 0x7f9e441066b0 2023-10-26T06:45:12.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.946+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e44101de0 con 0x7f9e441066b0 2023-10-26T06:45:12.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.947+0000 7f9e2bfff700 1 -- 172.21.15.89:0/1234055047 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22407 (secure 0 0 0) 0x7f9e44101de0 con 0x7f9e441066b0 2023-10-26T06:45:12.950 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:12.950 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:45:06.881272+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:45:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.949+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9e2c0a1560 msgr2=0x7f9e2c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.949+0000 7f9e493ff700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9e2c0a1560 0x7f9e2c0a3a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009d70 tx=0x7f9e34009260 comp rx=0 tx=0).stop 2023-10-26T06:45:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.949+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441066b0 msgr2=0x7f9e44196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:12.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.949+0000 7f9e493ff700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441066b0 0x7f9e44196980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007ee0 tx=0x7f9e3c00d3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 shutdown_connections 2023-10-26T06:45:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9e2c0a1560 0x7f9e2c0a3a20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e441066b0 0x7f9e44196980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e441006a0 0x7f9e4419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 --2- 172.21.15.89:0/1234055047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e44101060 0x7f9e4419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 >> 172.21.15.89:0/1234055047 conn(0x7f9e440fc170 msgr2=0x7f9e440fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 shutdown_connections 2023-10-26T06:45:12.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:12.950+0000 7f9e493ff700 1 -- 172.21.15.89:0/1234055047 wait complete. 2023-10-26T06:45:12.973 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T06:45:12.974 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T06:45:13.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 -- 172.21.15.89:0/2008347331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 msgr2=0x7f9c54100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 --2- 172.21.15.89:0/2008347331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 0x7f9c54100b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0098f0 tx=0x7f9c3c009c00 comp rx=0 tx=0).stop 2023-10-26T06:45:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 -- 172.21.15.89:0/2008347331 shutdown_connections 2023-10-26T06:45:13.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 --2- 172.21.15.89:0/2008347331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 0x7f9c5410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 --2- 172.21.15.89:0/2008347331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c541066b0 0x7f9c54106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 --2- 172.21.15.89:0/2008347331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 0x7f9c54100b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 -- 172.21.15.89:0/2008347331 >> 172.21.15.89:0/2008347331 conn(0x7f9c540fc170 msgr2=0x7f9c540fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 -- 172.21.15.89:0/2008347331 shutdown_connections 2023-10-26T06:45:13.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.170+0000 7f9c59bff700 1 -- 172.21.15.89:0/2008347331 wait complete. 2023-10-26T06:45:13.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.171+0000 7f9c59bff700 1 Processor -- start 2023-10-26T06:45:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.171+0000 7f9c59bff700 1 -- start start 2023-10-26T06:45:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c59bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 0x7f9c5419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c59bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 0x7f9c5419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c59bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c541066b0 0x7f9c54198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c59bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c541991f0 con 0x7f9c54101060 2023-10-26T06:45:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c59bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54199360 con 0x7f9c541006a0 2023-10-26T06:45:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c59bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54199520 con 0x7f9c541066b0 2023-10-26T06:45:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 0x7f9c5419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 0x7f9c5419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60276/0 (socket says 172.21.15.89:60276) 2023-10-26T06:45:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 -- 172.21.15.89:0/3406248618 learned_addr learned my addr 172.21.15.89:0/3406248618 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c53fff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c541066b0 0x7f9c54198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c537fe700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 0x7f9c5419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 -- 172.21.15.89:0/3406248618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c541066b0 msgr2=0x7f9c54198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c541066b0 0x7f9c54198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 -- 172.21.15.89:0/3406248618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 msgr2=0x7f9c5419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 0x7f9c5419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 -- 172.21.15.89:0/3406248618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c009d00 con 0x7f9c54101060 2023-10-26T06:45:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.172+0000 7f9c4bfff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 0x7f9c5419efa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c005950 tx=0x7f9c3c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:13.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.173+0000 7f9c517fa700 1 -- 172.21.15.89:0/3406248618 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c01c070 con 0x7f9c54101060 2023-10-26T06:45:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.173+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c009520 con 0x7f9c54101060 2023-10-26T06:45:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.173+0000 7f9c517fa700 1 -- 172.21.15.89:0/3406248618 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c3c00baa0 con 0x7f9c54101060 2023-10-26T06:45:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.173+0000 7f9c517fa700 1 -- 172.21.15.89:0/3406248618 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0038e0 con 0x7f9c54101060 2023-10-26T06:45:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.173+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c541a5870 con 0x7f9c54101060 2023-10-26T06:45:13.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.174+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c54101de0 con 0x7f9c54101060 2023-10-26T06:45:13.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.176+0000 7f9c517fa700 1 -- 172.21.15.89:0/3406248618 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9c3c020030 con 0x7f9c54101060 2023-10-26T06:45:13.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.177+0000 7f9c517fa700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c400a1560 0x7f9c400a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.177+0000 7f9c517fa700 1 -- 172.21.15.89:0/3406248618 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f9c3c0cddd0 con 0x7f9c54101060 2023-10-26T06:45:13.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.177+0000 7f9c537fe700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c400a1560 0x7f9c400a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.178+0000 7f9c537fe700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c400a1560 0x7f9c400a3a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009be0 tx=0x7f9c44009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:13.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.179+0000 7f9c517fa700 1 -- 172.21.15.89:0/3406248618 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9c3c097380 con 0x7f9c54101060 2023-10-26T06:45:13.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:13 smithi089 ceph-mon[95031]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:13.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:13 smithi089 ceph-mon[95031]: pgmap v257: 97 pgs: 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 901 B/s rd, 2.1 KiB/s wr, 7 op/s 2023-10-26T06:45:13.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:13 smithi089 ceph-mon[95031]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:13.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:13 smithi089 ceph-mon[95031]: mgrmap e21: x(active, since 6m), standbys: z, y 2023-10-26T06:45:13.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:13 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2845103052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:45:13.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:13 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1234055047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:13.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.378+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c54199fe0 con 0x7f9c400a1560 2023-10-26T06:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.390+0000 7f9c517fa700 1 -- 172.21.15.89:0/3406248618 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335167 (secure 0 0 0) 0x7f9c54199fe0 con 0x7f9c400a1560 2023-10-26T06:45:13.398 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:13.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:13 smithi192 ceph-mon[131661]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:13.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:13 smithi192 ceph-mon[131661]: pgmap v257: 97 pgs: 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 901 B/s rd, 2.1 KiB/s wr, 7 op/s 2023-10-26T06:45:13.402 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:13 smithi192 ceph-mon[131661]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:13.402 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:13 smithi192 ceph-mon[131661]: mgrmap e21: x(active, since 6m), standbys: z, y 2023-10-26T06:45:13.402 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:13 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2845103052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:45:13.402 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:13 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1234055047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c400a1560 msgr2=0x7f9c400a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c400a1560 0x7f9c400a3a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009be0 tx=0x7f9c44009260 comp rx=0 tx=0).stop 2023-10-26T06:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 msgr2=0x7f9c5419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 0x7f9c5419efa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c005950 tx=0x7f9c3c00fa00 comp rx=0 tx=0).stop 2023-10-26T06:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 shutdown_connections 2023-10-26T06:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c400a1560 0x7f9c400a3a20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c541066b0 0x7f9c54198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c541006a0 0x7f9c5419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 --2- 172.21.15.89:0/3406248618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9c54101060 0x7f9c5419efa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 >> 172.21.15.89:0/3406248618 conn(0x7f9c540fc170 msgr2=0x7f9c5410bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 shutdown_connections 2023-10-26T06:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.395+0000 7f9c59bff700 1 -- 172.21.15.89:0/3406248618 wait complete. 2023-10-26T06:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-26T06:45:13.416 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2023-10-26T06:45:11.714131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":142,"num_read_kb":125,"num_write":226,"num_write_kb":2952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":214,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336644,"kb_used_data":12376,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605180,"statfs":{"total":1151940427776,"available":1151595704320,"internally_reserved":0,"allocated":12673024,"data_stored":7068257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.817462"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600110+0000","last_change":"2023-10-26T06:45:05.903170+0000","last_active":"2023-10-26T06:45:08.600110+0000","last_peered":"2023-10-26T06:45:08.600110+0000","last_clean":"2023-10-26T06:45:08.600110+0000","last_became_active":"2023-10-26T06:45:05.902574+0000","last_became_peered":"2023-10-26T06:45:05.902574+0000","last_unstale":"2023-10-26T06:45:08.600110+0000","last_undegraded":"2023-10-26T06:45:08.600110+0000","last_fullsized":"2023-10-26T06:45:08.600110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:33.065828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.912572+0000","last_change":"2023-10-26T06:45:05.911249+0000","last_active":"2023-10-26T06:45:05.912572+0000","last_peered":"2023-10-26T06:45:05.912572+0000","last_clean":"2023-10-26T06:45:05.912572+0000","last_became_active":"2023-10-26T06:45:05.910859+0000","last_became_peered":"2023-10-26T06:45:05.910859+0000","last_unstale":"2023-10-26T06:45:05.912572+0000","last_undegraded":"2023-10-26T06:45:05.912572+0000","last_fullsized":"2023-10-26T06:45:05.912572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:48:44.368834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.912112+0000","last_change":"2023-10-26T06:45:05.909219+0000","last_active":"2023-10-26T06:45:05.912112+0000","last_peered":"2023-10-26T06:45:05.912112+0000","last_clean":"2023-10-26T06:45:05.912112+0000","last_became_active":"2023-10-26T06:45:05.906721+0000","last_became_peered":"2023-10-26T06:45:05.906721+0000","last_unstale":"2023-10-26T06:45:05.912112+0000","last_undegraded":"2023-10-26T06:45:05.912112+0000","last_fullsized":"2023-10-26T06:45:05.912112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:44.834023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823998+0000","last_change":"2023-10-26T06:45:05.897514+0000","last_active":"2023-10-26T06:45:07.823998+0000","last_peered":"2023-10-26T06:45:07.823998+0000","last_clean":"2023-10-26T06:45:07.823998+0000","last_became_active":"2023-10-26T06:45:05.897201+0000","last_became_peered":"2023-10-26T06:45:05.897201+0000","last_unstale":"2023-10-26T06:45:07.823998+0000","last_undegraded":"2023-10-26T06:45:07.823998+0000","last_fullsized":"2023-10-26T06:45:07.823998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:39.508654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689213+0000","last_change":"2023-10-26T06:45:05.900475+0000","last_active":"2023-10-26T06:45:07.689213+0000","last_peered":"2023-10-26T06:45:07.689213+0000","last_clean":"2023-10-26T06:45:07.689213+0000","last_became_active":"2023-10-26T06:45:05.900313+0000","last_became_peered":"2023-10-26T06:45:05.900313+0000","last_unstale":"2023-10-26T06:45:07.689213+0000","last_undegraded":"2023-10-26T06:45:07.689213+0000","last_fullsized":"2023-10-26T06:45:07.689213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:51.438741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910704+0000","last_change":"2023-10-26T06:45:05.909040+0000","last_active":"2023-10-26T06:45:05.910704+0000","last_peered":"2023-10-26T06:45:05.910704+0000","last_clean":"2023-10-26T06:45:05.910704+0000","last_became_active":"2023-10-26T06:45:05.908696+0000","last_became_peered":"2023-10-26T06:45:05.908696+0000","last_unstale":"2023-10-26T06:45:05.910704+0000","last_undegraded":"2023-10-26T06:45:05.910704+0000","last_fullsized":"2023-10-26T06:45:05.910704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:46:36.916293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.903516+0000","last_change":"2023-10-26T06:45:05.898356+0000","last_active":"2023-10-26T06:45:05.903516+0000","last_peered":"2023-10-26T06:45:05.903516+0000","last_clean":"2023-10-26T06:45:05.903516+0000","last_became_active":"2023-10-26T06:45:05.898008+0000","last_became_peered":"2023-10-26T06:45:05.898008+0000","last_unstale":"2023-10-26T06:45:05.903516+0000","last_undegraded":"2023-10-26T06:45:05.903516+0000","last_fullsized":"2023-10-26T06:45:05.903516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:50:18.545454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.912809+0000","last_change":"2023-10-26T06:45:05.911301+0000","last_active":"2023-10-26T06:45:05.912809+0000","last_peered":"2023-10-26T06:45:05.912809+0000","last_clean":"2023-10-26T06:45:05.912809+0000","last_became_active":"2023-10-26T06:45:05.910667+0000","last_became_peered":"2023-10-26T06:45:05.910667+0000","last_unstale":"2023-10-26T06:45:05.912809+0000","last_undegraded":"2023-10-26T06:45:05.912809+0000","last_fullsized":"2023-10-26T06:45:05.912809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:59.086636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885616+0000","last_change":"2023-10-26T06:45:05.904456+0000","last_active":"2023-10-26T06:45:06.885616+0000","last_peered":"2023-10-26T06:45:06.885616+0000","last_clean":"2023-10-26T06:45:06.885616+0000","last_became_active":"2023-10-26T06:45:05.904165+0000","last_became_peered":"2023-10-26T06:45:05.904165+0000","last_unstale":"2023-10-26T06:45:06.885616+0000","last_undegraded":"2023-10-26T06:45:06.885616+0000","last_fullsized":"2023-10-26T06:45:06.885616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:51:01.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910742+0000","last_change":"2023-10-26T06:45:05.909544+0000","last_active":"2023-10-26T06:45:05.910742+0000","last_peered":"2023-10-26T06:45:05.910742+0000","last_clean":"2023-10-26T06:45:05.910742+0000","last_became_active":"2023-10-26T06:45:05.909103+0000","last_became_peered":"2023-10-26T06:45:05.909103+0000","last_unstale":"2023-10-26T06:45:05.910742+0000","last_undegraded":"2023-10-26T06:45:05.910742+0000","last_fullsized":"2023-10-26T06:45:05.910742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:19:17.935675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910405+0000","last_change":"2023-10-26T06:45:05.906549+0000","last_active":"2023-10-26T06:45:05.910405+0000","last_peered":"2023-10-26T06:45:05.910405+0000","last_clean":"2023-10-26T06:45:05.910405+0000","last_became_active":"2023-10-26T06:45:05.905883+0000","last_became_peered":"2023-10-26T06:45:05.905883+0000","last_unstale":"2023-10-26T06:45:05.910405+0000","last_undegraded":"2023-10-26T06:45:05.910405+0000","last_fullsized":"2023-10-26T06:45:05.910405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:07:03.435494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823855+0000","last_change":"2023-10-26T06:45:05.909460+0000","last_active":"2023-10-26T06:45:07.823855+0000","last_peered":"2023-10-26T06:45:07.823855+0000","last_clean":"2023-10-26T06:45:07.823855+0000","last_became_active":"2023-10-26T06:45:05.909007+0000","last_became_peered":"2023-10-26T06:45:05.909007+0000","last_unstale":"2023-10-26T06:45:07.823855+0000","last_undegraded":"2023-10-26T06:45:07.823855+0000","last_fullsized":"2023-10-26T06:45:07.823855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:45:55.817606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822132+0000","last_change":"2023-10-26T06:45:05.908305+0000","last_active":"2023-10-26T06:45:07.822132+0000","last_peered":"2023-10-26T06:45:07.822132+0000","last_clean":"2023-10-26T06:45:07.822132+0000","last_became_active":"2023-10-26T06:45:05.907931+0000","last_became_peered":"2023-10-26T06:45:05.907931+0000","last_unstale":"2023-10-26T06:45:07.822132+0000","last_undegraded":"2023-10-26T06:45:07.822132+0000","last_fullsized":"2023-10-26T06:45:07.822132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:00:43.605324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.911791+0000","last_change":"2023-10-26T06:45:05.909737+0000","last_active":"2023-10-26T06:45:05.911791+0000","last_peered":"2023-10-26T06:45:05.911791+0000","last_clean":"2023-10-26T06:45:05.911791+0000","last_became_active":"2023-10-26T06:45:05.909445+0000","last_became_peered":"2023-10-26T06:45:05.909445+0000","last_unstale":"2023-10-26T06:45:05.911791+0000","last_undegraded":"2023-10-26T06:45:05.911791+0000","last_fullsized":"2023-10-26T06:45:05.911791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:17:44.064749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.911098+0000","last_change":"2023-10-26T06:45:05.907722+0000","last_active":"2023-10-26T06:45:05.911098+0000","last_peered":"2023-10-26T06:45:05.911098+0000","last_clean":"2023-10-26T06:45:05.911098+0000","last_became_active":"2023-10-26T06:45:05.905622+0000","last_became_peered":"2023-10-26T06:45:05.905622+0000","last_unstale":"2023-10-26T06:45:05.911098+0000","last_undegraded":"2023-10-26T06:45:05.911098+0000","last_fullsized":"2023-10-26T06:45:05.911098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:00.811857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885013+0000","last_change":"2023-10-26T06:45:05.907801+0000","last_active":"2023-10-26T06:45:06.885013+0000","last_peered":"2023-10-26T06:45:06.885013+0000","last_clean":"2023-10-26T06:45:06.885013+0000","last_became_active":"2023-10-26T06:45:05.907485+0000","last_became_peered":"2023-10-26T06:45:05.907485+0000","last_unstale":"2023-10-26T06:45:06.885013+0000","last_undegraded":"2023-10-26T06:45:06.885013+0000","last_fullsized":"2023-10-26T06:45:06.885013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:16.850992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600141+0000","last_change":"2023-10-26T06:45:05.902859+0000","last_active":"2023-10-26T06:45:08.600141+0000","last_peered":"2023-10-26T06:45:08.600141+0000","last_clean":"2023-10-26T06:45:08.600141+0000","last_became_active":"2023-10-26T06:45:05.902218+0000","last_became_peered":"2023-10-26T06:45:05.902218+0000","last_unstale":"2023-10-26T06:45:08.600141+0000","last_undegraded":"2023-10-26T06:45:08.600141+0000","last_fullsized":"2023-10-26T06:45:08.600141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:47.717227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884920+0000","last_change":"2023-10-26T06:45:05.910445+0000","last_active":"2023-10-26T06:45:06.884920+0000","last_peered":"2023-10-26T06:45:06.884920+0000","last_clean":"2023-10-26T06:45:06.884920+0000","last_became_active":"2023-10-26T06:45:05.910043+0000","last_became_peered":"2023-10-26T06:45:05.910043+0000","last_unstale":"2023-10-26T06:45:06.884920+0000","last_undegraded":"2023-10-26T06:45:06.884920+0000","last_fullsized":"2023-10-26T06:45:06.884920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:33.751134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884907+0000","last_change":"2023-10-26T06:45:05.905340+0000","last_active":"2023-10-26T06:45:06.884907+0000","last_peered":"2023-10-26T06:45:06.884907+0000","last_clean":"2023-10-26T06:45:06.884907+0000","last_became_active":"2023-10-26T06:45:05.905030+0000","last_became_peered":"2023-10-26T06:45:05.905030+0000","last_unstale":"2023-10-26T06:45:06.884907+0000","last_undegraded":"2023-10-26T06:45:06.884907+0000","last_fullsized":"2023-10-26T06:45:06.884907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.303210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884956+0000","last_change":"2023-10-26T06:45:04.919128+0000","last_active":"2023-10-26T06:45:06.884956+0000","last_peered":"2023-10-26T06:45:06.884956+0000","last_clean":"2023-10-26T06:45:06.884956+0000","last_became_active":"2023-10-26T06:45:04.913034+0000","last_became_peered":"2023-10-26T06:45:04.913034+0000","last_unstale":"2023-10-26T06:45:06.884956+0000","last_undegraded":"2023-10-26T06:45:06.884956+0000","last_fullsized":"2023-10-26T06:45:06.884956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:50.835037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.906882+0000","last_change":"2023-10-26T06:45:05.905315+0000","last_active":"2023-10-26T06:45:05.906882+0000","last_peered":"2023-10-26T06:45:05.906882+0000","last_clean":"2023-10-26T06:45:05.906882+0000","last_became_active":"2023-10-26T06:45:05.904260+0000","last_became_peered":"2023-10-26T06:45:05.904260+0000","last_unstale":"2023-10-26T06:45:05.906882+0000","last_undegraded":"2023-10-26T06:45:05.906882+0000","last_fullsized":"2023-10-26T06:45:05.906882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:31:13.696617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.061825+0000","last_change":"2023-10-26T06:45:04.920440+0000","last_active":"2023-10-26T06:45:06.061825+0000","last_peered":"2023-10-26T06:45:06.061825+0000","last_clean":"2023-10-26T06:45:06.061825+0000","last_became_active":"2023-10-26T06:45:04.920129+0000","last_became_peered":"2023-10-26T06:45:04.920129+0000","last_unstale":"2023-10-26T06:45:06.061825+0000","last_undegraded":"2023-10-26T06:45:06.061825+0000","last_fullsized":"2023-10-26T06:45:06.061825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:43.176830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.907143+0000","last_change":"2023-10-26T06:45:05.902898+0000","last_active":"2023-10-26T06:45:05.907143+0000","last_peered":"2023-10-26T06:45:05.907143+0000","last_clean":"2023-10-26T06:45:05.907143+0000","last_became_active":"2023-10-26T06:45:05.902161+0000","last_became_peered":"2023-10-26T06:45:05.902161+0000","last_unstale":"2023-10-26T06:45:05.907143+0000","last_undegraded":"2023-10-26T06:45:05.907143+0000","last_fullsized":"2023-10-26T06:45:05.907143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:06.626718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063544+0000","last_change":"2023-10-26T06:45:04.911835+0000","last_active":"2023-10-26T06:45:06.063544+0000","last_peered":"2023-10-26T06:45:06.063544+0000","last_clean":"2023-10-26T06:45:06.063544+0000","last_became_active":"2023-10-26T06:45:04.911505+0000","last_became_peered":"2023-10-26T06:45:04.911505+0000","last_unstale":"2023-10-26T06:45:06.063544+0000","last_undegraded":"2023-10-26T06:45:06.063544+0000","last_fullsized":"2023-10-26T06:45:06.063544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:01:18.971381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885164+0000","last_change":"2023-10-26T06:45:05.909088+0000","last_active":"2023-10-26T06:45:06.885164+0000","last_peered":"2023-10-26T06:45:06.885164+0000","last_clean":"2023-10-26T06:45:06.885164+0000","last_became_active":"2023-10-26T06:45:05.908528+0000","last_became_peered":"2023-10-26T06:45:05.908528+0000","last_unstale":"2023-10-26T06:45:06.885164+0000","last_undegraded":"2023-10-26T06:45:06.885164+0000","last_fullsized":"2023-10-26T06:45:06.885164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:07.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063027+0000","last_change":"2023-10-26T06:45:04.913604+0000","last_active":"2023-10-26T06:45:06.063027+0000","last_peered":"2023-10-26T06:45:06.063027+0000","last_clean":"2023-10-26T06:45:06.063027+0000","last_became_active":"2023-10-26T06:45:04.913318+0000","last_became_peered":"2023-10-26T06:45:04.913318+0000","last_unstale":"2023-10-26T06:45:06.063027+0000","last_undegraded":"2023-10-26T06:45:06.063027+0000","last_fullsized":"2023-10-26T06:45:06.063027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:19:51.144044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822837+0000","last_change":"2023-10-26T06:45:05.898795+0000","last_active":"2023-10-26T06:45:07.822837+0000","last_peered":"2023-10-26T06:45:07.822837+0000","last_clean":"2023-10-26T06:45:07.822837+0000","last_became_active":"2023-10-26T06:45:05.898489+0000","last_became_peered":"2023-10-26T06:45:05.898489+0000","last_unstale":"2023-10-26T06:45:07.822837+0000","last_undegraded":"2023-10-26T06:45:07.822837+0000","last_fullsized":"2023-10-26T06:45:07.822837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:55.343736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062447+0000","last_change":"2023-10-26T06:45:04.908881+0000","last_active":"2023-10-26T06:45:06.062447+0000","last_peered":"2023-10-26T06:45:06.062447+0000","last_clean":"2023-10-26T06:45:06.062447+0000","last_became_active":"2023-10-26T06:45:04.908515+0000","last_became_peered":"2023-10-26T06:45:04.908515+0000","last_unstale":"2023-10-26T06:45:06.062447+0000","last_undegraded":"2023-10-26T06:45:06.062447+0000","last_fullsized":"2023-10-26T06:45:06.062447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:39.200294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689203+0000","last_change":"2023-10-26T06:45:05.901281+0000","last_active":"2023-10-26T06:45:07.689203+0000","last_peered":"2023-10-26T06:45:07.689203+0000","last_clean":"2023-10-26T06:45:07.689203+0000","last_became_active":"2023-10-26T06:45:05.900995+0000","last_became_peered":"2023-10-26T06:45:05.900995+0000","last_unstale":"2023-10-26T06:45:07.689203+0000","last_undegraded":"2023-10-26T06:45:07.689203+0000","last_fullsized":"2023-10-26T06:45:07.689203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:44:34.977571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.061992+0000","last_change":"2023-10-26T06:45:04.918480+0000","last_active":"2023-10-26T06:45:06.061992+0000","last_peered":"2023-10-26T06:45:06.061992+0000","last_clean":"2023-10-26T06:45:06.061992+0000","last_became_active":"2023-10-26T06:45:04.918185+0000","last_became_peered":"2023-10-26T06:45:04.918185+0000","last_unstale":"2023-10-26T06:45:06.061992+0000","last_undegraded":"2023-10-26T06:45:06.061992+0000","last_fullsized":"2023-10-26T06:45:06.061992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:07:58.263116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822711+0000","last_change":"2023-10-26T06:45:05.901825+0000","last_active":"2023-10-26T06:45:07.822711+0000","last_peered":"2023-10-26T06:45:07.822711+0000","last_clean":"2023-10-26T06:45:07.822711+0000","last_became_active":"2023-10-26T06:45:05.901509+0000","last_became_peered":"2023-10-26T06:45:05.901509+0000","last_unstale":"2023-10-26T06:45:07.822711+0000","last_undegraded":"2023-10-26T06:45:07.822711+0000","last_fullsized":"2023-10-26T06:45:07.822711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:16.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534260+0000","last_change":"2023-10-26T06:45:04.919813+0000","last_active":"2023-10-26T06:45:06.534260+0000","last_peered":"2023-10-26T06:45:06.534260+0000","last_clean":"2023-10-26T06:45:06.534260+0000","last_became_active":"2023-10-26T06:45:04.918016+0000","last_became_peered":"2023-10-26T06:45:04.918016+0000","last_unstale":"2023-10-26T06:45:06.534260+0000","last_undegraded":"2023-10-26T06:45:06.534260+0000","last_fullsized":"2023-10-26T06:45:06.534260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:32.336126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.904987+0000","last_change":"2023-10-26T06:45:05.902182+0000","last_active":"2023-10-26T06:45:05.904987+0000","last_peered":"2023-10-26T06:45:05.904987+0000","last_clean":"2023-10-26T06:45:05.904987+0000","last_became_active":"2023-10-26T06:45:05.901428+0000","last_became_peered":"2023-10-26T06:45:05.901428+0000","last_unstale":"2023-10-26T06:45:05.904987+0000","last_undegraded":"2023-10-26T06:45:05.904987+0000","last_fullsized":"2023-10-26T06:45:05.904987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:33.150881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.890140+0000","last_change":"2023-10-26T06:45:04.909713+0000","last_active":"2023-10-26T06:45:05.890140+0000","last_peered":"2023-10-26T06:45:05.890140+0000","last_clean":"2023-10-26T06:45:05.890140+0000","last_became_active":"2023-10-26T06:45:04.909412+0000","last_became_peered":"2023-10-26T06:45:04.909412+0000","last_unstale":"2023-10-26T06:45:05.890140+0000","last_undegraded":"2023-10-26T06:45:05.890140+0000","last_fullsized":"2023-10-26T06:45:05.890140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:23.901745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884924+0000","last_change":"2023-10-26T06:45:05.908348+0000","last_active":"2023-10-26T06:45:06.884924+0000","last_peered":"2023-10-26T06:45:06.884924+0000","last_clean":"2023-10-26T06:45:06.884924+0000","last_became_active":"2023-10-26T06:45:05.908022+0000","last_became_peered":"2023-10-26T06:45:05.908022+0000","last_unstale":"2023-10-26T06:45:06.884924+0000","last_undegraded":"2023-10-26T06:45:06.884924+0000","last_fullsized":"2023-10-26T06:45:06.884924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:22:16.027732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.901821+0000","last_change":"2023-10-26T06:45:04.915491+0000","last_active":"2023-10-26T06:45:05.901821+0000","last_peered":"2023-10-26T06:45:05.901821+0000","last_clean":"2023-10-26T06:45:05.901821+0000","last_became_active":"2023-10-26T06:45:04.915191+0000","last_became_peered":"2023-10-26T06:45:04.915191+0000","last_unstale":"2023-10-26T06:45:05.901821+0000","last_undegraded":"2023-10-26T06:45:05.901821+0000","last_fullsized":"2023-10-26T06:45:05.901821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:57.758903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885545+0000","last_change":"2023-10-26T06:45:05.912676+0000","last_active":"2023-10-26T06:45:06.885545+0000","last_peered":"2023-10-26T06:45:06.885545+0000","last_clean":"2023-10-26T06:45:06.885545+0000","last_became_active":"2023-10-26T06:45:05.912372+0000","last_became_peered":"2023-10-26T06:45:05.912372+0000","last_unstale":"2023-10-26T06:45:06.885545+0000","last_undegraded":"2023-10-26T06:45:06.885545+0000","last_fullsized":"2023-10-26T06:45:06.885545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:33.299166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.893012+0000","last_change":"2023-10-26T06:45:04.912123+0000","last_active":"2023-10-26T06:45:05.893012+0000","last_peered":"2023-10-26T06:45:05.893012+0000","last_clean":"2023-10-26T06:45:05.893012+0000","last_became_active":"2023-10-26T06:45:04.911893+0000","last_became_peered":"2023-10-26T06:45:04.911893+0000","last_unstale":"2023-10-26T06:45:05.893012+0000","last_undegraded":"2023-10-26T06:45:05.893012+0000","last_fullsized":"2023-10-26T06:45:05.893012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:32.907496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.894888+0000","last_change":"2023-10-26T06:45:05.893434+0000","last_active":"2023-10-26T06:45:05.894888+0000","last_peered":"2023-10-26T06:45:05.894888+0000","last_clean":"2023-10-26T06:45:05.894888+0000","last_became_active":"2023-10-26T06:45:05.893100+0000","last_became_peered":"2023-10-26T06:45:05.893100+0000","last_unstale":"2023-10-26T06:45:05.894888+0000","last_undegraded":"2023-10-26T06:45:05.894888+0000","last_fullsized":"2023-10-26T06:45:05.894888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:46.669010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062642+0000","last_change":"2023-10-26T06:45:04.908910+0000","last_active":"2023-10-26T06:45:06.062642+0000","last_peered":"2023-10-26T06:45:06.062642+0000","last_clean":"2023-10-26T06:45:06.062642+0000","last_became_active":"2023-10-26T06:45:04.908592+0000","last_became_peered":"2023-10-26T06:45:04.908592+0000","last_unstale":"2023-10-26T06:45:06.062642+0000","last_undegraded":"2023-10-26T06:45:06.062642+0000","last_fullsized":"2023-10-26T06:45:06.062642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:40.266013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885272+0000","last_change":"2023-10-26T06:45:05.908887+0000","last_active":"2023-10-26T06:45:06.885272+0000","last_peered":"2023-10-26T06:45:06.885272+0000","last_clean":"2023-10-26T06:45:06.885272+0000","last_became_active":"2023-10-26T06:45:05.908163+0000","last_became_peered":"2023-10-26T06:45:05.908163+0000","last_unstale":"2023-10-26T06:45:06.885272+0000","last_undegraded":"2023-10-26T06:45:06.885272+0000","last_fullsized":"2023-10-26T06:45:06.885272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:09.339374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'6","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.064035+0000","last_change":"2023-10-26T06:45:04.905691+0000","last_active":"2023-10-26T06:45:06.064035+0000","last_peered":"2023-10-26T06:45:06.064035+0000","last_clean":"2023-10-26T06:45:06.064035+0000","last_became_active":"2023-10-26T06:45:04.905378+0000","last_became_peered":"2023-10-26T06:45:04.905378+0000","last_unstale":"2023-10-26T06:45:06.064035+0000","last_undegraded":"2023-10-26T06:45:06.064035+0000","last_fullsized":"2023-10-26T06:45:06.064035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:24:21.498141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689388+0000","last_change":"2023-10-26T06:45:05.897366+0000","last_active":"2023-10-26T06:45:07.689388+0000","last_peered":"2023-10-26T06:45:07.689388+0000","last_clean":"2023-10-26T06:45:07.689388+0000","last_became_active":"2023-10-26T06:45:05.897091+0000","last_became_peered":"2023-10-26T06:45:05.897091+0000","last_unstale":"2023-10-26T06:45:07.689388+0000","last_undegraded":"2023-10-26T06:45:07.689388+0000","last_fullsized":"2023-10-26T06:45:07.689388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:11.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884895+0000","last_change":"2023-10-26T06:45:04.914949+0000","last_active":"2023-10-26T06:45:06.884895+0000","last_peered":"2023-10-26T06:45:06.884895+0000","last_clean":"2023-10-26T06:45:06.884895+0000","last_became_active":"2023-10-26T06:45:04.914610+0000","last_became_peered":"2023-10-26T06:45:04.914610+0000","last_unstale":"2023-10-26T06:45:06.884895+0000","last_undegraded":"2023-10-26T06:45:06.884895+0000","last_fullsized":"2023-10-26T06:45:06.884895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:16.683352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822001+0000","last_change":"2023-10-26T06:45:05.911067+0000","last_active":"2023-10-26T06:45:07.822001+0000","last_peered":"2023-10-26T06:45:07.822001+0000","last_clean":"2023-10-26T06:45:07.822001+0000","last_became_active":"2023-10-26T06:45:05.908460+0000","last_became_peered":"2023-10-26T06:45:05.908460+0000","last_unstale":"2023-10-26T06:45:07.822001+0000","last_undegraded":"2023-10-26T06:45:07.822001+0000","last_fullsized":"2023-10-26T06:45:07.822001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:17.461781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.895349+0000","last_change":"2023-10-26T06:45:04.919947+0000","last_active":"2023-10-26T06:45:05.895349+0000","last_peered":"2023-10-26T06:45:05.895349+0000","last_clean":"2023-10-26T06:45:05.895349+0000","last_became_active":"2023-10-26T06:45:04.919535+0000","last_became_peered":"2023-10-26T06:45:04.919535+0000","last_unstale":"2023-10-26T06:45:05.895349+0000","last_undegraded":"2023-10-26T06:45:05.895349+0000","last_fullsized":"2023-10-26T06:45:05.895349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:46:09.201139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910356+0000","last_change":"2023-10-26T06:45:05.906381+0000","last_active":"2023-10-26T06:45:05.910356+0000","last_peered":"2023-10-26T06:45:05.910356+0000","last_clean":"2023-10-26T06:45:05.910356+0000","last_became_active":"2023-10-26T06:45:05.904487+0000","last_became_peered":"2023-10-26T06:45:05.904487+0000","last_unstale":"2023-10-26T06:45:05.910356+0000","last_undegraded":"2023-10-26T06:45:05.910356+0000","last_fullsized":"2023-10-26T06:45:05.910356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:54:34.443026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887797+0000","last_change":"2023-10-26T06:45:04.908779+0000","last_active":"2023-10-26T06:45:05.887797+0000","last_peered":"2023-10-26T06:45:05.887797+0000","last_clean":"2023-10-26T06:45:05.887797+0000","last_became_active":"2023-10-26T06:45:04.908490+0000","last_became_peered":"2023-10-26T06:45:04.908490+0000","last_unstale":"2023-10-26T06:45:05.887797+0000","last_undegraded":"2023-10-26T06:45:05.887797+0000","last_fullsized":"2023-10-26T06:45:05.887797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:12:13.199863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910431+0000","last_change":"2023-10-26T06:45:05.909408+0000","last_active":"2023-10-26T06:45:05.910431+0000","last_peered":"2023-10-26T06:45:05.910431+0000","last_clean":"2023-10-26T06:45:05.910431+0000","last_became_active":"2023-10-26T06:45:05.909107+0000","last_became_peered":"2023-10-26T06:45:05.909107+0000","last_unstale":"2023-10-26T06:45:05.910431+0000","last_undegraded":"2023-10-26T06:45:05.910431+0000","last_fullsized":"2023-10-26T06:45:05.910431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:27.961305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063862+0000","last_change":"2023-10-26T06:45:04.918194+0000","last_active":"2023-10-26T06:45:06.063862+0000","last_peered":"2023-10-26T06:45:06.063862+0000","last_clean":"2023-10-26T06:45:06.063862+0000","last_became_active":"2023-10-26T06:45:04.917850+0000","last_became_peered":"2023-10-26T06:45:04.917850+0000","last_unstale":"2023-10-26T06:45:06.063862+0000","last_undegraded":"2023-10-26T06:45:06.063862+0000","last_fullsized":"2023-10-26T06:45:06.063862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:25.743939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822581+0000","last_change":"2023-10-26T06:45:05.902156+0000","last_active":"2023-10-26T06:45:07.822581+0000","last_peered":"2023-10-26T06:45:07.822581+0000","last_clean":"2023-10-26T06:45:07.822581+0000","last_became_active":"2023-10-26T06:45:05.901865+0000","last_became_peered":"2023-10-26T06:45:05.901865+0000","last_unstale":"2023-10-26T06:45:07.822581+0000","last_undegraded":"2023-10-26T06:45:07.822581+0000","last_fullsized":"2023-10-26T06:45:07.822581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:06:08.167280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689452+0000","last_change":"2023-10-26T06:45:04.913071+0000","last_active":"2023-10-26T06:45:07.689452+0000","last_peered":"2023-10-26T06:45:07.689452+0000","last_clean":"2023-10-26T06:45:07.689452+0000","last_became_active":"2023-10-26T06:45:04.912736+0000","last_became_peered":"2023-10-26T06:45:04.912736+0000","last_unstale":"2023-10-26T06:45:07.689452+0000","last_undegraded":"2023-10-26T06:45:07.689452+0000","last_fullsized":"2023-10-26T06:45:07.689452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:59.690355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600012+0000","last_change":"2023-10-26T06:45:05.891949+0000","last_active":"2023-10-26T06:45:08.600012+0000","last_peered":"2023-10-26T06:45:08.600012+0000","last_clean":"2023-10-26T06:45:08.600012+0000","last_became_active":"2023-10-26T06:45:05.891583+0000","last_became_peered":"2023-10-26T06:45:05.891583+0000","last_unstale":"2023-10-26T06:45:08.600012+0000","last_undegraded":"2023-10-26T06:45:08.600012+0000","last_fullsized":"2023-10-26T06:45:08.600012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:54:18.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.883535+0000","last_change":"2023-10-26T06:45:04.913969+0000","last_active":"2023-10-26T06:45:05.883535+0000","last_peered":"2023-10-26T06:45:05.883535+0000","last_clean":"2023-10-26T06:45:05.883535+0000","last_became_active":"2023-10-26T06:45:04.913533+0000","last_became_peered":"2023-10-26T06:45:04.913533+0000","last_unstale":"2023-10-26T06:45:05.883535+0000","last_undegraded":"2023-10-26T06:45:05.883535+0000","last_fullsized":"2023-10-26T06:45:05.883535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:47:38.755102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.905200+0000","last_change":"2023-10-26T06:45:05.904832+0000","last_active":"2023-10-26T06:45:05.905200+0000","last_peered":"2023-10-26T06:45:05.905200+0000","last_clean":"2023-10-26T06:45:05.905200+0000","last_became_active":"2023-10-26T06:45:05.904527+0000","last_became_peered":"2023-10-26T06:45:05.904527+0000","last_unstale":"2023-10-26T06:45:05.905200+0000","last_undegraded":"2023-10-26T06:45:05.905200+0000","last_fullsized":"2023-10-26T06:45:05.905200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:53:15.849181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.894250+0000","last_change":"2023-10-26T06:45:04.922832+0000","last_active":"2023-10-26T06:45:05.894250+0000","last_peered":"2023-10-26T06:45:05.894250+0000","last_clean":"2023-10-26T06:45:05.894250+0000","last_became_active":"2023-10-26T06:45:04.922349+0000","last_became_peered":"2023-10-26T06:45:04.922349+0000","last_unstale":"2023-10-26T06:45:05.894250+0000","last_undegraded":"2023-10-26T06:45:05.894250+0000","last_fullsized":"2023-10-26T06:45:05.894250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:37.839093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910839+0000","last_change":"2023-10-26T06:45:05.909946+0000","last_active":"2023-10-26T06:45:05.910839+0000","last_peered":"2023-10-26T06:45:05.910839+0000","last_clean":"2023-10-26T06:45:05.910839+0000","last_became_active":"2023-10-26T06:45:05.909650+0000","last_became_peered":"2023-10-26T06:45:05.909650+0000","last_unstale":"2023-10-26T06:45:05.910839+0000","last_undegraded":"2023-10-26T06:45:05.910839+0000","last_fullsized":"2023-10-26T06:45:05.910839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:44:26.545423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822476+0000","last_change":"2023-10-26T06:45:05.903400+0000","last_active":"2023-10-26T06:45:07.822476+0000","last_peered":"2023-10-26T06:45:07.822476+0000","last_clean":"2023-10-26T06:45:07.822476+0000","last_became_active":"2023-10-26T06:45:05.903101+0000","last_became_peered":"2023-10-26T06:45:05.903101+0000","last_unstale":"2023-10-26T06:45:07.822476+0000","last_undegraded":"2023-10-26T06:45:07.822476+0000","last_fullsized":"2023-10-26T06:45:07.822476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534251+0000","last_change":"2023-10-26T06:45:04.923960+0000","last_active":"2023-10-26T06:45:06.534251+0000","last_peered":"2023-10-26T06:45:06.534251+0000","last_clean":"2023-10-26T06:45:06.534251+0000","last_became_active":"2023-10-26T06:45:04.923613+0000","last_became_peered":"2023-10-26T06:45:04.923613+0000","last_unstale":"2023-10-26T06:45:06.534251+0000","last_undegraded":"2023-10-26T06:45:06.534251+0000","last_fullsized":"2023-10-26T06:45:06.534251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:53.342463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.906822+0000","last_change":"2023-10-26T06:45:05.903966+0000","last_active":"2023-10-26T06:45:05.906822+0000","last_peered":"2023-10-26T06:45:05.906822+0000","last_clean":"2023-10-26T06:45:05.906822+0000","last_became_active":"2023-10-26T06:45:05.903583+0000","last_became_peered":"2023-10-26T06:45:05.903583+0000","last_unstale":"2023-10-26T06:45:05.906822+0000","last_undegraded":"2023-10-26T06:45:05.906822+0000","last_fullsized":"2023-10-26T06:45:05.906822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:34:59.561676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887855+0000","last_change":"2023-10-26T06:45:04.907838+0000","last_active":"2023-10-26T06:45:05.887855+0000","last_peered":"2023-10-26T06:45:05.887855+0000","last_clean":"2023-10-26T06:45:05.887855+0000","last_became_active":"2023-10-26T06:45:04.907508+0000","last_became_peered":"2023-10-26T06:45:04.907508+0000","last_unstale":"2023-10-26T06:45:05.887855+0000","last_undegraded":"2023-10-26T06:45:05.887855+0000","last_fullsized":"2023-10-26T06:45:05.887855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:34.852243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":318,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-26T06:45:04.908865+0000","last_change":"2023-10-26T06:43:59.765466+0000","last_active":"2023-10-26T06:45:04.908865+0000","last_peered":"2023-10-26T06:45:04.908865+0000","last_clean":"2023-10-26T06:45:04.908865+0000","last_became_active":"2023-10-26T06:43:59.765005+0000","last_became_peered":"2023-10-26T06:43:59.765005+0000","last_unstale":"2023-10-26T06:45:04.908865+0000","last_undegraded":"2023-10-26T06:45:04.908865+0000","last_fullsized":"2023-10-26T06:45:04.908865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_clean_scrub_stamp":"2023-10-26T06:40:50.556448+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:18.688941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.821877+0000","last_change":"2023-10-26T06:45:05.906909+0000","last_active":"2023-10-26T06:45:07.821877+0000","last_peered":"2023-10-26T06:45:07.821877+0000","last_clean":"2023-10-26T06:45:07.821877+0000","last_became_active":"2023-10-26T06:45:05.906531+0000","last_became_peered":"2023-10-26T06:45:05.906531+0000","last_unstale":"2023-10-26T06:45:07.821877+0000","last_undegraded":"2023-10-26T06:45:07.821877+0000","last_fullsized":"2023-10-26T06:45:07.821877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.892221+0000","last_change":"2023-10-26T06:45:04.911635+0000","last_active":"2023-10-26T06:45:05.892221+0000","last_peered":"2023-10-26T06:45:05.892221+0000","last_clean":"2023-10-26T06:45:05.892221+0000","last_became_active":"2023-10-26T06:45:04.911289+0000","last_became_peered":"2023-10-26T06:45:04.911289+0000","last_unstale":"2023-10-26T06:45:05.892221+0000","last_undegraded":"2023-10-26T06:45:05.892221+0000","last_fullsized":"2023-10-26T06:45:05.892221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.115211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888625+0000","last_change":"2023-10-26T06:45:04.919853+0000","last_active":"2023-10-26T06:45:05.888625+0000","last_peered":"2023-10-26T06:45:05.888625+0000","last_clean":"2023-10-26T06:45:05.888625+0000","last_became_active":"2023-10-26T06:45:04.919488+0000","last_became_peered":"2023-10-26T06:45:04.919488+0000","last_unstale":"2023-10-26T06:45:05.888625+0000","last_undegraded":"2023-10-26T06:45:05.888625+0000","last_fullsized":"2023-10-26T06:45:05.888625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:06.555851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822290+0000","last_change":"2023-10-26T06:45:05.903975+0000","last_active":"2023-10-26T06:45:07.822290+0000","last_peered":"2023-10-26T06:45:07.822290+0000","last_clean":"2023-10-26T06:45:07.822290+0000","last_became_active":"2023-10-26T06:45:05.903539+0000","last_became_peered":"2023-10-26T06:45:05.903539+0000","last_unstale":"2023-10-26T06:45:07.822290+0000","last_undegraded":"2023-10-26T06:45:07.822290+0000","last_fullsized":"2023-10-26T06:45:07.822290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:30.648947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885007+0000","last_change":"2023-10-26T06:45:04.919815+0000","last_active":"2023-10-26T06:45:06.885007+0000","last_peered":"2023-10-26T06:45:06.885007+0000","last_clean":"2023-10-26T06:45:06.885007+0000","last_became_active":"2023-10-26T06:45:04.918141+0000","last_became_peered":"2023-10-26T06:45:04.918141+0000","last_unstale":"2023-10-26T06:45:06.885007+0000","last_undegraded":"2023-10-26T06:45:06.885007+0000","last_fullsized":"2023-10-26T06:45:06.885007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:53.024534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.901748+0000","last_change":"2023-10-26T06:45:05.898180+0000","last_active":"2023-10-26T06:45:05.901748+0000","last_peered":"2023-10-26T06:45:05.901748+0000","last_clean":"2023-10-26T06:45:05.901748+0000","last_became_active":"2023-10-26T06:45:05.896937+0000","last_became_peered":"2023-10-26T06:45:05.896937+0000","last_unstale":"2023-10-26T06:45:05.901748+0000","last_undegraded":"2023-10-26T06:45:05.901748+0000","last_fullsized":"2023-10-26T06:45:05.901748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:23:23.957017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.061365+0000","last_change":"2023-10-26T06:45:04.922456+0000","last_active":"2023-10-26T06:45:06.061365+0000","last_peered":"2023-10-26T06:45:06.061365+0000","last_clean":"2023-10-26T06:45:06.061365+0000","last_became_active":"2023-10-26T06:45:04.922112+0000","last_became_peered":"2023-10-26T06:45:04.922112+0000","last_unstale":"2023-10-26T06:45:06.061365+0000","last_undegraded":"2023-10-26T06:45:06.061365+0000","last_fullsized":"2023-10-26T06:45:06.061365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:44.055569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.899049+0000","last_change":"2023-10-26T06:45:05.895440+0000","last_active":"2023-10-26T06:45:05.899049+0000","last_peered":"2023-10-26T06:45:05.899049+0000","last_clean":"2023-10-26T06:45:05.899049+0000","last_became_active":"2023-10-26T06:45:05.892605+0000","last_became_peered":"2023-10-26T06:45:05.892605+0000","last_unstale":"2023-10-26T06:45:05.899049+0000","last_undegraded":"2023-10-26T06:45:05.899049+0000","last_fullsized":"2023-10-26T06:45:05.899049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:30:17.954936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.899985+0000","last_change":"2023-10-26T06:45:04.916042+0000","last_active":"2023-10-26T06:45:05.899985+0000","last_peered":"2023-10-26T06:45:05.899985+0000","last_clean":"2023-10-26T06:45:05.899985+0000","last_became_active":"2023-10-26T06:45:04.915649+0000","last_became_peered":"2023-10-26T06:45:04.915649+0000","last_unstale":"2023-10-26T06:45:05.899985+0000","last_undegraded":"2023-10-26T06:45:05.899985+0000","last_fullsized":"2023-10-26T06:45:05.899985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:30.060074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689096+0000","last_change":"2023-10-26T06:45:05.908179+0000","last_active":"2023-10-26T06:45:07.689096+0000","last_peered":"2023-10-26T06:45:07.689096+0000","last_clean":"2023-10-26T06:45:07.689096+0000","last_became_active":"2023-10-26T06:45:05.907857+0000","last_became_peered":"2023-10-26T06:45:05.907857+0000","last_unstale":"2023-10-26T06:45:07.689096+0000","last_undegraded":"2023-10-26T06:45:07.689096+0000","last_fullsized":"2023-10-26T06:45:07.689096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:20.017194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.891280+0000","last_change":"2023-10-26T06:45:04.921016+0000","last_active":"2023-10-26T06:45:05.891280+0000","last_peered":"2023-10-26T06:45:05.891280+0000","last_clean":"2023-10-26T06:45:05.891280+0000","last_became_active":"2023-10-26T06:45:04.920741+0000","last_became_peered":"2023-10-26T06:45:04.920741+0000","last_unstale":"2023-10-26T06:45:05.891280+0000","last_undegraded":"2023-10-26T06:45:05.891280+0000","last_fullsized":"2023-10-26T06:45:05.891280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:05:06.218205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599772+0000","last_change":"2023-10-26T06:45:05.893562+0000","last_active":"2023-10-26T06:45:08.599772+0000","last_peered":"2023-10-26T06:45:08.599772+0000","last_clean":"2023-10-26T06:45:08.599772+0000","last_became_active":"2023-10-26T06:45:05.893111+0000","last_became_peered":"2023-10-26T06:45:05.893111+0000","last_unstale":"2023-10-26T06:45:08.599772+0000","last_undegraded":"2023-10-26T06:45:08.599772+0000","last_fullsized":"2023-10-26T06:45:08.599772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.305664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888454+0000","last_change":"2023-10-26T06:45:04.914936+0000","last_active":"2023-10-26T06:45:05.888454+0000","last_peered":"2023-10-26T06:45:05.888454+0000","last_clean":"2023-10-26T06:45:05.888454+0000","last_became_active":"2023-10-26T06:45:04.914617+0000","last_became_peered":"2023-10-26T06:45:04.914617+0000","last_unstale":"2023-10-26T06:45:05.888454+0000","last_undegraded":"2023-10-26T06:45:05.888454+0000","last_fullsized":"2023-10-26T06:45:05.888454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:15.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.909656+0000","last_change":"2023-10-26T06:45:05.909101+0000","last_active":"2023-10-26T06:45:05.909656+0000","last_peered":"2023-10-26T06:45:05.909656+0000","last_clean":"2023-10-26T06:45:05.909656+0000","last_became_active":"2023-10-26T06:45:05.908800+0000","last_became_peered":"2023-10-26T06:45:05.908800+0000","last_unstale":"2023-10-26T06:45:05.909656+0000","last_undegraded":"2023-10-26T06:45:05.909656+0000","last_fullsized":"2023-10-26T06:45:05.909656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:52:19.878102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689147+0000","last_change":"2023-10-26T06:45:04.911714+0000","last_active":"2023-10-26T06:45:07.689147+0000","last_peered":"2023-10-26T06:45:07.689147+0000","last_clean":"2023-10-26T06:45:07.689147+0000","last_became_active":"2023-10-26T06:45:04.911343+0000","last_became_peered":"2023-10-26T06:45:04.911343+0000","last_unstale":"2023-10-26T06:45:07.689147+0000","last_undegraded":"2023-10-26T06:45:07.689147+0000","last_fullsized":"2023-10-26T06:45:07.689147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:30:35.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910336+0000","last_change":"2023-10-26T06:45:05.906804+0000","last_active":"2023-10-26T06:45:05.910336+0000","last_peered":"2023-10-26T06:45:05.910336+0000","last_clean":"2023-10-26T06:45:05.910336+0000","last_became_active":"2023-10-26T06:45:05.906522+0000","last_became_peered":"2023-10-26T06:45:05.906522+0000","last_unstale":"2023-10-26T06:45:05.910336+0000","last_undegraded":"2023-10-26T06:45:05.910336+0000","last_fullsized":"2023-10-26T06:45:05.910336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:27.787840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888946+0000","last_change":"2023-10-26T06:45:04.906345+0000","last_active":"2023-10-26T06:45:05.888946+0000","last_peered":"2023-10-26T06:45:05.888946+0000","last_clean":"2023-10-26T06:45:05.888946+0000","last_became_active":"2023-10-26T06:45:04.906034+0000","last_became_peered":"2023-10-26T06:45:04.906034+0000","last_unstale":"2023-10-26T06:45:05.888946+0000","last_undegraded":"2023-10-26T06:45:05.888946+0000","last_fullsized":"2023-10-26T06:45:05.888946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:53.690788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.910387+0000","last_change":"2023-10-26T06:45:05.906368+0000","last_active":"2023-10-26T06:45:05.910387+0000","last_peered":"2023-10-26T06:45:05.910387+0000","last_clean":"2023-10-26T06:45:05.910387+0000","last_became_active":"2023-10-26T06:45:05.905471+0000","last_became_peered":"2023-10-26T06:45:05.905471+0000","last_unstale":"2023-10-26T06:45:05.910387+0000","last_undegraded":"2023-10-26T06:45:05.910387+0000","last_fullsized":"2023-10-26T06:45:05.910387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:42.866620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822626+0000","last_change":"2023-10-26T06:45:04.917530+0000","last_active":"2023-10-26T06:45:07.822626+0000","last_peered":"2023-10-26T06:45:07.822626+0000","last_clean":"2023-10-26T06:45:07.822626+0000","last_became_active":"2023-10-26T06:45:04.917193+0000","last_became_peered":"2023-10-26T06:45:04.917193+0000","last_unstale":"2023-10-26T06:45:07.822626+0000","last_undegraded":"2023-10-26T06:45:07.822626+0000","last_fullsized":"2023-10-26T06:45:07.822626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.965048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.903901+0000","last_change":"2023-10-26T06:45:05.899429+0000","last_active":"2023-10-26T06:45:05.903901+0000","last_peered":"2023-10-26T06:45:05.903901+0000","last_clean":"2023-10-26T06:45:05.903901+0000","last_became_active":"2023-10-26T06:45:05.899081+0000","last_became_peered":"2023-10-26T06:45:05.899081+0000","last_unstale":"2023-10-26T06:45:05.903901+0000","last_undegraded":"2023-10-26T06:45:05.903901+0000","last_fullsized":"2023-10-26T06:45:05.903901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:18:55.984836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534374+0000","last_change":"2023-10-26T06:45:04.920295+0000","last_active":"2023-10-26T06:45:06.534374+0000","last_peered":"2023-10-26T06:45:06.534374+0000","last_clean":"2023-10-26T06:45:06.534374+0000","last_became_active":"2023-10-26T06:45:04.919920+0000","last_became_peered":"2023-10-26T06:45:04.919920+0000","last_unstale":"2023-10-26T06:45:06.534374+0000","last_undegraded":"2023-10-26T06:45:06.534374+0000","last_fullsized":"2023-10-26T06:45:06.534374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:59.260841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.903942+0000","last_change":"2023-10-26T06:45:05.899610+0000","last_active":"2023-10-26T06:45:05.903942+0000","last_peered":"2023-10-26T06:45:05.903942+0000","last_clean":"2023-10-26T06:45:05.903942+0000","last_became_active":"2023-10-26T06:45:05.899283+0000","last_became_peered":"2023-10-26T06:45:05.899283+0000","last_unstale":"2023-10-26T06:45:05.903942+0000","last_undegraded":"2023-10-26T06:45:05.903942+0000","last_fullsized":"2023-10-26T06:45:05.903942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:45.381049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599798+0000","last_change":"2023-10-26T06:45:05.892012+0000","last_active":"2023-10-26T06:45:08.599798+0000","last_peered":"2023-10-26T06:45:08.599798+0000","last_clean":"2023-10-26T06:45:08.599798+0000","last_became_active":"2023-10-26T06:45:05.891744+0000","last_became_peered":"2023-10-26T06:45:05.891744+0000","last_unstale":"2023-10-26T06:45:08.599798+0000","last_undegraded":"2023-10-26T06:45:08.599798+0000","last_fullsized":"2023-10-26T06:45:08.599798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:29.599184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884873+0000","last_change":"2023-10-26T06:45:05.904348+0000","last_active":"2023-10-26T06:45:06.884873+0000","last_peered":"2023-10-26T06:45:06.884873+0000","last_clean":"2023-10-26T06:45:06.884873+0000","last_became_active":"2023-10-26T06:45:05.903949+0000","last_became_peered":"2023-10-26T06:45:05.903949+0000","last_unstale":"2023-10-26T06:45:06.884873+0000","last_undegraded":"2023-10-26T06:45:06.884873+0000","last_fullsized":"2023-10-26T06:45:06.884873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:29.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884744+0000","last_change":"2023-10-26T06:45:05.901313+0000","last_active":"2023-10-26T06:45:06.884744+0000","last_peered":"2023-10-26T06:45:06.884744+0000","last_clean":"2023-10-26T06:45:06.884744+0000","last_became_active":"2023-10-26T06:45:05.900993+0000","last_became_peered":"2023-10-26T06:45:05.900993+0000","last_unstale":"2023-10-26T06:45:06.884744+0000","last_undegraded":"2023-10-26T06:45:06.884744+0000","last_fullsized":"2023-10-26T06:45:06.884744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:00.093645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.903553+0000","last_change":"2023-10-26T06:45:05.901225+0000","last_active":"2023-10-26T06:45:05.903553+0000","last_peered":"2023-10-26T06:45:05.903553+0000","last_clean":"2023-10-26T06:45:05.903553+0000","last_became_active":"2023-10-26T06:45:05.900743+0000","last_became_peered":"2023-10-26T06:45:05.900743+0000","last_unstale":"2023-10-26T06:45:05.903553+0000","last_undegraded":"2023-10-26T06:45:05.903553+0000","last_fullsized":"2023-10-26T06:45:05.903553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:40:31.626289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822222+0000","last_change":"2023-10-26T06:45:05.906805+0000","last_active":"2023-10-26T06:45:07.822222+0000","last_peered":"2023-10-26T06:45:07.822222+0000","last_clean":"2023-10-26T06:45:07.822222+0000","last_became_active":"2023-10-26T06:45:05.906464+0000","last_became_peered":"2023-10-26T06:45:05.906464+0000","last_unstale":"2023-10-26T06:45:07.822222+0000","last_undegraded":"2023-10-26T06:45:07.822222+0000","last_fullsized":"2023-10-26T06:45:07.822222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:24.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599992+0000","last_change":"2023-10-26T06:45:05.898568+0000","last_active":"2023-10-26T06:45:08.599992+0000","last_peered":"2023-10-26T06:45:08.599992+0000","last_clean":"2023-10-26T06:45:08.599992+0000","last_became_active":"2023-10-26T06:45:05.897886+0000","last_became_peered":"2023-10-26T06:45:05.897886+0000","last_unstale":"2023-10-26T06:45:08.599992+0000","last_undegraded":"2023-10-26T06:45:08.599992+0000","last_fullsized":"2023-10-26T06:45:08.599992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:14.998233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885177+0000","last_change":"2023-10-26T06:45:05.899326+0000","last_active":"2023-10-26T06:45:06.885177+0000","last_peered":"2023-10-26T06:45:06.885177+0000","last_clean":"2023-10-26T06:45:06.885177+0000","last_became_active":"2023-10-26T06:45:05.899040+0000","last_became_peered":"2023-10-26T06:45:05.899040+0000","last_unstale":"2023-10-26T06:45:06.885177+0000","last_undegraded":"2023-10-26T06:45:06.885177+0000","last_fullsized":"2023-10-26T06:45:06.885177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:55:35.666122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.905884+0000","last_change":"2023-10-26T06:45:05.904155+0000","last_active":"2023-10-26T06:45:05.905884+0000","last_peered":"2023-10-26T06:45:05.905884+0000","last_clean":"2023-10-26T06:45:05.905884+0000","last_became_active":"2023-10-26T06:45:05.903734+0000","last_became_peered":"2023-10-26T06:45:05.903734+0000","last_unstale":"2023-10-26T06:45:05.905884+0000","last_undegraded":"2023-10-26T06:45:05.905884+0000","last_fullsized":"2023-10-26T06:45:05.905884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:31.210136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822424+0000","last_change":"2023-10-26T06:45:05.897174+0000","last_active":"2023-10-26T06:45:07.822424+0000","last_peered":"2023-10-26T06:45:07.822424+0000","last_clean":"2023-10-26T06:45:07.822424+0000","last_became_active":"2023-10-26T06:45:05.896893+0000","last_became_peered":"2023-10-26T06:45:05.896893+0000","last_unstale":"2023-10-26T06:45:07.822424+0000","last_undegraded":"2023-10-26T06:45:07.822424+0000","last_fullsized":"2023-10-26T06:45:07.822424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:14.291682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.906981+0000","last_change":"2023-10-26T06:45:05.905987+0000","last_active":"2023-10-26T06:45:05.906981+0000","last_peered":"2023-10-26T06:45:05.906981+0000","last_clean":"2023-10-26T06:45:05.906981+0000","last_became_active":"2023-10-26T06:45:05.904933+0000","last_became_peered":"2023-10-26T06:45:05.904933+0000","last_unstale":"2023-10-26T06:45:05.906981+0000","last_undegraded":"2023-10-26T06:45:05.906981+0000","last_fullsized":"2023-10-26T06:45:05.906981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:08:04.388627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885189+0000","last_change":"2023-10-26T06:45:05.912781+0000","last_active":"2023-10-26T06:45:06.885189+0000","last_peered":"2023-10-26T06:45:06.885189+0000","last_clean":"2023-10-26T06:45:06.885189+0000","last_became_active":"2023-10-26T06:45:05.912415+0000","last_became_peered":"2023-10-26T06:45:05.912415+0000","last_unstale":"2023-10-26T06:45:06.885189+0000","last_undegraded":"2023-10-26T06:45:06.885189+0000","last_fullsized":"2023-10-26T06:45:06.885189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:25.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822121+0000","last_change":"2023-10-26T06:45:05.909106+0000","last_active":"2023-10-26T06:45:07.822121+0000","last_peered":"2023-10-26T06:45:07.822121+0000","last_clean":"2023-10-26T06:45:07.822121+0000","last_became_active":"2023-10-26T06:45:05.908751+0000","last_became_peered":"2023-10-26T06:45:05.908751+0000","last_unstale":"2023-10-26T06:45:07.822121+0000","last_undegraded":"2023-10-26T06:45:07.822121+0000","last_fullsized":"2023-10-26T06:45:07.822121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:17.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599847+0000","last_change":"2023-10-26T06:45:05.898262+0000","last_active":"2023-10-26T06:45:08.599847+0000","last_peered":"2023-10-26T06:45:08.599847+0000","last_clean":"2023-10-26T06:45:08.599847+0000","last_became_active":"2023-10-26T06:45:05.897947+0000","last_became_peered":"2023-10-26T06:45:05.897947+0000","last_unstale":"2023-10-26T06:45:08.599847+0000","last_undegraded":"2023-10-26T06:45:08.599847+0000","last_fullsized":"2023-10-26T06:45:08.599847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:39:45.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":9635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416396,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":435803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]}]},{"osd":10,"up_from":72,"seq":309237645329,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1511424,"data_stored":1027977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.502}]},{"osd":5,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41699999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58399999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.747},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.623}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":445170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.435},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63}]},{"osd":2,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.749}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.77200000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":444861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.0609999999999999,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.749},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.628},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":5.0940000000000003,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.79700000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83499999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":7,"up_from":52,"seq":223338299422,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1490944,"data_stored":1026753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.45400000000000001,"15min":0.44600000000000001},"min":{"1min":0.34399999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.495},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.54500000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.873,"5min":1.006,"15min":1.006},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"last":0.442},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48699999999999999,"15min":0.47899999999999998},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.34599999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.433}]},{"osd":6,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.621,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.63600000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61499999999999999}]},{"osd":9,"last update":"Thu Oct 26 06:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57799999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.53800000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462947,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":444711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.44900000000000001,"15min":0.443},"min":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.432,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.54100000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67600000000000005,"15min":0.68100000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.81200000000000006,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.437},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.55800000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0089999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85899999999999999,"5min":1.002,"15min":1.002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Oct 26 06:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.58499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.71499999999999997,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80000000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Thu Oct 26 06:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.72099999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.86099999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":446200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.434,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.498},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73499999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Oct 26 06:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81399999999999995}]},{"osd":4,"last update":"Thu Oct 26 06:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.435,"15min":0.435},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.29499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:45:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47099999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69999999999999996}]},{"osd":8,"last update":"Thu Oct 26 06:44:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705727,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":897024,"data_stored":435893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 06:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.59999999999999998,"15min":0.58099999999999996},"min":{"1min":0.48499999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0049999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.46000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.434,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93400000000000005,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90800000000000003,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Thu Oct 26 06:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":2.198,"15min":2.198},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97599999999999998,"5min":1.48,"15min":1.48},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Oct 26 06:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":1.514,"15min":1.514},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93400000000000005,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88400000000000001,"5min":1.002,"15min":1.002},"last":0.623},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.68799999999999994,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.88}]},{"osd":9,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65700000000000003}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.1000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378678,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":445565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81699999999999995,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.39900000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Oct 26 06:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.46100000000000002},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.47699999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.78700000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.997,"15min":0.997},"last":1.3480000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.3919999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.68999999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.79300000000000004}]},{"osd":10,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":897024,"data_stored":435576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.39100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.83499999999999996,"5min":1.103,"15min":1.103},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.03,"5min":1.335,"15min":1.335},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.48499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.04,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Oct 26 06:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.50700000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91500000000000004,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.48299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90600000000000003,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Oct 26 06:44:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.51700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1220000000000001,"5min":1.274,"15min":1.274},"last":0.504},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.58499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.024,"5min":1.304,"15min":1.304},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.68400000000000005},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72999999999999998,"15min":0.69499999999999995},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Oct 26 06:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.44800000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Oct 26 06:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.083,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77100000000000002,"15min":0.77000000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0089999999999999,"5min":1.202,"15min":1.202},"last":1.02}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.0640000000000001}]},{"osd":10,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.95299999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888108,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":444610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.434}]},{"osd":1,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.37,"15min":0.34599999999999997},"min":{"1min":0.28999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"min":{"1min":0.253,"5min":0.245,"15min":0.245},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.443,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61799999999999999,"15min":0.59199999999999997},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42399999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.439,"15min":0.439},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.372},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.35399999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.437}]},{"osd":9,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.59099999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.505},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.44600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724585,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1035138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.47499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.32500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39000000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.56399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Oct 26 06:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56200000000000006}]},{"osd":8,"last update":"Thu Oct 26 06:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.627},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57499999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68300000000000005}]},{"osd":10,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-26T06:45:13.425 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T06:45:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:13 smithi135 ceph-mon[131403]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:13 smithi135 ceph-mon[131403]: pgmap v257: 97 pgs: 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 901 B/s rd, 2.1 KiB/s wr, 7 op/s 2023-10-26T06:45:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:13 smithi135 ceph-mon[131403]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:45:13.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:13 smithi135 ceph-mon[131403]: mgrmap e21: x(active, since 6m), standbys: z, y 2023-10-26T06:45:13.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:13 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2845103052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:45:13.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:13 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1234055047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:13.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1836167148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec100f80 msgr2=0x7fe1ec10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1836167148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec100f80 0x7fe1ec10d490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400b0a0 tx=0x7fe1e400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1836167148 shutdown_connections 2023-10-26T06:45:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1836167148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec100f80 0x7fe1ec10d490 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1836167148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec106660 0x7fe1ec106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1836167148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1ec1005c0 0x7fe1ec100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1836167148 >> 172.21.15.89:0/1836167148 conn(0x7fe1ec0fc070 msgr2=0x7fe1ec0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1836167148 shutdown_connections 2023-10-26T06:45:13.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.654+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1836167148 wait complete. 2023-10-26T06:45:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 Processor -- start 2023-10-26T06:45:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 -- start start 2023-10-26T06:45:13.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec1005c0 0x7fe1ec19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1ec100f80 0x7fe1ec19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec106660 0x7fe1ec196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec196db0 con 0x7fe1ec1005c0 2023-10-26T06:45:13.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec196f20 con 0x7fe1ec100f80 2023-10-26T06:45:13.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.655+0000 7fe1f0fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec1970e0 con 0x7fe1ec106660 2023-10-26T06:45:13.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec1005c0 0x7fe1ec19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec1005c0 0x7fe1ec19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60304/0 (socket says 172.21.15.89:60304) 2023-10-26T06:45:13.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 -- 172.21.15.89:0/1959411409 learned_addr learned my addr 172.21.15.89:0/1959411409 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:13.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1e3fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1ec100f80 0x7fe1ec19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ead9d700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec106660 0x7fe1ec196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 -- 172.21.15.89:0/1959411409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec106660 msgr2=0x7fe1ec196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec106660 0x7fe1ec196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 -- 172.21.15.89:0/1959411409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1ec100f80 msgr2=0x7fe1ec19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1ec100f80 0x7fe1ec19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 -- 172.21.15.89:0/1959411409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4009d00 con 0x7fe1ec1005c0 2023-10-26T06:45:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1ea59c700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec1005c0 0x7fe1ec19c620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00e970 tx=0x7fe1dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1e37fe700 1 -- 172.21.15.89:0/1959411409 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc009790 con 0x7fe1ec1005c0 2023-10-26T06:45:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1e37fe700 1 -- 172.21.15.89:0/1959411409 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1dc00eec0 con 0x7fe1ec1005c0 2023-10-26T06:45:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec1973c0 con 0x7fe1ec1005c0 2023-10-26T06:45:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.656+0000 7fe1e37fe700 1 -- 172.21.15.89:0/1959411409 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc003cf0 con 0x7fe1ec1005c0 2023-10-26T06:45:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.657+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec1a3610 con 0x7fe1ec1005c0 2023-10-26T06:45:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.657+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec101d00 con 0x7fe1ec1005c0 2023-10-26T06:45:13.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.659+0000 7fe1e37fe700 1 -- 172.21.15.89:0/1959411409 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe1dc00ccd0 con 0x7fe1ec1005c0 2023-10-26T06:45:13.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.661+0000 7fe1e37fe700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1d80a1560 0x7fe1d80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:13.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.661+0000 7fe1e37fe700 1 -- 172.21.15.89:0/1959411409 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fe1dc014070 con 0x7fe1ec1005c0 2023-10-26T06:45:13.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.661+0000 7fe1e3fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1d80a1560 0x7fe1d80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:13.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.661+0000 7fe1e3fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1d80a1560 0x7fe1d80a3a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec197e60 tx=0x7fe1d4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:13.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.664+0000 7fe1e37fe700 1 -- 172.21.15.89:0/1959411409 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1dc096660 con 0x7fe1ec1005c0 2023-10-26T06:45:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.868+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1ec110560 con 0x7fe1d80a1560 2023-10-26T06:45:13.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.878+0000 7fe1e37fe700 1 -- 172.21.15.89:0/1959411409 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335603 (secure 0 0 0) 0x7fe1ec110560 con 0x7fe1d80a1560 2023-10-26T06:45:13.885 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1d80a1560 msgr2=0x7fe1d80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1d80a1560 0x7fe1d80a3a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec197e60 tx=0x7fe1d4009260 comp rx=0 tx=0).stop 2023-10-26T06:45:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec1005c0 msgr2=0x7fe1ec19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec1005c0 0x7fe1ec19c620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00e970 tx=0x7fe1dc00ec80 comp rx=0 tx=0).stop 2023-10-26T06:45:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 shutdown_connections 2023-10-26T06:45:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1d80a1560 0x7fe1d80a3a20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1ec1005c0 0x7fe1ec19c620 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec106660 0x7fe1ec196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 --2- 172.21.15.89:0/1959411409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1ec100f80 0x7fe1ec19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 >> 172.21.15.89:0/1959411409 conn(0x7fe1ec0fc070 msgr2=0x7fe1ec0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 shutdown_connections 2023-10-26T06:45:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:13.883+0000 7fe1f0fff700 1 -- 172.21.15.89:0/1959411409 wait complete. 2023-10-26T06:45:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-26T06:45:13.904 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2023-10-26T06:45:13.714633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":627697,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":154,"num_read_kb":165,"num_write":238,"num_write_kb":2989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336708,"kb_used_data":12456,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605116,"statfs":{"total":1151940427776,"available":1151595638784,"internally_reserved":0,"allocated":12754944,"data_stored":7121254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37329,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":18,"num_read_kb":46,"num_write":57,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001691"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600110+0000","last_change":"2023-10-26T06:45:05.903170+0000","last_active":"2023-10-26T06:45:08.600110+0000","last_peered":"2023-10-26T06:45:08.600110+0000","last_clean":"2023-10-26T06:45:08.600110+0000","last_became_active":"2023-10-26T06:45:05.902574+0000","last_became_peered":"2023-10-26T06:45:05.902574+0000","last_unstale":"2023-10-26T06:45:08.600110+0000","last_undegraded":"2023-10-26T06:45:08.600110+0000","last_fullsized":"2023-10-26T06:45:08.600110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:33.065828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505541+0000","last_change":"2023-10-26T06:45:05.911249+0000","last_active":"2023-10-26T06:45:07.505541+0000","last_peered":"2023-10-26T06:45:07.505541+0000","last_clean":"2023-10-26T06:45:07.505541+0000","last_became_active":"2023-10-26T06:45:05.910859+0000","last_became_peered":"2023-10-26T06:45:05.910859+0000","last_unstale":"2023-10-26T06:45:07.505541+0000","last_undegraded":"2023-10-26T06:45:07.505541+0000","last_fullsized":"2023-10-26T06:45:07.505541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:07:13.362460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505391+0000","last_change":"2023-10-26T06:45:05.909219+0000","last_active":"2023-10-26T06:45:07.505391+0000","last_peered":"2023-10-26T06:45:07.505391+0000","last_clean":"2023-10-26T06:45:07.505391+0000","last_became_active":"2023-10-26T06:45:05.906721+0000","last_became_peered":"2023-10-26T06:45:05.906721+0000","last_unstale":"2023-10-26T06:45:07.505391+0000","last_undegraded":"2023-10-26T06:45:07.505391+0000","last_fullsized":"2023-10-26T06:45:07.505391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:41.461357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823998+0000","last_change":"2023-10-26T06:45:05.897514+0000","last_active":"2023-10-26T06:45:07.823998+0000","last_peered":"2023-10-26T06:45:07.823998+0000","last_clean":"2023-10-26T06:45:07.823998+0000","last_became_active":"2023-10-26T06:45:05.897201+0000","last_became_peered":"2023-10-26T06:45:05.897201+0000","last_unstale":"2023-10-26T06:45:07.823998+0000","last_undegraded":"2023-10-26T06:45:07.823998+0000","last_fullsized":"2023-10-26T06:45:07.823998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:39.508654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689213+0000","last_change":"2023-10-26T06:45:05.900475+0000","last_active":"2023-10-26T06:45:07.689213+0000","last_peered":"2023-10-26T06:45:07.689213+0000","last_clean":"2023-10-26T06:45:07.689213+0000","last_became_active":"2023-10-26T06:45:05.900313+0000","last_became_peered":"2023-10-26T06:45:05.900313+0000","last_unstale":"2023-10-26T06:45:07.689213+0000","last_undegraded":"2023-10-26T06:45:07.689213+0000","last_fullsized":"2023-10-26T06:45:07.689213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:51.438741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823465+0000","last_change":"2023-10-26T06:45:05.909040+0000","last_active":"2023-10-26T06:45:07.823465+0000","last_peered":"2023-10-26T06:45:07.823465+0000","last_clean":"2023-10-26T06:45:07.823465+0000","last_became_active":"2023-10-26T06:45:05.908696+0000","last_became_peered":"2023-10-26T06:45:05.908696+0000","last_unstale":"2023-10-26T06:45:07.823465+0000","last_undegraded":"2023-10-26T06:45:07.823465+0000","last_fullsized":"2023-10-26T06:45:07.823465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:00.147712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823545+0000","last_change":"2023-10-26T06:45:05.898356+0000","last_active":"2023-10-26T06:45:07.823545+0000","last_peered":"2023-10-26T06:45:07.823545+0000","last_clean":"2023-10-26T06:45:07.823545+0000","last_became_active":"2023-10-26T06:45:05.898008+0000","last_became_peered":"2023-10-26T06:45:05.898008+0000","last_unstale":"2023-10-26T06:45:07.823545+0000","last_undegraded":"2023-10-26T06:45:07.823545+0000","last_fullsized":"2023-10-26T06:45:07.823545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:23.209987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137846+0000","last_change":"2023-10-26T06:45:05.911301+0000","last_active":"2023-10-26T06:45:07.137846+0000","last_peered":"2023-10-26T06:45:07.137846+0000","last_clean":"2023-10-26T06:45:07.137846+0000","last_became_active":"2023-10-26T06:45:05.910667+0000","last_became_peered":"2023-10-26T06:45:05.910667+0000","last_unstale":"2023-10-26T06:45:07.137846+0000","last_undegraded":"2023-10-26T06:45:07.137846+0000","last_fullsized":"2023-10-26T06:45:07.137846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:03:36.856248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885616+0000","last_change":"2023-10-26T06:45:05.904456+0000","last_active":"2023-10-26T06:45:06.885616+0000","last_peered":"2023-10-26T06:45:06.885616+0000","last_clean":"2023-10-26T06:45:06.885616+0000","last_became_active":"2023-10-26T06:45:05.904165+0000","last_became_peered":"2023-10-26T06:45:05.904165+0000","last_unstale":"2023-10-26T06:45:06.885616+0000","last_undegraded":"2023-10-26T06:45:06.885616+0000","last_fullsized":"2023-10-26T06:45:06.885616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:51:01.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824115+0000","last_change":"2023-10-26T06:45:05.909544+0000","last_active":"2023-10-26T06:45:07.824115+0000","last_peered":"2023-10-26T06:45:07.824115+0000","last_clean":"2023-10-26T06:45:07.824115+0000","last_became_active":"2023-10-26T06:45:05.909103+0000","last_became_peered":"2023-10-26T06:45:05.909103+0000","last_unstale":"2023-10-26T06:45:07.824115+0000","last_undegraded":"2023-10-26T06:45:07.824115+0000","last_fullsized":"2023-10-26T06:45:07.824115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:18.069186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823939+0000","last_change":"2023-10-26T06:45:05.906549+0000","last_active":"2023-10-26T06:45:07.823939+0000","last_peered":"2023-10-26T06:45:07.823939+0000","last_clean":"2023-10-26T06:45:07.823939+0000","last_became_active":"2023-10-26T06:45:05.905883+0000","last_became_peered":"2023-10-26T06:45:05.905883+0000","last_unstale":"2023-10-26T06:45:07.823939+0000","last_undegraded":"2023-10-26T06:45:07.823939+0000","last_fullsized":"2023-10-26T06:45:07.823939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:54.771031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823855+0000","last_change":"2023-10-26T06:45:05.909460+0000","last_active":"2023-10-26T06:45:07.823855+0000","last_peered":"2023-10-26T06:45:07.823855+0000","last_clean":"2023-10-26T06:45:07.823855+0000","last_became_active":"2023-10-26T06:45:05.909007+0000","last_became_peered":"2023-10-26T06:45:05.909007+0000","last_unstale":"2023-10-26T06:45:07.823855+0000","last_undegraded":"2023-10-26T06:45:07.823855+0000","last_fullsized":"2023-10-26T06:45:07.823855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:45:55.817606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822132+0000","last_change":"2023-10-26T06:45:05.908305+0000","last_active":"2023-10-26T06:45:07.822132+0000","last_peered":"2023-10-26T06:45:07.822132+0000","last_clean":"2023-10-26T06:45:07.822132+0000","last_became_active":"2023-10-26T06:45:05.907931+0000","last_became_peered":"2023-10-26T06:45:05.907931+0000","last_unstale":"2023-10-26T06:45:07.822132+0000","last_undegraded":"2023-10-26T06:45:07.822132+0000","last_fullsized":"2023-10-26T06:45:07.822132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:00:43.605324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505175+0000","last_change":"2023-10-26T06:45:05.909737+0000","last_active":"2023-10-26T06:45:07.505175+0000","last_peered":"2023-10-26T06:45:07.505175+0000","last_clean":"2023-10-26T06:45:07.505175+0000","last_became_active":"2023-10-26T06:45:05.909445+0000","last_became_peered":"2023-10-26T06:45:05.909445+0000","last_unstale":"2023-10-26T06:45:07.505175+0000","last_undegraded":"2023-10-26T06:45:07.505175+0000","last_fullsized":"2023-10-26T06:45:07.505175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:38.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.504905+0000","last_change":"2023-10-26T06:45:05.907722+0000","last_active":"2023-10-26T06:45:07.504905+0000","last_peered":"2023-10-26T06:45:07.504905+0000","last_clean":"2023-10-26T06:45:07.504905+0000","last_became_active":"2023-10-26T06:45:05.905622+0000","last_became_peered":"2023-10-26T06:45:05.905622+0000","last_unstale":"2023-10-26T06:45:07.504905+0000","last_undegraded":"2023-10-26T06:45:07.504905+0000","last_fullsized":"2023-10-26T06:45:07.504905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:30:01.048032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885013+0000","last_change":"2023-10-26T06:45:05.907801+0000","last_active":"2023-10-26T06:45:06.885013+0000","last_peered":"2023-10-26T06:45:06.885013+0000","last_clean":"2023-10-26T06:45:06.885013+0000","last_became_active":"2023-10-26T06:45:05.907485+0000","last_became_peered":"2023-10-26T06:45:05.907485+0000","last_unstale":"2023-10-26T06:45:06.885013+0000","last_undegraded":"2023-10-26T06:45:06.885013+0000","last_fullsized":"2023-10-26T06:45:06.885013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:16.850992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600141+0000","last_change":"2023-10-26T06:45:05.902859+0000","last_active":"2023-10-26T06:45:08.600141+0000","last_peered":"2023-10-26T06:45:08.600141+0000","last_clean":"2023-10-26T06:45:08.600141+0000","last_became_active":"2023-10-26T06:45:05.902218+0000","last_became_peered":"2023-10-26T06:45:05.902218+0000","last_unstale":"2023-10-26T06:45:08.600141+0000","last_undegraded":"2023-10-26T06:45:08.600141+0000","last_fullsized":"2023-10-26T06:45:08.600141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:47.717227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884920+0000","last_change":"2023-10-26T06:45:05.910445+0000","last_active":"2023-10-26T06:45:06.884920+0000","last_peered":"2023-10-26T06:45:06.884920+0000","last_clean":"2023-10-26T06:45:06.884920+0000","last_became_active":"2023-10-26T06:45:05.910043+0000","last_became_peered":"2023-10-26T06:45:05.910043+0000","last_unstale":"2023-10-26T06:45:06.884920+0000","last_undegraded":"2023-10-26T06:45:06.884920+0000","last_fullsized":"2023-10-26T06:45:06.884920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:33.751134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884907+0000","last_change":"2023-10-26T06:45:05.905340+0000","last_active":"2023-10-26T06:45:06.884907+0000","last_peered":"2023-10-26T06:45:06.884907+0000","last_clean":"2023-10-26T06:45:06.884907+0000","last_became_active":"2023-10-26T06:45:05.905030+0000","last_became_peered":"2023-10-26T06:45:05.905030+0000","last_unstale":"2023-10-26T06:45:06.884907+0000","last_undegraded":"2023-10-26T06:45:06.884907+0000","last_fullsized":"2023-10-26T06:45:06.884907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.303210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884956+0000","last_change":"2023-10-26T06:45:04.919128+0000","last_active":"2023-10-26T06:45:06.884956+0000","last_peered":"2023-10-26T06:45:06.884956+0000","last_clean":"2023-10-26T06:45:06.884956+0000","last_became_active":"2023-10-26T06:45:04.913034+0000","last_became_peered":"2023-10-26T06:45:04.913034+0000","last_unstale":"2023-10-26T06:45:06.884956+0000","last_undegraded":"2023-10-26T06:45:06.884956+0000","last_fullsized":"2023-10-26T06:45:06.884956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:50.835037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138323+0000","last_change":"2023-10-26T06:45:05.905315+0000","last_active":"2023-10-26T06:45:07.138323+0000","last_peered":"2023-10-26T06:45:07.138323+0000","last_clean":"2023-10-26T06:45:07.138323+0000","last_became_active":"2023-10-26T06:45:05.904260+0000","last_became_peered":"2023-10-26T06:45:05.904260+0000","last_unstale":"2023-10-26T06:45:07.138323+0000","last_undegraded":"2023-10-26T06:45:07.138323+0000","last_fullsized":"2023-10-26T06:45:07.138323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:02.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.080681+0000","last_change":"2023-10-26T06:45:04.920440+0000","last_active":"2023-10-26T06:45:07.080681+0000","last_peered":"2023-10-26T06:45:07.080681+0000","last_clean":"2023-10-26T06:45:07.080681+0000","last_became_active":"2023-10-26T06:45:04.920129+0000","last_became_peered":"2023-10-26T06:45:04.920129+0000","last_unstale":"2023-10-26T06:45:07.080681+0000","last_undegraded":"2023-10-26T06:45:07.080681+0000","last_fullsized":"2023-10-26T06:45:07.080681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:43.176830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505711+0000","last_change":"2023-10-26T06:45:05.902898+0000","last_active":"2023-10-26T06:45:07.505711+0000","last_peered":"2023-10-26T06:45:07.505711+0000","last_clean":"2023-10-26T06:45:07.505711+0000","last_became_active":"2023-10-26T06:45:05.902161+0000","last_became_peered":"2023-10-26T06:45:05.902161+0000","last_unstale":"2023-10-26T06:45:07.505711+0000","last_undegraded":"2023-10-26T06:45:07.505711+0000","last_fullsized":"2023-10-26T06:45:07.505711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:27.872225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063544+0000","last_change":"2023-10-26T06:45:04.911835+0000","last_active":"2023-10-26T06:45:06.063544+0000","last_peered":"2023-10-26T06:45:06.063544+0000","last_clean":"2023-10-26T06:45:06.063544+0000","last_became_active":"2023-10-26T06:45:04.911505+0000","last_became_peered":"2023-10-26T06:45:04.911505+0000","last_unstale":"2023-10-26T06:45:06.063544+0000","last_undegraded":"2023-10-26T06:45:06.063544+0000","last_fullsized":"2023-10-26T06:45:06.063544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:01:18.971381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885164+0000","last_change":"2023-10-26T06:45:05.909088+0000","last_active":"2023-10-26T06:45:06.885164+0000","last_peered":"2023-10-26T06:45:06.885164+0000","last_clean":"2023-10-26T06:45:06.885164+0000","last_became_active":"2023-10-26T06:45:05.908528+0000","last_became_peered":"2023-10-26T06:45:05.908528+0000","last_unstale":"2023-10-26T06:45:06.885164+0000","last_undegraded":"2023-10-26T06:45:06.885164+0000","last_fullsized":"2023-10-26T06:45:06.885164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:07.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063027+0000","last_change":"2023-10-26T06:45:04.913604+0000","last_active":"2023-10-26T06:45:06.063027+0000","last_peered":"2023-10-26T06:45:06.063027+0000","last_clean":"2023-10-26T06:45:06.063027+0000","last_became_active":"2023-10-26T06:45:04.913318+0000","last_became_peered":"2023-10-26T06:45:04.913318+0000","last_unstale":"2023-10-26T06:45:06.063027+0000","last_undegraded":"2023-10-26T06:45:06.063027+0000","last_fullsized":"2023-10-26T06:45:06.063027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:19:51.144044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822837+0000","last_change":"2023-10-26T06:45:05.898795+0000","last_active":"2023-10-26T06:45:07.822837+0000","last_peered":"2023-10-26T06:45:07.822837+0000","last_clean":"2023-10-26T06:45:07.822837+0000","last_became_active":"2023-10-26T06:45:05.898489+0000","last_became_peered":"2023-10-26T06:45:05.898489+0000","last_unstale":"2023-10-26T06:45:07.822837+0000","last_undegraded":"2023-10-26T06:45:07.822837+0000","last_fullsized":"2023-10-26T06:45:07.822837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:55.343736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062447+0000","last_change":"2023-10-26T06:45:04.908881+0000","last_active":"2023-10-26T06:45:06.062447+0000","last_peered":"2023-10-26T06:45:06.062447+0000","last_clean":"2023-10-26T06:45:06.062447+0000","last_became_active":"2023-10-26T06:45:04.908515+0000","last_became_peered":"2023-10-26T06:45:04.908515+0000","last_unstale":"2023-10-26T06:45:06.062447+0000","last_undegraded":"2023-10-26T06:45:06.062447+0000","last_fullsized":"2023-10-26T06:45:06.062447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:39.200294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689203+0000","last_change":"2023-10-26T06:45:05.901281+0000","last_active":"2023-10-26T06:45:07.689203+0000","last_peered":"2023-10-26T06:45:07.689203+0000","last_clean":"2023-10-26T06:45:07.689203+0000","last_became_active":"2023-10-26T06:45:05.900995+0000","last_became_peered":"2023-10-26T06:45:05.900995+0000","last_unstale":"2023-10-26T06:45:07.689203+0000","last_undegraded":"2023-10-26T06:45:07.689203+0000","last_fullsized":"2023-10-26T06:45:07.689203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:44:34.977571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.285852+0000","last_change":"2023-10-26T06:45:04.918480+0000","last_active":"2023-10-26T06:45:07.285852+0000","last_peered":"2023-10-26T06:45:07.285852+0000","last_clean":"2023-10-26T06:45:07.285852+0000","last_became_active":"2023-10-26T06:45:04.918185+0000","last_became_peered":"2023-10-26T06:45:04.918185+0000","last_unstale":"2023-10-26T06:45:07.285852+0000","last_undegraded":"2023-10-26T06:45:07.285852+0000","last_fullsized":"2023-10-26T06:45:07.285852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:07:58.263116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822711+0000","last_change":"2023-10-26T06:45:05.901825+0000","last_active":"2023-10-26T06:45:07.822711+0000","last_peered":"2023-10-26T06:45:07.822711+0000","last_clean":"2023-10-26T06:45:07.822711+0000","last_became_active":"2023-10-26T06:45:05.901509+0000","last_became_peered":"2023-10-26T06:45:05.901509+0000","last_unstale":"2023-10-26T06:45:07.822711+0000","last_undegraded":"2023-10-26T06:45:07.822711+0000","last_fullsized":"2023-10-26T06:45:07.822711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:16.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534260+0000","last_change":"2023-10-26T06:45:04.919813+0000","last_active":"2023-10-26T06:45:06.534260+0000","last_peered":"2023-10-26T06:45:06.534260+0000","last_clean":"2023-10-26T06:45:06.534260+0000","last_became_active":"2023-10-26T06:45:04.918016+0000","last_became_peered":"2023-10-26T06:45:04.918016+0000","last_unstale":"2023-10-26T06:45:06.534260+0000","last_undegraded":"2023-10-26T06:45:06.534260+0000","last_fullsized":"2023-10-26T06:45:06.534260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:32.336126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137929+0000","last_change":"2023-10-26T06:45:05.902182+0000","last_active":"2023-10-26T06:45:07.137929+0000","last_peered":"2023-10-26T06:45:07.137929+0000","last_clean":"2023-10-26T06:45:07.137929+0000","last_became_active":"2023-10-26T06:45:05.901428+0000","last_became_peered":"2023-10-26T06:45:05.901428+0000","last_unstale":"2023-10-26T06:45:07.137929+0000","last_undegraded":"2023-10-26T06:45:07.137929+0000","last_fullsized":"2023-10-26T06:45:07.137929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:12.722532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.890140+0000","last_change":"2023-10-26T06:45:04.909713+0000","last_active":"2023-10-26T06:45:05.890140+0000","last_peered":"2023-10-26T06:45:05.890140+0000","last_clean":"2023-10-26T06:45:05.890140+0000","last_became_active":"2023-10-26T06:45:04.909412+0000","last_became_peered":"2023-10-26T06:45:04.909412+0000","last_unstale":"2023-10-26T06:45:05.890140+0000","last_undegraded":"2023-10-26T06:45:05.890140+0000","last_fullsized":"2023-10-26T06:45:05.890140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:23.901745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884924+0000","last_change":"2023-10-26T06:45:05.908348+0000","last_active":"2023-10-26T06:45:06.884924+0000","last_peered":"2023-10-26T06:45:06.884924+0000","last_clean":"2023-10-26T06:45:06.884924+0000","last_became_active":"2023-10-26T06:45:05.908022+0000","last_became_peered":"2023-10-26T06:45:05.908022+0000","last_unstale":"2023-10-26T06:45:06.884924+0000","last_undegraded":"2023-10-26T06:45:06.884924+0000","last_fullsized":"2023-10-26T06:45:06.884924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:22:16.027732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.285938+0000","last_change":"2023-10-26T06:45:04.915491+0000","last_active":"2023-10-26T06:45:07.285938+0000","last_peered":"2023-10-26T06:45:07.285938+0000","last_clean":"2023-10-26T06:45:07.285938+0000","last_became_active":"2023-10-26T06:45:04.915191+0000","last_became_peered":"2023-10-26T06:45:04.915191+0000","last_unstale":"2023-10-26T06:45:07.285938+0000","last_undegraded":"2023-10-26T06:45:07.285938+0000","last_fullsized":"2023-10-26T06:45:07.285938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:57.758903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885545+0000","last_change":"2023-10-26T06:45:05.912676+0000","last_active":"2023-10-26T06:45:06.885545+0000","last_peered":"2023-10-26T06:45:06.885545+0000","last_clean":"2023-10-26T06:45:06.885545+0000","last_became_active":"2023-10-26T06:45:05.912372+0000","last_became_peered":"2023-10-26T06:45:05.912372+0000","last_unstale":"2023-10-26T06:45:06.885545+0000","last_undegraded":"2023-10-26T06:45:06.885545+0000","last_fullsized":"2023-10-26T06:45:06.885545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:33.299166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502215+0000","last_change":"2023-10-26T06:45:04.912123+0000","last_active":"2023-10-26T06:45:07.502215+0000","last_peered":"2023-10-26T06:45:07.502215+0000","last_clean":"2023-10-26T06:45:07.502215+0000","last_became_active":"2023-10-26T06:45:04.911893+0000","last_became_peered":"2023-10-26T06:45:04.911893+0000","last_unstale":"2023-10-26T06:45:07.502215+0000","last_undegraded":"2023-10-26T06:45:07.502215+0000","last_fullsized":"2023-10-26T06:45:07.502215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:32.907496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505559+0000","last_change":"2023-10-26T06:45:05.893434+0000","last_active":"2023-10-26T06:45:07.505559+0000","last_peered":"2023-10-26T06:45:07.505559+0000","last_clean":"2023-10-26T06:45:07.505559+0000","last_became_active":"2023-10-26T06:45:05.893100+0000","last_became_peered":"2023-10-26T06:45:05.893100+0000","last_unstale":"2023-10-26T06:45:07.505559+0000","last_undegraded":"2023-10-26T06:45:07.505559+0000","last_fullsized":"2023-10-26T06:45:07.505559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:22:48.317428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062642+0000","last_change":"2023-10-26T06:45:04.908910+0000","last_active":"2023-10-26T06:45:06.062642+0000","last_peered":"2023-10-26T06:45:06.062642+0000","last_clean":"2023-10-26T06:45:06.062642+0000","last_became_active":"2023-10-26T06:45:04.908592+0000","last_became_peered":"2023-10-26T06:45:04.908592+0000","last_unstale":"2023-10-26T06:45:06.062642+0000","last_undegraded":"2023-10-26T06:45:06.062642+0000","last_fullsized":"2023-10-26T06:45:06.062642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:40.266013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885272+0000","last_change":"2023-10-26T06:45:05.908887+0000","last_active":"2023-10-26T06:45:06.885272+0000","last_peered":"2023-10-26T06:45:06.885272+0000","last_clean":"2023-10-26T06:45:06.885272+0000","last_became_active":"2023-10-26T06:45:05.908163+0000","last_became_peered":"2023-10-26T06:45:05.908163+0000","last_unstale":"2023-10-26T06:45:06.885272+0000","last_undegraded":"2023-10-26T06:45:06.885272+0000","last_fullsized":"2023-10-26T06:45:06.885272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:09.339374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'16","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:12.087059+0000","last_change":"2023-10-26T06:45:04.905691+0000","last_active":"2023-10-26T06:45:12.087059+0000","last_peered":"2023-10-26T06:45:12.087059+0000","last_clean":"2023-10-26T06:45:12.087059+0000","last_became_active":"2023-10-26T06:45:04.905378+0000","last_became_peered":"2023-10-26T06:45:04.905378+0000","last_unstale":"2023-10-26T06:45:12.087059+0000","last_undegraded":"2023-10-26T06:45:12.087059+0000","last_fullsized":"2023-10-26T06:45:12.087059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:24:21.498141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36285,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":40,"num_write":16,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689388+0000","last_change":"2023-10-26T06:45:05.897366+0000","last_active":"2023-10-26T06:45:07.689388+0000","last_peered":"2023-10-26T06:45:07.689388+0000","last_clean":"2023-10-26T06:45:07.689388+0000","last_became_active":"2023-10-26T06:45:05.897091+0000","last_became_peered":"2023-10-26T06:45:05.897091+0000","last_unstale":"2023-10-26T06:45:07.689388+0000","last_undegraded":"2023-10-26T06:45:07.689388+0000","last_fullsized":"2023-10-26T06:45:07.689388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:11.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884895+0000","last_change":"2023-10-26T06:45:04.914949+0000","last_active":"2023-10-26T06:45:06.884895+0000","last_peered":"2023-10-26T06:45:06.884895+0000","last_clean":"2023-10-26T06:45:06.884895+0000","last_became_active":"2023-10-26T06:45:04.914610+0000","last_became_peered":"2023-10-26T06:45:04.914610+0000","last_unstale":"2023-10-26T06:45:06.884895+0000","last_undegraded":"2023-10-26T06:45:06.884895+0000","last_fullsized":"2023-10-26T06:45:06.884895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:16.683352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822001+0000","last_change":"2023-10-26T06:45:05.911067+0000","last_active":"2023-10-26T06:45:07.822001+0000","last_peered":"2023-10-26T06:45:07.822001+0000","last_clean":"2023-10-26T06:45:07.822001+0000","last_became_active":"2023-10-26T06:45:05.908460+0000","last_became_peered":"2023-10-26T06:45:05.908460+0000","last_unstale":"2023-10-26T06:45:07.822001+0000","last_undegraded":"2023-10-26T06:45:07.822001+0000","last_fullsized":"2023-10-26T06:45:07.822001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:17.461781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.895349+0000","last_change":"2023-10-26T06:45:04.919947+0000","last_active":"2023-10-26T06:45:05.895349+0000","last_peered":"2023-10-26T06:45:05.895349+0000","last_clean":"2023-10-26T06:45:05.895349+0000","last_became_active":"2023-10-26T06:45:04.919535+0000","last_became_peered":"2023-10-26T06:45:04.919535+0000","last_unstale":"2023-10-26T06:45:05.895349+0000","last_undegraded":"2023-10-26T06:45:05.895349+0000","last_fullsized":"2023-10-26T06:45:05.895349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:46:09.201139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823839+0000","last_change":"2023-10-26T06:45:05.906381+0000","last_active":"2023-10-26T06:45:07.823839+0000","last_peered":"2023-10-26T06:45:07.823839+0000","last_clean":"2023-10-26T06:45:07.823839+0000","last_became_active":"2023-10-26T06:45:05.904487+0000","last_became_peered":"2023-10-26T06:45:05.904487+0000","last_unstale":"2023-10-26T06:45:07.823839+0000","last_undegraded":"2023-10-26T06:45:07.823839+0000","last_fullsized":"2023-10-26T06:45:07.823839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:00.164936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887797+0000","last_change":"2023-10-26T06:45:04.908779+0000","last_active":"2023-10-26T06:45:05.887797+0000","last_peered":"2023-10-26T06:45:05.887797+0000","last_clean":"2023-10-26T06:45:05.887797+0000","last_became_active":"2023-10-26T06:45:04.908490+0000","last_became_peered":"2023-10-26T06:45:04.908490+0000","last_unstale":"2023-10-26T06:45:05.887797+0000","last_undegraded":"2023-10-26T06:45:05.887797+0000","last_fullsized":"2023-10-26T06:45:05.887797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:12:13.199863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823130+0000","last_change":"2023-10-26T06:45:05.909408+0000","last_active":"2023-10-26T06:45:07.823130+0000","last_peered":"2023-10-26T06:45:07.823130+0000","last_clean":"2023-10-26T06:45:07.823130+0000","last_became_active":"2023-10-26T06:45:05.909107+0000","last_became_peered":"2023-10-26T06:45:05.909107+0000","last_unstale":"2023-10-26T06:45:07.823130+0000","last_undegraded":"2023-10-26T06:45:07.823130+0000","last_fullsized":"2023-10-26T06:45:07.823130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:05.887472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063862+0000","last_change":"2023-10-26T06:45:04.918194+0000","last_active":"2023-10-26T06:45:06.063862+0000","last_peered":"2023-10-26T06:45:06.063862+0000","last_clean":"2023-10-26T06:45:06.063862+0000","last_became_active":"2023-10-26T06:45:04.917850+0000","last_became_peered":"2023-10-26T06:45:04.917850+0000","last_unstale":"2023-10-26T06:45:06.063862+0000","last_undegraded":"2023-10-26T06:45:06.063862+0000","last_fullsized":"2023-10-26T06:45:06.063862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:25.743939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822581+0000","last_change":"2023-10-26T06:45:05.902156+0000","last_active":"2023-10-26T06:45:07.822581+0000","last_peered":"2023-10-26T06:45:07.822581+0000","last_clean":"2023-10-26T06:45:07.822581+0000","last_became_active":"2023-10-26T06:45:05.901865+0000","last_became_peered":"2023-10-26T06:45:05.901865+0000","last_unstale":"2023-10-26T06:45:07.822581+0000","last_undegraded":"2023-10-26T06:45:07.822581+0000","last_fullsized":"2023-10-26T06:45:07.822581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:06:08.167280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689452+0000","last_change":"2023-10-26T06:45:04.913071+0000","last_active":"2023-10-26T06:45:07.689452+0000","last_peered":"2023-10-26T06:45:07.689452+0000","last_clean":"2023-10-26T06:45:07.689452+0000","last_became_active":"2023-10-26T06:45:04.912736+0000","last_became_peered":"2023-10-26T06:45:04.912736+0000","last_unstale":"2023-10-26T06:45:07.689452+0000","last_undegraded":"2023-10-26T06:45:07.689452+0000","last_fullsized":"2023-10-26T06:45:07.689452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:59.690355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600012+0000","last_change":"2023-10-26T06:45:05.891949+0000","last_active":"2023-10-26T06:45:08.600012+0000","last_peered":"2023-10-26T06:45:08.600012+0000","last_clean":"2023-10-26T06:45:08.600012+0000","last_became_active":"2023-10-26T06:45:05.891583+0000","last_became_peered":"2023-10-26T06:45:05.891583+0000","last_unstale":"2023-10-26T06:45:08.600012+0000","last_undegraded":"2023-10-26T06:45:08.600012+0000","last_fullsized":"2023-10-26T06:45:08.600012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:54:18.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.883535+0000","last_change":"2023-10-26T06:45:04.913969+0000","last_active":"2023-10-26T06:45:05.883535+0000","last_peered":"2023-10-26T06:45:05.883535+0000","last_clean":"2023-10-26T06:45:05.883535+0000","last_became_active":"2023-10-26T06:45:04.913533+0000","last_became_peered":"2023-10-26T06:45:04.913533+0000","last_unstale":"2023-10-26T06:45:05.883535+0000","last_undegraded":"2023-10-26T06:45:05.883535+0000","last_fullsized":"2023-10-26T06:45:05.883535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:47:38.755102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824112+0000","last_change":"2023-10-26T06:45:05.904832+0000","last_active":"2023-10-26T06:45:07.824112+0000","last_peered":"2023-10-26T06:45:07.824112+0000","last_clean":"2023-10-26T06:45:07.824112+0000","last_became_active":"2023-10-26T06:45:05.904527+0000","last_became_peered":"2023-10-26T06:45:05.904527+0000","last_unstale":"2023-10-26T06:45:07.824112+0000","last_undegraded":"2023-10-26T06:45:07.824112+0000","last_fullsized":"2023-10-26T06:45:07.824112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:30.546198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502122+0000","last_change":"2023-10-26T06:45:04.922832+0000","last_active":"2023-10-26T06:45:07.502122+0000","last_peered":"2023-10-26T06:45:07.502122+0000","last_clean":"2023-10-26T06:45:07.502122+0000","last_became_active":"2023-10-26T06:45:04.922349+0000","last_became_peered":"2023-10-26T06:45:04.922349+0000","last_unstale":"2023-10-26T06:45:07.502122+0000","last_undegraded":"2023-10-26T06:45:07.502122+0000","last_fullsized":"2023-10-26T06:45:07.502122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:37.839093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823836+0000","last_change":"2023-10-26T06:45:05.909946+0000","last_active":"2023-10-26T06:45:07.823836+0000","last_peered":"2023-10-26T06:45:07.823836+0000","last_clean":"2023-10-26T06:45:07.823836+0000","last_became_active":"2023-10-26T06:45:05.909650+0000","last_became_peered":"2023-10-26T06:45:05.909650+0000","last_unstale":"2023-10-26T06:45:07.823836+0000","last_undegraded":"2023-10-26T06:45:07.823836+0000","last_fullsized":"2023-10-26T06:45:07.823836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:41.813928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822476+0000","last_change":"2023-10-26T06:45:05.903400+0000","last_active":"2023-10-26T06:45:07.822476+0000","last_peered":"2023-10-26T06:45:07.822476+0000","last_clean":"2023-10-26T06:45:07.822476+0000","last_became_active":"2023-10-26T06:45:05.903101+0000","last_became_peered":"2023-10-26T06:45:05.903101+0000","last_unstale":"2023-10-26T06:45:07.822476+0000","last_undegraded":"2023-10-26T06:45:07.822476+0000","last_fullsized":"2023-10-26T06:45:07.822476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534251+0000","last_change":"2023-10-26T06:45:04.923960+0000","last_active":"2023-10-26T06:45:06.534251+0000","last_peered":"2023-10-26T06:45:06.534251+0000","last_clean":"2023-10-26T06:45:06.534251+0000","last_became_active":"2023-10-26T06:45:04.923613+0000","last_became_peered":"2023-10-26T06:45:04.923613+0000","last_unstale":"2023-10-26T06:45:06.534251+0000","last_undegraded":"2023-10-26T06:45:06.534251+0000","last_fullsized":"2023-10-26T06:45:06.534251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:53.342463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138101+0000","last_change":"2023-10-26T06:45:05.903966+0000","last_active":"2023-10-26T06:45:07.138101+0000","last_peered":"2023-10-26T06:45:07.138101+0000","last_clean":"2023-10-26T06:45:07.138101+0000","last_became_active":"2023-10-26T06:45:05.903583+0000","last_became_peered":"2023-10-26T06:45:05.903583+0000","last_unstale":"2023-10-26T06:45:07.138101+0000","last_undegraded":"2023-10-26T06:45:07.138101+0000","last_fullsized":"2023-10-26T06:45:07.138101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:18.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887855+0000","last_change":"2023-10-26T06:45:04.907838+0000","last_active":"2023-10-26T06:45:05.887855+0000","last_peered":"2023-10-26T06:45:05.887855+0000","last_clean":"2023-10-26T06:45:05.887855+0000","last_became_active":"2023-10-26T06:45:04.907508+0000","last_became_peered":"2023-10-26T06:45:04.907508+0000","last_unstale":"2023-10-26T06:45:05.887855+0000","last_undegraded":"2023-10-26T06:45:05.887855+0000","last_fullsized":"2023-10-26T06:45:05.887855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:34.852243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":319,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.892156+0000","last_change":"2023-10-26T06:43:59.765466+0000","last_active":"2023-10-26T06:45:05.892156+0000","last_peered":"2023-10-26T06:45:05.892156+0000","last_clean":"2023-10-26T06:45:05.892156+0000","last_became_active":"2023-10-26T06:43:59.765005+0000","last_became_peered":"2023-10-26T06:43:59.765005+0000","last_unstale":"2023-10-26T06:45:05.892156+0000","last_undegraded":"2023-10-26T06:45:05.892156+0000","last_fullsized":"2023-10-26T06:45:05.892156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_clean_scrub_stamp":"2023-10-26T06:40:50.556448+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:18.688941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.821877+0000","last_change":"2023-10-26T06:45:05.906909+0000","last_active":"2023-10-26T06:45:07.821877+0000","last_peered":"2023-10-26T06:45:07.821877+0000","last_clean":"2023-10-26T06:45:07.821877+0000","last_became_active":"2023-10-26T06:45:05.906531+0000","last_became_peered":"2023-10-26T06:45:05.906531+0000","last_unstale":"2023-10-26T06:45:07.821877+0000","last_undegraded":"2023-10-26T06:45:07.821877+0000","last_fullsized":"2023-10-26T06:45:07.821877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502092+0000","last_change":"2023-10-26T06:45:04.911635+0000","last_active":"2023-10-26T06:45:07.502092+0000","last_peered":"2023-10-26T06:45:07.502092+0000","last_clean":"2023-10-26T06:45:07.502092+0000","last_became_active":"2023-10-26T06:45:04.911289+0000","last_became_peered":"2023-10-26T06:45:04.911289+0000","last_unstale":"2023-10-26T06:45:07.502092+0000","last_undegraded":"2023-10-26T06:45:07.502092+0000","last_fullsized":"2023-10-26T06:45:07.502092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.115211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888625+0000","last_change":"2023-10-26T06:45:04.919853+0000","last_active":"2023-10-26T06:45:05.888625+0000","last_peered":"2023-10-26T06:45:05.888625+0000","last_clean":"2023-10-26T06:45:05.888625+0000","last_became_active":"2023-10-26T06:45:04.919488+0000","last_became_peered":"2023-10-26T06:45:04.919488+0000","last_unstale":"2023-10-26T06:45:05.888625+0000","last_undegraded":"2023-10-26T06:45:05.888625+0000","last_fullsized":"2023-10-26T06:45:05.888625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:06.555851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822290+0000","last_change":"2023-10-26T06:45:05.903975+0000","last_active":"2023-10-26T06:45:07.822290+0000","last_peered":"2023-10-26T06:45:07.822290+0000","last_clean":"2023-10-26T06:45:07.822290+0000","last_became_active":"2023-10-26T06:45:05.903539+0000","last_became_peered":"2023-10-26T06:45:05.903539+0000","last_unstale":"2023-10-26T06:45:07.822290+0000","last_undegraded":"2023-10-26T06:45:07.822290+0000","last_fullsized":"2023-10-26T06:45:07.822290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:30.648947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885007+0000","last_change":"2023-10-26T06:45:04.919815+0000","last_active":"2023-10-26T06:45:06.885007+0000","last_peered":"2023-10-26T06:45:06.885007+0000","last_clean":"2023-10-26T06:45:06.885007+0000","last_became_active":"2023-10-26T06:45:04.918141+0000","last_became_peered":"2023-10-26T06:45:04.918141+0000","last_unstale":"2023-10-26T06:45:06.885007+0000","last_undegraded":"2023-10-26T06:45:06.885007+0000","last_fullsized":"2023-10-26T06:45:06.885007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:53.024534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138397+0000","last_change":"2023-10-26T06:45:05.898180+0000","last_active":"2023-10-26T06:45:07.138397+0000","last_peered":"2023-10-26T06:45:07.138397+0000","last_clean":"2023-10-26T06:45:07.138397+0000","last_became_active":"2023-10-26T06:45:05.896937+0000","last_became_peered":"2023-10-26T06:45:05.896937+0000","last_unstale":"2023-10-26T06:45:07.138397+0000","last_undegraded":"2023-10-26T06:45:07.138397+0000","last_fullsized":"2023-10-26T06:45:07.138397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:48.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.061365+0000","last_change":"2023-10-26T06:45:04.922456+0000","last_active":"2023-10-26T06:45:06.061365+0000","last_peered":"2023-10-26T06:45:06.061365+0000","last_clean":"2023-10-26T06:45:06.061365+0000","last_became_active":"2023-10-26T06:45:04.922112+0000","last_became_peered":"2023-10-26T06:45:04.922112+0000","last_unstale":"2023-10-26T06:45:06.061365+0000","last_undegraded":"2023-10-26T06:45:06.061365+0000","last_fullsized":"2023-10-26T06:45:06.061365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:44.055569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505613+0000","last_change":"2023-10-26T06:45:05.895440+0000","last_active":"2023-10-26T06:45:07.505613+0000","last_peered":"2023-10-26T06:45:07.505613+0000","last_clean":"2023-10-26T06:45:07.505613+0000","last_became_active":"2023-10-26T06:45:05.892605+0000","last_became_peered":"2023-10-26T06:45:05.892605+0000","last_unstale":"2023-10-26T06:45:07.505613+0000","last_undegraded":"2023-10-26T06:45:07.505613+0000","last_fullsized":"2023-10-26T06:45:07.505613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:09:27.392772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.899985+0000","last_change":"2023-10-26T06:45:04.916042+0000","last_active":"2023-10-26T06:45:05.899985+0000","last_peered":"2023-10-26T06:45:05.899985+0000","last_clean":"2023-10-26T06:45:05.899985+0000","last_became_active":"2023-10-26T06:45:04.915649+0000","last_became_peered":"2023-10-26T06:45:04.915649+0000","last_unstale":"2023-10-26T06:45:05.899985+0000","last_undegraded":"2023-10-26T06:45:05.899985+0000","last_fullsized":"2023-10-26T06:45:05.899985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:30.060074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689096+0000","last_change":"2023-10-26T06:45:05.908179+0000","last_active":"2023-10-26T06:45:07.689096+0000","last_peered":"2023-10-26T06:45:07.689096+0000","last_clean":"2023-10-26T06:45:07.689096+0000","last_became_active":"2023-10-26T06:45:05.907857+0000","last_became_peered":"2023-10-26T06:45:05.907857+0000","last_unstale":"2023-10-26T06:45:07.689096+0000","last_undegraded":"2023-10-26T06:45:07.689096+0000","last_fullsized":"2023-10-26T06:45:07.689096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:20.017194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.080625+0000","last_change":"2023-10-26T06:45:04.921016+0000","last_active":"2023-10-26T06:45:07.080625+0000","last_peered":"2023-10-26T06:45:07.080625+0000","last_clean":"2023-10-26T06:45:07.080625+0000","last_became_active":"2023-10-26T06:45:04.920741+0000","last_became_peered":"2023-10-26T06:45:04.920741+0000","last_unstale":"2023-10-26T06:45:07.080625+0000","last_undegraded":"2023-10-26T06:45:07.080625+0000","last_fullsized":"2023-10-26T06:45:07.080625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:05:06.218205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599772+0000","last_change":"2023-10-26T06:45:05.893562+0000","last_active":"2023-10-26T06:45:08.599772+0000","last_peered":"2023-10-26T06:45:08.599772+0000","last_clean":"2023-10-26T06:45:08.599772+0000","last_became_active":"2023-10-26T06:45:05.893111+0000","last_became_peered":"2023-10-26T06:45:05.893111+0000","last_unstale":"2023-10-26T06:45:08.599772+0000","last_undegraded":"2023-10-26T06:45:08.599772+0000","last_fullsized":"2023-10-26T06:45:08.599772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.305664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888454+0000","last_change":"2023-10-26T06:45:04.914936+0000","last_active":"2023-10-26T06:45:05.888454+0000","last_peered":"2023-10-26T06:45:05.888454+0000","last_clean":"2023-10-26T06:45:05.888454+0000","last_became_active":"2023-10-26T06:45:04.914617+0000","last_became_peered":"2023-10-26T06:45:04.914617+0000","last_unstale":"2023-10-26T06:45:05.888454+0000","last_undegraded":"2023-10-26T06:45:05.888454+0000","last_fullsized":"2023-10-26T06:45:05.888454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:15.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138153+0000","last_change":"2023-10-26T06:45:05.909101+0000","last_active":"2023-10-26T06:45:07.138153+0000","last_peered":"2023-10-26T06:45:07.138153+0000","last_clean":"2023-10-26T06:45:07.138153+0000","last_became_active":"2023-10-26T06:45:05.908800+0000","last_became_peered":"2023-10-26T06:45:05.908800+0000","last_unstale":"2023-10-26T06:45:07.138153+0000","last_undegraded":"2023-10-26T06:45:07.138153+0000","last_fullsized":"2023-10-26T06:45:07.138153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:37:18.605102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689147+0000","last_change":"2023-10-26T06:45:04.911714+0000","last_active":"2023-10-26T06:45:07.689147+0000","last_peered":"2023-10-26T06:45:07.689147+0000","last_clean":"2023-10-26T06:45:07.689147+0000","last_became_active":"2023-10-26T06:45:04.911343+0000","last_became_peered":"2023-10-26T06:45:04.911343+0000","last_unstale":"2023-10-26T06:45:07.689147+0000","last_undegraded":"2023-10-26T06:45:07.689147+0000","last_fullsized":"2023-10-26T06:45:07.689147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:30:35.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823521+0000","last_change":"2023-10-26T06:45:05.906804+0000","last_active":"2023-10-26T06:45:07.823521+0000","last_peered":"2023-10-26T06:45:07.823521+0000","last_clean":"2023-10-26T06:45:07.823521+0000","last_became_active":"2023-10-26T06:45:05.906522+0000","last_became_peered":"2023-10-26T06:45:05.906522+0000","last_unstale":"2023-10-26T06:45:07.823521+0000","last_undegraded":"2023-10-26T06:45:07.823521+0000","last_fullsized":"2023-10-26T06:45:07.823521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:32.707204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888946+0000","last_change":"2023-10-26T06:45:04.906345+0000","last_active":"2023-10-26T06:45:05.888946+0000","last_peered":"2023-10-26T06:45:05.888946+0000","last_clean":"2023-10-26T06:45:05.888946+0000","last_became_active":"2023-10-26T06:45:04.906034+0000","last_became_peered":"2023-10-26T06:45:04.906034+0000","last_unstale":"2023-10-26T06:45:05.888946+0000","last_undegraded":"2023-10-26T06:45:05.888946+0000","last_fullsized":"2023-10-26T06:45:05.888946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:53.690788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823815+0000","last_change":"2023-10-26T06:45:05.906368+0000","last_active":"2023-10-26T06:45:07.823815+0000","last_peered":"2023-10-26T06:45:07.823815+0000","last_clean":"2023-10-26T06:45:07.823815+0000","last_became_active":"2023-10-26T06:45:05.905471+0000","last_became_peered":"2023-10-26T06:45:05.905471+0000","last_unstale":"2023-10-26T06:45:07.823815+0000","last_undegraded":"2023-10-26T06:45:07.823815+0000","last_fullsized":"2023-10-26T06:45:07.823815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:45.493704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822626+0000","last_change":"2023-10-26T06:45:04.917530+0000","last_active":"2023-10-26T06:45:07.822626+0000","last_peered":"2023-10-26T06:45:07.822626+0000","last_clean":"2023-10-26T06:45:07.822626+0000","last_became_active":"2023-10-26T06:45:04.917193+0000","last_became_peered":"2023-10-26T06:45:04.917193+0000","last_unstale":"2023-10-26T06:45:07.822626+0000","last_undegraded":"2023-10-26T06:45:07.822626+0000","last_fullsized":"2023-10-26T06:45:07.822626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.965048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823728+0000","last_change":"2023-10-26T06:45:05.899429+0000","last_active":"2023-10-26T06:45:07.823728+0000","last_peered":"2023-10-26T06:45:07.823728+0000","last_clean":"2023-10-26T06:45:07.823728+0000","last_became_active":"2023-10-26T06:45:05.899081+0000","last_became_peered":"2023-10-26T06:45:05.899081+0000","last_unstale":"2023-10-26T06:45:07.823728+0000","last_undegraded":"2023-10-26T06:45:07.823728+0000","last_fullsized":"2023-10-26T06:45:07.823728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:41.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534374+0000","last_change":"2023-10-26T06:45:04.920295+0000","last_active":"2023-10-26T06:45:06.534374+0000","last_peered":"2023-10-26T06:45:06.534374+0000","last_clean":"2023-10-26T06:45:06.534374+0000","last_became_active":"2023-10-26T06:45:04.919920+0000","last_became_peered":"2023-10-26T06:45:04.919920+0000","last_unstale":"2023-10-26T06:45:06.534374+0000","last_undegraded":"2023-10-26T06:45:06.534374+0000","last_fullsized":"2023-10-26T06:45:06.534374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:59.260841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824231+0000","last_change":"2023-10-26T06:45:05.899610+0000","last_active":"2023-10-26T06:45:07.824231+0000","last_peered":"2023-10-26T06:45:07.824231+0000","last_clean":"2023-10-26T06:45:07.824231+0000","last_became_active":"2023-10-26T06:45:05.899283+0000","last_became_peered":"2023-10-26T06:45:05.899283+0000","last_unstale":"2023-10-26T06:45:07.824231+0000","last_undegraded":"2023-10-26T06:45:07.824231+0000","last_fullsized":"2023-10-26T06:45:07.824231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:48.730703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599798+0000","last_change":"2023-10-26T06:45:05.892012+0000","last_active":"2023-10-26T06:45:08.599798+0000","last_peered":"2023-10-26T06:45:08.599798+0000","last_clean":"2023-10-26T06:45:08.599798+0000","last_became_active":"2023-10-26T06:45:05.891744+0000","last_became_peered":"2023-10-26T06:45:05.891744+0000","last_unstale":"2023-10-26T06:45:08.599798+0000","last_undegraded":"2023-10-26T06:45:08.599798+0000","last_fullsized":"2023-10-26T06:45:08.599798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:29.599184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884873+0000","last_change":"2023-10-26T06:45:05.904348+0000","last_active":"2023-10-26T06:45:06.884873+0000","last_peered":"2023-10-26T06:45:06.884873+0000","last_clean":"2023-10-26T06:45:06.884873+0000","last_became_active":"2023-10-26T06:45:05.903949+0000","last_became_peered":"2023-10-26T06:45:05.903949+0000","last_unstale":"2023-10-26T06:45:06.884873+0000","last_undegraded":"2023-10-26T06:45:06.884873+0000","last_fullsized":"2023-10-26T06:45:06.884873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:29.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884744+0000","last_change":"2023-10-26T06:45:05.901313+0000","last_active":"2023-10-26T06:45:06.884744+0000","last_peered":"2023-10-26T06:45:06.884744+0000","last_clean":"2023-10-26T06:45:06.884744+0000","last_became_active":"2023-10-26T06:45:05.900993+0000","last_became_peered":"2023-10-26T06:45:05.900993+0000","last_unstale":"2023-10-26T06:45:06.884744+0000","last_undegraded":"2023-10-26T06:45:06.884744+0000","last_fullsized":"2023-10-26T06:45:06.884744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:00.093645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505407+0000","last_change":"2023-10-26T06:45:05.901225+0000","last_active":"2023-10-26T06:45:07.505407+0000","last_peered":"2023-10-26T06:45:07.505407+0000","last_clean":"2023-10-26T06:45:07.505407+0000","last_became_active":"2023-10-26T06:45:05.900743+0000","last_became_peered":"2023-10-26T06:45:05.900743+0000","last_unstale":"2023-10-26T06:45:07.505407+0000","last_undegraded":"2023-10-26T06:45:07.505407+0000","last_fullsized":"2023-10-26T06:45:07.505407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:02.359654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822222+0000","last_change":"2023-10-26T06:45:05.906805+0000","last_active":"2023-10-26T06:45:07.822222+0000","last_peered":"2023-10-26T06:45:07.822222+0000","last_clean":"2023-10-26T06:45:07.822222+0000","last_became_active":"2023-10-26T06:45:05.906464+0000","last_became_peered":"2023-10-26T06:45:05.906464+0000","last_unstale":"2023-10-26T06:45:07.822222+0000","last_undegraded":"2023-10-26T06:45:07.822222+0000","last_fullsized":"2023-10-26T06:45:07.822222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:24.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599992+0000","last_change":"2023-10-26T06:45:05.898568+0000","last_active":"2023-10-26T06:45:08.599992+0000","last_peered":"2023-10-26T06:45:08.599992+0000","last_clean":"2023-10-26T06:45:08.599992+0000","last_became_active":"2023-10-26T06:45:05.897886+0000","last_became_peered":"2023-10-26T06:45:05.897886+0000","last_unstale":"2023-10-26T06:45:08.599992+0000","last_undegraded":"2023-10-26T06:45:08.599992+0000","last_fullsized":"2023-10-26T06:45:08.599992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:14.998233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885177+0000","last_change":"2023-10-26T06:45:05.899326+0000","last_active":"2023-10-26T06:45:06.885177+0000","last_peered":"2023-10-26T06:45:06.885177+0000","last_clean":"2023-10-26T06:45:06.885177+0000","last_became_active":"2023-10-26T06:45:05.899040+0000","last_became_peered":"2023-10-26T06:45:05.899040+0000","last_unstale":"2023-10-26T06:45:06.885177+0000","last_undegraded":"2023-10-26T06:45:06.885177+0000","last_fullsized":"2023-10-26T06:45:06.885177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:55:35.666122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138250+0000","last_change":"2023-10-26T06:45:05.904155+0000","last_active":"2023-10-26T06:45:07.138250+0000","last_peered":"2023-10-26T06:45:07.138250+0000","last_clean":"2023-10-26T06:45:07.138250+0000","last_became_active":"2023-10-26T06:45:05.903734+0000","last_became_peered":"2023-10-26T06:45:05.903734+0000","last_unstale":"2023-10-26T06:45:07.138250+0000","last_undegraded":"2023-10-26T06:45:07.138250+0000","last_fullsized":"2023-10-26T06:45:07.138250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:43.252789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822424+0000","last_change":"2023-10-26T06:45:05.897174+0000","last_active":"2023-10-26T06:45:07.822424+0000","last_peered":"2023-10-26T06:45:07.822424+0000","last_clean":"2023-10-26T06:45:07.822424+0000","last_became_active":"2023-10-26T06:45:05.896893+0000","last_became_peered":"2023-10-26T06:45:05.896893+0000","last_unstale":"2023-10-26T06:45:07.822424+0000","last_undegraded":"2023-10-26T06:45:07.822424+0000","last_fullsized":"2023-10-26T06:45:07.822424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:14.291682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137904+0000","last_change":"2023-10-26T06:45:05.905987+0000","last_active":"2023-10-26T06:45:07.137904+0000","last_peered":"2023-10-26T06:45:07.137904+0000","last_clean":"2023-10-26T06:45:07.137904+0000","last_became_active":"2023-10-26T06:45:05.904933+0000","last_became_peered":"2023-10-26T06:45:05.904933+0000","last_unstale":"2023-10-26T06:45:07.137904+0000","last_undegraded":"2023-10-26T06:45:07.137904+0000","last_fullsized":"2023-10-26T06:45:07.137904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:19.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885189+0000","last_change":"2023-10-26T06:45:05.912781+0000","last_active":"2023-10-26T06:45:06.885189+0000","last_peered":"2023-10-26T06:45:06.885189+0000","last_clean":"2023-10-26T06:45:06.885189+0000","last_became_active":"2023-10-26T06:45:05.912415+0000","last_became_peered":"2023-10-26T06:45:05.912415+0000","last_unstale":"2023-10-26T06:45:06.885189+0000","last_undegraded":"2023-10-26T06:45:06.885189+0000","last_fullsized":"2023-10-26T06:45:06.885189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:25.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822121+0000","last_change":"2023-10-26T06:45:05.909106+0000","last_active":"2023-10-26T06:45:07.822121+0000","last_peered":"2023-10-26T06:45:07.822121+0000","last_clean":"2023-10-26T06:45:07.822121+0000","last_became_active":"2023-10-26T06:45:05.908751+0000","last_became_peered":"2023-10-26T06:45:05.908751+0000","last_unstale":"2023-10-26T06:45:07.822121+0000","last_undegraded":"2023-10-26T06:45:07.822121+0000","last_fullsized":"2023-10-26T06:45:07.822121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:17.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599847+0000","last_change":"2023-10-26T06:45:05.898262+0000","last_active":"2023-10-26T06:45:08.599847+0000","last_peered":"2023-10-26T06:45:08.599847+0000","last_clean":"2023-10-26T06:45:08.599847+0000","last_became_active":"2023-10-26T06:45:05.897947+0000","last_became_peered":"2023-10-26T06:45:05.897947+0000","last_unstale":"2023-10-26T06:45:08.599847+0000","last_undegraded":"2023-10-26T06:45:08.599847+0000","last_fullsized":"2023-10-26T06:45:08.599847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:39:45.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":37083,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":18,"num_read_kb":46,"num_write":55,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":44308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49,"ondisk_log_size":49,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416396,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":892928,"data_stored":435803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]}]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1054250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.502}]},{"osd":5,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41699999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58399999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.747},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.623}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":445170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.435},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63}]},{"osd":2,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.749}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.77200000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":444861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.0609999999999999,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.749},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.628},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":5.0940000000000003,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Oct 26 06:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.79700000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83499999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":7,"up_from":52,"seq":223338299423,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1035661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.41999999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.45400000000000001,"15min":0.44600000000000001},"min":{"1min":0.34399999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003}]},{"osd":3,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.54500000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.873,"5min":1.006,"15min":1.006},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72299999999999998,"5min":0.748,"15min":0.748},"last":0.373},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48699999999999999,"15min":0.47899999999999998},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.438}]},{"osd":5,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.621,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92000000000000004,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60799999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.69099999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.748}]},{"osd":10,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.627}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462947,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":444711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.44900000000000001,"15min":0.443},"min":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.91100000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.432,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.54100000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67600000000000005,"15min":0.68100000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.81200000000000006,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.437},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.55800000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0089999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85899999999999999,"5min":1.002,"15min":1.002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Oct 26 06:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.58499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.71499999999999997,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80000000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Thu Oct 26 06:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.72099999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.86099999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":446200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.434,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.498},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73499999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Oct 26 06:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81399999999999995}]},{"osd":4,"last update":"Thu Oct 26 06:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.435,"15min":0.435},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.29499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:45:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47099999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69999999999999996}]},{"osd":8,"last update":"Thu Oct 26 06:44:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705728,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":444678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 06:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.59999999999999998,"15min":0.58099999999999996},"min":{"1min":0.48499999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0049999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.46000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.434,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93400000000000005,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90800000000000003,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Oct 26 06:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":2.198,"15min":2.198},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97599999999999998,"5min":1.48,"15min":1.48},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Oct 26 06:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0920000000000001,"5min":1.514,"15min":1.514},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.61899999999999999},"min":{"1min":0.47399999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93400000000000005,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88400000000000001,"5min":1.002,"15min":1.002},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":1.02},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.68799999999999994,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.089,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.089},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.66300000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.131,"5min":1.294,"15min":1.294},"last":0.95699999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.123}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.92500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378678,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":445565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81699999999999995,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.39900000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Oct 26 06:44:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56599999999999995,"15min":0.55300000000000005},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.46100000000000002},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.47699999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.78700000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.997,"15min":0.997},"last":1.3480000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.3919999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.68999999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.79300000000000004}]},{"osd":10,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149746,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.377},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.442},"min":{"1min":0.39100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.83499999999999996,"5min":1.103,"15min":1.103},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Oct 26 06:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.03,"5min":1.335,"15min":1.335},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.48499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.04,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.747}]},{"osd":2,"last update":"Thu Oct 26 06:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.50700000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91500000000000004,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.48299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90600000000000003,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Oct 26 06:44:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.51700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1220000000000001,"5min":1.274,"15min":1.274},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.58499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.024,"5min":1.304,"15min":1.304},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.68400000000000005},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72999999999999998,"15min":0.69499999999999995},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.44800000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.083,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77100000000000002,"15min":0.77000000000000002},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0089999999999999,"5min":1.202,"15min":1.202},"last":0.71299999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.57199999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.623}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888108,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":444610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.434}]},{"osd":1,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.37,"15min":0.34599999999999997},"min":{"1min":0.28999999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"min":{"1min":0.253,"5min":0.245,"15min":0.245},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.443,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61799999999999999,"15min":0.59199999999999997},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42399999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:44:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.41699999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.67500000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.439,"15min":0.439},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.372},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.35399999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.437}]},{"osd":9,"last update":"Thu Oct 26 06:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.59099999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.505},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.44600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724585,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1511424,"data_stored":1035138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.47499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.32500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39000000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.56399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Oct 26 06:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67500000000000004,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56200000000000006}]},{"osd":8,"last update":"Thu Oct 26 06:44:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.627},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57499999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68300000000000005}]},{"osd":10,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":36479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-26T06:45:13.912 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T06:45:13.913 INFO:tasks.cephadm:Setting up client nodes... 2023-10-26T06:45:13.914 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-26T06:45:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:15.239 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:15 smithi089 ceph-mon[95031]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:45:15.239 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:15 smithi089 ceph-mon[95031]: pgmap v258: 97 pgs: 97 active+clean; 613 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-10-26T06:45:15.239 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:15 smithi089 ceph-mon[95031]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:45:15.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:15 smithi192 ceph-mon[131661]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:45:15.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:15 smithi192 ceph-mon[131661]: pgmap v258: 97 pgs: 97 active+clean; 613 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-10-26T06:45:15.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:15 smithi192 ceph-mon[131661]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:45:15.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:15 smithi135 ceph-mon[131403]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:45:15.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:15 smithi135 ceph-mon[131403]: pgmap v258: 97 pgs: 97 active+clean; 613 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-10-26T06:45:15.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:15 smithi135 ceph-mon[131403]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:45:16.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 -- 172.21.15.89:0/1590796453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 msgr2=0x7f479410c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:16.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 --2- 172.21.15.89:0/1590796453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 0x7f479410c520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b0d0 tx=0x7f478c00b3e0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 -- 172.21.15.89:0/1590796453 shutdown_connections 2023-10-26T06:45:16.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 --2- 172.21.15.89:0/1590796453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 0x7f479410c520 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 --2- 172.21.15.89:0/1590796453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4794104540 0x7f4794104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 --2- 172.21.15.89:0/1590796453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47940ff720 0x7f47940ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 -- 172.21.15.89:0/1590796453 >> 172.21.15.89:0/1590796453 conn(0x7f4794075710 msgr2=0x7f4794075b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:16.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 -- 172.21.15.89:0/1590796453 shutdown_connections 2023-10-26T06:45:16.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.322+0000 7f4799d05700 1 -- 172.21.15.89:0/1590796453 wait complete. 2023-10-26T06:45:16.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.323+0000 7f4799d05700 1 Processor -- start 2023-10-26T06:45:16.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4799d05700 1 -- start start 2023-10-26T06:45:16.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4799d05700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47940ff720 0x7f4794101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:16.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4799d05700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 0x7f4794101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:16.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4799d05700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794104540 0x7f4794101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:16.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4799d05700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47941a5110 con 0x7f47941000e0 2023-10-26T06:45:16.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4799d05700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47941a5280 con 0x7f47940ff720 2023-10-26T06:45:16.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4799d05700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47941a5440 con 0x7f4794104540 2023-10-26T06:45:16.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4792ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 0x7f4794101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:16.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f47937fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47940ff720 0x7f4794101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f47937fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47940ff720 0x7f4794101240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:53440/0 (socket says 172.21.15.89:53440) 2023-10-26T06:45:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f47937fe700 1 -- 172.21.15.89:0/1942550125 learned_addr learned my addr 172.21.15.89:0/1942550125 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.324+0000 7f4793fff700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794104540 0x7f4794101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f47937fe700 1 -- 172.21.15.89:0/1942550125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794104540 msgr2=0x7f4794101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f47937fe700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794104540 0x7f4794101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f47937fe700 1 -- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 msgr2=0x7f4794101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f47937fe700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 0x7f4794101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f47937fe700 1 -- 172.21.15.89:0/1942550125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c009d00 con 0x7f47940ff720 2023-10-26T06:45:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f4793fff700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794104540 0x7f4794101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f4792ffd700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 0x7f4794101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f47937fe700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47940ff720 0x7f4794101240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f478400e9a0 tx=0x7f478400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f4790ff9700 1 -- 172.21.15.89:0/1942550125 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784009790 con 0x7f47940ff720 2023-10-26T06:45:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47941a5720 con 0x7f47940ff720 2023-10-26T06:45:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f4790ff9700 1 -- 172.21.15.89:0/1942550125 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478400cbd0 con 0x7f47940ff720 2023-10-26T06:45:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f4790ff9700 1 -- 172.21.15.89:0/1942550125 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784018680 con 0x7f47940ff720 2023-10-26T06:45:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.325+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47941a5c20 con 0x7f47940ff720 2023-10-26T06:45:16.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.327+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4794108c10 con 0x7f47940ff720 2023-10-26T06:45:16.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.328+0000 7f4790ff9700 1 -- 172.21.15.89:0/1942550125 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f478400cd40 con 0x7f47940ff720 2023-10-26T06:45:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.329+0000 7f4790ff9700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f477c0a1560 0x7f477c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.330+0000 7f4790ff9700 1 -- 172.21.15.89:0/1942550125 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f4784014070 con 0x7f47940ff720 2023-10-26T06:45:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.330+0000 7f4792ffd700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f477c0a1560 0x7f477c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.330+0000 7f4792ffd700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f477c0a1560 0x7f477c0a3a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4794102970 tx=0x7f47880073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.333+0000 7f4790ff9700 1 -- 172.21.15.89:0/1942550125 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4784052100 con 0x7f47940ff720 2023-10-26T06:45:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.609+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4794100db0 con 0x7f47940ff720 2023-10-26T06:45:16.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.615+0000 7f4790ff9700 1 -- 172.21.15.89:0/1942550125 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f4784096600 con 0x7f47940ff720 2023-10-26T06:45:16.617 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2023-10-26T06:45:16.617 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQD8CjplkJp6JBAAw+bz5vmRE3mwfE9QbyDn0w== 2023-10-26T06:45:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f477c0a1560 msgr2=0x7f477c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f477c0a1560 0x7f477c0a3a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4794102970 tx=0x7f47880073d0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47940ff720 msgr2=0x7f4794101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47940ff720 0x7f4794101240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f478400e9a0 tx=0x7f478400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 shutdown_connections 2023-10-26T06:45:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f477c0a1560 0x7f477c0a3a20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794104540 0x7f4794101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47940ff720 0x7f4794101240 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 --2- 172.21.15.89:0/1942550125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47941000e0 0x7f4794101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.617+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 >> 172.21.15.89:0/1942550125 conn(0x7f4794075710 msgr2=0x7f4794068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.618+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 shutdown_connections 2023-10-26T06:45:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:16.618+0000 7f4799d05700 1 -- 172.21.15.89:0/1942550125 wait complete. 2023-10-26T06:45:17.154 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:45:17.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-26T06:45:17.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-26T06:45:17.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph config log 1 --format=json 2023-10-26T06:45:17.353 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:17 smithi089 ceph-mon[95031]: pgmap v259: 97 pgs: 97 active+clean; 613 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2023-10-26T06:45:17.353 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:17 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1942550125' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-26T06:45:17.354 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:17 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-26T06:45:17.354 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:17 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-26T06:45:17.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:17 smithi192 ceph-mon[131661]: pgmap v259: 97 pgs: 97 active+clean; 613 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2023-10-26T06:45:17.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:17 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1942550125' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-26T06:45:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:17 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-26T06:45:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:17 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-26T06:45:17.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:17 smithi135 ceph-mon[131403]: pgmap v259: 97 pgs: 97 active+clean; 613 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2023-10-26T06:45:17.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:17 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1942550125' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-26T06:45:17.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:17 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-26T06:45:17.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:17 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-26T06:45:18.026 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:19 smithi192 ceph-mon[131661]: pgmap v260: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2023-10-26T06:45:19.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 -- 172.21.15.89:0/3734224852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8100fb0 msgr2=0x7fdfb810d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3734224852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8100fb0 0x7fdfb810d4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000b0a0 tx=0x7fdfb000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 -- 172.21.15.89:0/3734224852 shutdown_connections 2023-10-26T06:45:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3734224852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8100fb0 0x7fdfb810d4c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3734224852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb8106680 0x7fdfb8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3734224852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfb81005f0 0x7fdfb8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 -- 172.21.15.89:0/3734224852 >> 172.21.15.89:0/3734224852 conn(0x7fdfb80fc120 msgr2=0x7fdfb80fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 -- 172.21.15.89:0/3734224852 shutdown_connections 2023-10-26T06:45:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.430+0000 7fdfb759e700 1 -- 172.21.15.89:0/3734224852 wait complete. 2023-10-26T06:45:19.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.431+0000 7fdfb759e700 1 Processor -- start 2023-10-26T06:45:19.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb759e700 1 -- start start 2023-10-26T06:45:19.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb759e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb81005f0 0x7fdfb8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:19.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb759e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfb8100fb0 0x7fdfb8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:19.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8106680 0x7fdfb819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb759e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb819dc80 con 0x7fdfb8106680 2023-10-26T06:45:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb759e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb819ddf0 con 0x7fdfb81005f0 2023-10-26T06:45:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb759e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb819dfb0 con 0x7fdfb8100fb0 2023-10-26T06:45:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8106680 0x7fdfb819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:19.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8106680 0x7fdfb819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60342/0 (socket says 172.21.15.89:60342) 2023-10-26T06:45:19.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb6d9d700 1 -- 172.21.15.89:0/3853187388 learned_addr learned my addr 172.21.15.89:0/3853187388 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:19.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb659c700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb81005f0 0x7fdfb8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:19.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.432+0000 7fdfb5d9b700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfb8100fb0 0x7fdfb8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:19.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb6d9d700 1 -- 172.21.15.89:0/3853187388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfb8100fb0 msgr2=0x7fdfb8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:19.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb6d9d700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfb8100fb0 0x7fdfb8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb6d9d700 1 -- 172.21.15.89:0/3853187388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb81005f0 msgr2=0x7fdfb8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb6d9d700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb81005f0 0x7fdfb8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb6d9d700 1 -- 172.21.15.89:0/3853187388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa8009520 con 0x7fdfb8106680 2023-10-26T06:45:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb659c700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb81005f0 0x7fdfb8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:19.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb6d9d700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8106680 0x7fdfb819d670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000b070 tx=0x7fdfb0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:19.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfa77fe700 1 -- 172.21.15.89:0/3853187388 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000e040 con 0x7fdfb8106680 2023-10-26T06:45:19.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfa77fe700 1 -- 172.21.15.89:0/3853187388 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfb001d470 con 0x7fdfb8106680 2023-10-26T06:45:19.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfa77fe700 1 -- 172.21.15.89:0/3853187388 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00052a0 con 0x7fdfb8106680 2023-10-26T06:45:19.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb0009d00 con 0x7fdfb8106680 2023-10-26T06:45:19.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.433+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb81a5920 con 0x7fdfb8106680 2023-10-26T06:45:19.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.434+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb8101d30 con 0x7fdfb8106680 2023-10-26T06:45:19.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.436+0000 7fdfa77fe700 1 -- 172.21.15.89:0/3853187388 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fdfb001d850 con 0x7fdfb8106680 2023-10-26T06:45:19.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.437+0000 7fdfa77fe700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdfa00a15e0 0x7fdfa00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:19.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.438+0000 7fdfa77fe700 1 -- 172.21.15.89:0/3853187388 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fdfb00cdf30 con 0x7fdfb8106680 2023-10-26T06:45:19.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.438+0000 7fdfb659c700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdfa00a15e0 0x7fdfa00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:19.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.438+0000 7fdfb659c700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdfa00a15e0 0x7fdfa00a3aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8101730 tx=0x7fdfa8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:19.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.442+0000 7fdfa77fe700 1 -- 172.21.15.89:0/3853187388 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdfb0097480 con 0x7fdfb8106680 2023-10-26T06:45:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:19 smithi135 ceph-mon[131403]: pgmap v260: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2023-10-26T06:45:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:19 smithi089 ceph-mon[95031]: pgmap v260: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 KiB/s wr, 7 op/s 2023-10-26T06:45:19.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.642+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7fdfb804e4c0 con 0x7fdfb8106680 2023-10-26T06:45:19.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.642+0000 7fdfa77fe700 1 -- 172.21.15.89:0/3853187388 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v46) v1 ==== 86+0+133 (secure 0 0 0) 0x7fdfb001d5e0 con 0x7fdfb8106680 2023-10-26T06:45:19.645 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.644+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdfa00a15e0 msgr2=0x7fdfa00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.644+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdfa00a15e0 0x7fdfa00a3aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8101730 tx=0x7fdfa8009260 comp rx=0 tx=0).stop 2023-10-26T06:45:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.644+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8106680 msgr2=0x7fdfb819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.644+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8106680 0x7fdfb819d670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000b070 tx=0x7fdfb0007930 comp rx=0 tx=0).stop 2023-10-26T06:45:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 shutdown_connections 2023-10-26T06:45:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdfa00a15e0 0x7fdfa00a3aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdfb8106680 0x7fdfb819d670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb81005f0 0x7fdfb8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 --2- 172.21.15.89:0/3853187388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfb8100fb0 0x7fdfb8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 >> 172.21.15.89:0/3853187388 conn(0x7fdfb80fc120 msgr2=0x7fdfb80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 shutdown_connections 2023-10-26T06:45:19.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:19.645+0000 7fdfb759e700 1 -- 172.21.15.89:0/3853187388 wait complete. 2023-10-26T06:45:20.205 INFO:teuthology.orchestra.run.smithi089.stdout:[{"version":46,"timestamp":"2023-10-26T06:44:59.077729+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-10-26T06:45:20.205 INFO:tasks.ceph_manager:config epoch is 46 2023-10-26T06:45:20.206 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-26T06:45:20.206 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-26T06:45:20.207 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-26T06:45:20.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:20 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3853187388' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-26T06:45:20.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:20 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3853187388' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-26T06:45:20.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:20 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3853187388' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-26T06:45:20.943 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:21 smithi192 ceph-mon[131661]: pgmap v261: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2023-10-26T06:45:21.428 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:21 smithi089 ceph-mon[95031]: pgmap v261: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2023-10-26T06:45:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:21 smithi135 ceph-mon[131403]: pgmap v261: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2023-10-26T06:45:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.436+0000 7fe6a60cb700 1 -- 172.21.15.89:0/2893468632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a0101260 msgr2=0x7fe6a010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.436+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/2893468632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a0101260 0x7fe6a010d650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b0a0 tx=0x7fe69800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.436+0000 7fe6a60cb700 1 -- 172.21.15.89:0/2893468632 shutdown_connections 2023-10-26T06:45:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.436+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/2893468632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a0101260 0x7fe6a010d650 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.436+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/2893468632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6a01057c0 0x7fe6a0105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.436+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/2893468632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a01008a0 0x7fe6a0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.436+0000 7fe6a60cb700 1 -- 172.21.15.89:0/2893468632 >> 172.21.15.89:0/2893468632 conn(0x7fe6a00779e0 msgr2=0x7fe6a0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.437+0000 7fe6a60cb700 1 -- 172.21.15.89:0/2893468632 shutdown_connections 2023-10-26T06:45:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.437+0000 7fe6a60cb700 1 -- 172.21.15.89:0/2893468632 wait complete. 2023-10-26T06:45:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 Processor -- start 2023-10-26T06:45:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 -- start start 2023-10-26T06:45:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6a01008a0 0x7fe6a0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a0101260 0x7fe6a0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a01057c0 0x7fe6a0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a01a5050 con 0x7fe6a01057c0 2023-10-26T06:45:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a01a51c0 con 0x7fe6a01008a0 2023-10-26T06:45:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe6a60cb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a01a5380 con 0x7fe6a0101260 2023-10-26T06:45:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe69effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a0101260 0x7fe6a0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe69ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a01057c0 0x7fe6a0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.438+0000 7fe69f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6a01008a0 0x7fe6a0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6a01008a0 0x7fe6a0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:49914/0 (socket says 172.21.15.89:49914) 2023-10-26T06:45:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69f7fe700 1 -- 172.21.15.89:0/3456379653 learned_addr learned my addr 172.21.15.89:0/3456379653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a0101260 0x7fe6a0073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60436/0 (socket says 172.21.15.89:60436) 2023-10-26T06:45:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69ffff700 1 -- 172.21.15.89:0/3456379653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a0101260 msgr2=0x7fe6a0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69ffff700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a0101260 0x7fe6a0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69ffff700 1 -- 172.21.15.89:0/3456379653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6a01008a0 msgr2=0x7fe6a0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69ffff700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6a01008a0 0x7fe6a0072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69ffff700 1 -- 172.21.15.89:0/3456379653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698009d00 con 0x7fe6a01057c0 2023-10-26T06:45:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69effd700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a0101260 0x7fe6a0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69ffff700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a01057c0 0x7fe6a0073570 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b070 tx=0x7fe698007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69cff9700 1 -- 172.21.15.89:0/3456379653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800e040 con 0x7fe6a01057c0 2023-10-26T06:45:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a01a5600 con 0x7fe6a01057c0 2023-10-26T06:45:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.439+0000 7fe69cff9700 1 -- 172.21.15.89:0/3456379653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe69801d470 con 0x7fe6a01057c0 2023-10-26T06:45:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.440+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a01a5a70 con 0x7fe6a01057c0 2023-10-26T06:45:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.440+0000 7fe69cff9700 1 -- 172.21.15.89:0/3456379653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698005240 con 0x7fe6a01057c0 2023-10-26T06:45:22.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.441+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a0101f30 con 0x7fe6a01057c0 2023-10-26T06:45:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.444+0000 7fe69cff9700 1 -- 172.21.15.89:0/3456379653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe69801d5e0 con 0x7fe6a01057c0 2023-10-26T06:45:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.445+0000 7fe69cff9700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6880a1560 0x7fe6880a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.445+0000 7fe69cff9700 1 -- 172.21.15.89:0/3456379653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fe6980ce0e0 con 0x7fe6a01057c0 2023-10-26T06:45:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.446+0000 7fe69f7fe700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6880a1560 0x7fe6880a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.446+0000 7fe69f7fe700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6880a1560 0x7fe6880a3a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe69000bcf0 tx=0x7fe69000b4a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.448+0000 7fe69cff9700 1 -- 172.21.15.89:0/3456379653 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe698097630 con 0x7fe6a01057c0 2023-10-26T06:45:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.704+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe6a004e4c0 con 0x7fe6a01057c0 2023-10-26T06:45:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.709+0000 7fe69cff9700 1 -- 172.21.15.89:0/3456379653 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) v1 ==== 74+0+260422 (secure 0 0 0) 0x7fe69809ac50 con 0x7fe6a01057c0 2023-10-26T06:45:22.715 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6880a1560 msgr2=0x7fe6880a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6880a1560 0x7fe6880a3a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe69000bcf0 tx=0x7fe69000b4a0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a01057c0 msgr2=0x7fe6a0073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a01057c0 0x7fe6a0073570 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b070 tx=0x7fe698007860 comp rx=0 tx=0).stop 2023-10-26T06:45:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 shutdown_connections 2023-10-26T06:45:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6880a1560 0x7fe6880a3a20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6a01057c0 0x7fe6a0073570 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6a01008a0 0x7fe6a0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 --2- 172.21.15.89:0/3456379653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a0101260 0x7fe6a0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 >> 172.21.15.89:0/3456379653 conn(0x7fe6a00779e0 msgr2=0x7fe6a00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 shutdown_connections 2023-10-26T06:45:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:22.714+0000 7fe6a60cb700 1 -- 172.21.15.89:0/3456379653 wait complete. 2023-10-26T06:45:23.122 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":21,"active_gid":14217,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1106080795},{"type":"v1","addr":"172.21.15.89:6801","nonce":1106080795}]},"active_addr":"172.21.15.89:6801/1106080795","active_change":"2023-10-26T06:39:01.640088+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24163,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24167,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.89:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1736390285}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":962950851}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3681015116}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":4099734451}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":747575372}]}]} 2023-10-26T06:45:23.129 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-26T06:45:23.130 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-26T06:45:23.130 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd dump --format=json 2023-10-26T06:45:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:23 smithi192 ceph-mon[131661]: pgmap v262: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2023-10-26T06:45:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:23 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3456379653' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-26T06:45:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:23 smithi135 ceph-mon[131403]: pgmap v262: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2023-10-26T06:45:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:23 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3456379653' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-26T06:45:23.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:23 smithi089 ceph-mon[95031]: pgmap v262: 97 pgs: 97 active+clean; 617 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2023-10-26T06:45:23.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:23 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3456379653' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-26T06:45:23.869 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:25.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 -- 172.21.15.89:0/2186389983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe908101260 msgr2=0x7fe90810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:25.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2186389983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe908101260 0x7fe90810d650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe90000b0a0 tx=0x7fe90000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 -- 172.21.15.89:0/2186389983 shutdown_connections 2023-10-26T06:45:25.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2186389983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe908101260 0x7fe90810d650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2186389983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9081057c0 0x7fe908105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2186389983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9081008a0 0x7fe908100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 -- 172.21.15.89:0/2186389983 >> 172.21.15.89:0/2186389983 conn(0x7fe9080779e0 msgr2=0x7fe908077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:25.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 -- 172.21.15.89:0/2186389983 shutdown_connections 2023-10-26T06:45:25.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.329+0000 7fe90eea9700 1 -- 172.21.15.89:0/2186389983 wait complete. 2023-10-26T06:45:25.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 Processor -- start 2023-10-26T06:45:25.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 -- start start 2023-10-26T06:45:25.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9081008a0 0x7fe908198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:25.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe908101260 0x7fe908199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:25.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9081057c0 0x7fe90819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:25.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90819de10 con 0x7fe9081008a0 2023-10-26T06:45:25.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90819df80 con 0x7fe9081057c0 2023-10-26T06:45:25.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.331+0000 7fe90eea9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90819e140 con 0x7fe908101260 2023-10-26T06:45:25.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9081008a0 0x7fe908198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:25.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9081008a0 0x7fe908198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50872/0 (socket says 172.21.15.89:50872) 2023-10-26T06:45:25.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 -- 172.21.15.89:0/2704693304 learned_addr learned my addr 172.21.15.89:0/2704693304 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:25.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe907fff700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe908101260 0x7fe908199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:25.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90d446700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9081057c0 0x7fe90819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:25.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 -- 172.21.15.89:0/2704693304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe908101260 msgr2=0x7fe908199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:25.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe908101260 0x7fe908199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 -- 172.21.15.89:0/2704693304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9081057c0 msgr2=0x7fe90819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:25.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9081057c0 0x7fe90819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 -- 172.21.15.89:0/2704693304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe900009d00 con 0x7fe9081008a0 2023-10-26T06:45:25.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe90cc45700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9081008a0 0x7fe908198f10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e970 tx=0x7fe8fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:25.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.332+0000 7fe905ffb700 1 -- 172.21.15.89:0/2704693304 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc009790 con 0x7fe9081008a0 2023-10-26T06:45:25.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.333+0000 7fe905ffb700 1 -- 172.21.15.89:0/2704693304 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8fc00eec0 con 0x7fe9081008a0 2023-10-26T06:45:25.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.333+0000 7fe905ffb700 1 -- 172.21.15.89:0/2704693304 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc003cf0 con 0x7fe9081008a0 2023-10-26T06:45:25.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.333+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90819e420 con 0x7fe9081008a0 2023-10-26T06:45:25.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.333+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9081a5c80 con 0x7fe9081008a0 2023-10-26T06:45:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.334+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe908101f30 con 0x7fe9081008a0 2023-10-26T06:45:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.336+0000 7fe905ffb700 1 -- 172.21.15.89:0/2704693304 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe8fc00ccd0 con 0x7fe9081008a0 2023-10-26T06:45:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.337+0000 7fe905ffb700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8f80a1560 0x7fe8f80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:25.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.337+0000 7fe905ffb700 1 -- 172.21.15.89:0/2704693304 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fe8fc014070 con 0x7fe9081008a0 2023-10-26T06:45:25.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.337+0000 7fe907fff700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8f80a1560 0x7fe8f80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:25.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.338+0000 7fe907fff700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8f80a1560 0x7fe8f80a3a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe90819a470 tx=0x7fe8f40073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:25.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.341+0000 7fe905ffb700 1 -- 172.21.15.89:0/2704693304 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe8fc0965f0 con 0x7fe9081008a0 2023-10-26T06:45:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:25 smithi192 ceph-mon[131661]: pgmap v263: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2023-10-26T06:45:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:25 smithi135 ceph-mon[131403]: pgmap v263: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2023-10-26T06:45:25.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:25 smithi089 ceph-mon[95031]: pgmap v263: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2023-10-26T06:45:25.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.538+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe90804e4c0 con 0x7fe9081008a0 2023-10-26T06:45:25.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.540+0000 7fe905ffb700 1 -- 172.21.15.89:0/2704693304 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22407 (secure 0 0 0) 0x7fe8fc07f7a0 con 0x7fe9081008a0 2023-10-26T06:45:25.542 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:25.542 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:45:06.881272+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:45:25.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8f80a1560 msgr2=0x7fe8f80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8f80a1560 0x7fe8f80a3a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe90819a470 tx=0x7fe8f40073d0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9081008a0 msgr2=0x7fe908198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9081008a0 0x7fe908198f10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e970 tx=0x7fe8fc00ec80 comp rx=0 tx=0).stop 2023-10-26T06:45:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 shutdown_connections 2023-10-26T06:45:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8f80a1560 0x7fe8f80a3a20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9081008a0 0x7fe908198f10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9081057c0 0x7fe90819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 --2- 172.21.15.89:0/2704693304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe908101260 0x7fe908199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.542+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 >> 172.21.15.89:0/2704693304 conn(0x7fe9080779e0 msgr2=0x7fe90810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.543+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 shutdown_connections 2023-10-26T06:45:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:25.543+0000 7fe90eea9700 1 -- 172.21.15.89:0/2704693304 wait complete. 2023-10-26T06:45:25.951 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-26T06:45:25.952 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd dump --format=json 2023-10-26T06:45:26.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:26 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2704693304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:26.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:26 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2704693304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:26.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:26 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2704693304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:27.261 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:27 smithi089 ceph-mon[95031]: pgmap v264: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2023-10-26T06:45:27.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:27 smithi192 ceph-mon[131661]: pgmap v264: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2023-10-26T06:45:27.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:27 smithi135 ceph-mon[131403]: pgmap v264: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2023-10-26T06:45:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.162+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/777966962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381057c0 msgr2=0x7f7e38105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.162+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/777966962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381057c0 0x7f7e38105ba0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009990 tx=0x7f7e34009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.163+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/777966962 shutdown_connections 2023-10-26T06:45:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.163+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/777966962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381057c0 0x7f7e38105ba0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.163+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/777966962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e38101260 0x7f7e3810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.163+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/777966962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e381008a0 0x7f7e38100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.163+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/777966962 >> 172.21.15.89:0/777966962 conn(0x7f7e380779e0 msgr2=0x7f7e38077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:28.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.163+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/777966962 shutdown_connections 2023-10-26T06:45:28.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.163+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/777966962 wait complete. 2023-10-26T06:45:28.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 Processor -- start 2023-10-26T06:45:28.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 -- start start 2023-10-26T06:45:28.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381008a0 0x7f7e38072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:28.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e38101260 0x7f7e38073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:28.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e381057c0 0x7f7e38073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:28.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e381a4fd0 con 0x7f7e381008a0 2023-10-26T06:45:28.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e381a5140 con 0x7f7e381057c0 2023-10-26T06:45:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3f3fe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e381a5300 con 0x7f7e38101260 2023-10-26T06:45:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.164+0000 7f7e3d19a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381008a0 0x7f7e38072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381008a0 0x7f7e38072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50880/0 (socket says 172.21.15.89:50880) 2023-10-26T06:45:28.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 -- 172.21.15.89:0/3104121565 learned_addr learned my addr 172.21.15.89:0/3104121565 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3c999700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e38101260 0x7f7e38073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d99b700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e381057c0 0x7f7e38073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 -- 172.21.15.89:0/3104121565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e38101260 msgr2=0x7f7e38073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e38101260 0x7f7e38073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 -- 172.21.15.89:0/3104121565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e381057c0 msgr2=0x7f7e38073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e381057c0 0x7f7e38073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 -- 172.21.15.89:0/3104121565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e340095f0 con 0x7f7e381008a0 2023-10-26T06:45:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d99b700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e381057c0 0x7f7e38073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3d19a700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381008a0 0x7f7e38072af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e34000c00 tx=0x7f7e34009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:28.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e2e7fc700 1 -- 172.21.15.89:0/3104121565 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3401d070 con 0x7f7e381008a0 2023-10-26T06:45:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e2e7fc700 1 -- 172.21.15.89:0/3104121565 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e34021410 con 0x7f7e381008a0 2023-10-26T06:45:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e381a5580 con 0x7f7e381008a0 2023-10-26T06:45:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.165+0000 7f7e2e7fc700 1 -- 172.21.15.89:0/3104121565 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34017650 con 0x7f7e381008a0 2023-10-26T06:45:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.166+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e381a5990 con 0x7f7e381008a0 2023-10-26T06:45:28.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.167+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e38108d80 con 0x7f7e381008a0 2023-10-26T06:45:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.169+0000 7f7e2e7fc700 1 -- 172.21.15.89:0/3104121565 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f7e34021580 con 0x7f7e381008a0 2023-10-26T06:45:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.170+0000 7f7e2e7fc700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e240a1560 0x7f7e240a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.170+0000 7f7e2e7fc700 1 -- 172.21.15.89:0/3104121565 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f7e340ce260 con 0x7f7e381008a0 2023-10-26T06:45:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.170+0000 7f7e3c999700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e240a1560 0x7f7e240a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.171+0000 7f7e3c999700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e240a1560 0x7f7e240a3a20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e38074220 tx=0x7f7e2800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.174+0000 7f7e2e7fc700 1 -- 172.21.15.89:0/3104121565 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e340977b0 con 0x7f7e381008a0 2023-10-26T06:45:28.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.375+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e38073fd0 con 0x7f7e381008a0 2023-10-26T06:45:28.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.376+0000 7f7e2e7fc700 1 -- 172.21.15.89:0/3104121565 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22407 (secure 0 0 0) 0x7f7e3409add0 con 0x7f7e381008a0 2023-10-26T06:45:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:28.378 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":84,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:45:06.881272+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:45:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e240a1560 msgr2=0x7f7e240a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:28.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e240a1560 0x7f7e240a3a20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e38074220 tx=0x7f7e2800b410 comp rx=0 tx=0).stop 2023-10-26T06:45:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381008a0 msgr2=0x7f7e38072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381008a0 0x7f7e38072af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e34000c00 tx=0x7f7e34009580 comp rx=0 tx=0).stop 2023-10-26T06:45:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 shutdown_connections 2023-10-26T06:45:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e240a1560 0x7f7e240a3a20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e381008a0 0x7f7e38072af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e381057c0 0x7f7e38073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 --2- 172.21.15.89:0/3104121565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e38101260 0x7f7e38073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 >> 172.21.15.89:0/3104121565 conn(0x7f7e380779e0 msgr2=0x7f7e380fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:28.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 shutdown_connections 2023-10-26T06:45:28.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:28.379+0000 7f7e3f3fe700 1 -- 172.21.15.89:0/3104121565 wait complete. 2023-10-26T06:45:28.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-26T06:45:28.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-26T06:45:28.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-26T06:45:28.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-26T06:45:28.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-26T06:45:28.870 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-26T06:45:28.870 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-26T06:45:28.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-26T06:45:28.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-26T06:45:28.872 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-26T06:45:28.873 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-26T06:45:28.873 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-26T06:45:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:29 smithi192 ceph-mon[131661]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2023-10-26T06:45:29.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:29 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3104121565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:29 smithi135 ceph-mon[131403]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2023-10-26T06:45:29.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:29 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3104121565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:29.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:29 smithi089 ceph-mon[95031]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2023-10-26T06:45:29.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:29 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3104121565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:45:30.514 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:30.663 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:31.108 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:31.109 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:31 smithi192 ceph-mon[131661]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-10-26T06:45:31.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:31 smithi135 ceph-mon[131403]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-10-26T06:45:31.508 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:31 smithi089 ceph-mon[95031]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-10-26T06:45:31.965 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:31.966 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:32.690 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:33.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 -- 172.21.15.89:0/602788491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 msgr2=0x7ff2b4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:33.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 --2- 172.21.15.89:0/602788491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 0x7ff2b4100d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40098f0 tx=0x7ff2a4009c00 comp rx=0 tx=0).stop 2023-10-26T06:45:33.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 -- 172.21.15.89:0/602788491 shutdown_connections 2023-10-26T06:45:33.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 --2- 172.21.15.89:0/602788491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b4101260 0x7ff2b410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:33.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 --2- 172.21.15.89:0/602788491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2b41057c0 0x7ff2b4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:33.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 --2- 172.21.15.89:0/602788491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 0x7ff2b4100d20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:33.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 -- 172.21.15.89:0/602788491 >> 172.21.15.89:0/602788491 conn(0x7ff2b40779e0 msgr2=0x7ff2b4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:33.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 -- 172.21.15.89:0/602788491 shutdown_connections 2023-10-26T06:45:33.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.180+0000 7ff2bb90f700 1 -- 172.21.15.89:0/602788491 wait complete. 2023-10-26T06:45:33.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.181+0000 7ff2bb90f700 1 Processor -- start 2023-10-26T06:45:33.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2bb90f700 1 -- start start 2023-10-26T06:45:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2bb90f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 0x7ff2b419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2bb90f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2b4101260 0x7ff2b419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2bb90f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b41057c0 0x7ff2b4196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:33.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2bb90f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4197120 con 0x7ff2b41057c0 2023-10-26T06:45:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2bb90f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4197290 con 0x7ff2b4101260 2023-10-26T06:45:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2bb90f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4197450 con 0x7ff2b41008a0 2023-10-26T06:45:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2b8eaa700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2b4101260 0x7ff2b419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.183+0000 7ff2b96ab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 0x7ff2b419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b9eac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b41057c0 0x7ff2b4196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:33.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b8eaa700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2b4101260 0x7ff2b419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:49834/0 (socket says 172.21.15.89:49834) 2023-10-26T06:45:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b8eaa700 1 -- 172.21.15.89:0/289145151 learned_addr learned my addr 172.21.15.89:0/289145151 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b96ab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 0x7ff2b419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57526/0 (socket says 172.21.15.89:57526) 2023-10-26T06:45:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b96ab700 1 -- 172.21.15.89:0/289145151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2b4101260 msgr2=0x7ff2b419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b96ab700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2b4101260 0x7ff2b419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:33.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b96ab700 1 -- 172.21.15.89:0/289145151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b41057c0 msgr2=0x7ff2b4196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:33.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b96ab700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b41057c0 0x7ff2b4196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:33.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b96ab700 1 -- 172.21.15.89:0/289145151 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a4009520 con 0x7ff2b41008a0 2023-10-26T06:45:33.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.184+0000 7ff2b8eaa700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2b4101260 0x7ff2b419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:33.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.185+0000 7ff2b96ab700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 0x7ff2b419c980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000e9c0 tx=0x7ff2b000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:33.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.185+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004220 con 0x7ff2b41008a0 2023-10-26T06:45:33.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.185+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b4197730 con 0x7ff2b41008a0 2023-10-26T06:45:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.185+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b0004380 con 0x7ff2b41008a0 2023-10-26T06:45:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.185+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0003be0 con 0x7ff2b41008a0 2023-10-26T06:45:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.185+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b41a3970 con 0x7ff2b41008a0 2023-10-26T06:45:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.186+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2b4110a60 con 0x7ff2b41008a0 2023-10-26T06:45:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.190+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ff2b0003d40 con 0x7ff2b41008a0 2023-10-26T06:45:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.191+0000 7ff2aa7fc700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff2a00a1560 0x7ff2a00a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.192+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7ff2b0014070 con 0x7ff2b41008a0 2023-10-26T06:45:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.192+0000 7ff2aa7fc700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] conn(0x7ff2a00aaed0 0x7ff2a00ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.192+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 --> [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2a00ad9a0 con 0x7ff2a00aaed0 2023-10-26T06:45:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.192+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b00cd4e0 con 0x7ff2b41008a0 2023-10-26T06:45:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.192+0000 7ff2b8eaa700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff2a00a1560 0x7ff2a00a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.192+0000 7ff2b9eac700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] conn(0x7ff2a00aaed0 0x7ff2a00ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.193+0000 7ff2b8eaa700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff2a00a1560 0x7ff2a00a3a20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4197c80 tx=0x7ff2a4019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.193+0000 7ff2b9eac700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] conn(0x7ff2a00aaed0 0x7ff2a00ad2f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.194+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== osd.2 v2:172.21.15.192:6800/2357909960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7ff2a00ad9a0 con 0x7ff2a00aaed0 2023-10-26T06:45:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.215+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 --> [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff2b4060e70 con 0x7ff2a00aaed0 2023-10-26T06:45:33.216 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:33 smithi089 ceph-mon[95031]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-10-26T06:45:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2aa7fc700 1 -- 172.21.15.89:0/289145151 <== osd.2 v2:172.21.15.192:6800/2357909960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff2b4060e70 con 0x7ff2a00aaed0 2023-10-26T06:45:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] conn(0x7ff2a00aaed0 msgr2=0x7ff2a00ad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2bb90f700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] conn(0x7ff2a00aaed0 0x7ff2a00ad2f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff2a00a1560 msgr2=0x7ff2a00a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2bb90f700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff2a00a1560 0x7ff2a00a3a20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4197c80 tx=0x7ff2a4019040 comp rx=0 tx=0).stop 2023-10-26T06:45:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 msgr2=0x7ff2b419c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2bb90f700 1 --2- 172.21.15.89:0/289145151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b41008a0 0x7ff2b419c980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000e9c0 tx=0x7ff2b000ecd0 comp rx=0 tx=0).stop 2023-10-26T06:45:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.216+0000 7ff2b9eac700 1 -- 172.21.15.89:0/289145151 reap_dead start 2023-10-26T06:45:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.218+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 shutdown_connections 2023-10-26T06:45:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.218+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 >> 172.21.15.89:0/289145151 conn(0x7ff2b40779e0 msgr2=0x7ff2b40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.218+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 shutdown_connections 2023-10-26T06:45:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:33.218+0000 7ff2bb90f700 1 -- 172.21.15.89:0/289145151 wait complete. 2023-10-26T06:45:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:33 smithi192 ceph-mon[131661]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-10-26T06:45:33.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:33 smithi135 ceph-mon[131403]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2023-10-26T06:45:34.655 INFO:teuthology.orchestra.run.smithi089.stdout:81604378683 2023-10-26T06:45:34.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-26T06:45:34.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 -- 172.21.15.89:0/4143553187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 msgr2=0x7f40bc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:34.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 --2- 172.21.15.89:0/4143553187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 0x7f40bc100d50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009990 tx=0x7f40b0009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 -- 172.21.15.89:0/4143553187 shutdown_connections 2023-10-26T06:45:34.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 --2- 172.21.15.89:0/4143553187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40bc1067d0 0x7f40bc106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 --2- 172.21.15.89:0/4143553187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40bc101290 0x7f40bc10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 --2- 172.21.15.89:0/4143553187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 0x7f40bc100d50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 -- 172.21.15.89:0/4143553187 >> 172.21.15.89:0/4143553187 conn(0x7f40bc0778d0 msgr2=0x7f40bc077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:34.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 -- 172.21.15.89:0/4143553187 shutdown_connections 2023-10-26T06:45:34.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.802+0000 7f40c0c1d700 1 -- 172.21.15.89:0/4143553187 wait complete. 2023-10-26T06:45:34.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.803+0000 7f40c0c1d700 1 Processor -- start 2023-10-26T06:45:34.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.803+0000 7f40c0c1d700 1 -- start start 2023-10-26T06:45:34.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40c0c1d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 0x7f40bc19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:34.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40c0c1d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40bc101290 0x7f40bc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:34.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40c0c1d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40bc1067d0 0x7f40bc196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:34.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40c0c1d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc197110 con 0x7f40bc101290 2023-10-26T06:45:34.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40c0c1d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc197280 con 0x7f40bc1067d0 2023-10-26T06:45:34.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40c0c1d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc197440 con 0x7f40bc1008d0 2023-10-26T06:45:34.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40ba59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 0x7f40bc19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:34.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.804+0000 7f40ba59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 0x7f40bc19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57538/0 (socket says 172.21.15.89:57538) 2023-10-26T06:45:34.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40ba59c700 1 -- 172.21.15.89:0/1301706900 learned_addr learned my addr 172.21.15.89:0/1301706900 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:34.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40b9d9b700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40bc101290 0x7f40bc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:34.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40bad9d700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40bc1067d0 0x7f40bc196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:34.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40b9d9b700 1 -- 172.21.15.89:0/1301706900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 msgr2=0x7f40bc19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:34.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40b9d9b700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40bc1008d0 0x7f40bc19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40b9d9b700 1 -- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40bc1067d0 msgr2=0x7f40bc196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40b9d9b700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40bc1067d0 0x7f40bc196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40b9d9b700 1 -- 172.21.15.89:0/1301706900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00095f0 con 0x7f40bc101290 2023-10-26T06:45:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.805+0000 7f40bad9d700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40bc1067d0 0x7f40bc196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:45:34.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.806+0000 7f40b9d9b700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40bc101290 0x7f40bc19ceb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000c00 tx=0x7f40b000bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.806+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b001d070 con 0x7f40bc101290 2023-10-26T06:45:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.806+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b00047e0 con 0x7f40bc101290 2023-10-26T06:45:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.806+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0005290 con 0x7f40bc101290 2023-10-26T06:45:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.806+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc1976c0 con 0x7f40bc101290 2023-10-26T06:45:34.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.808+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc072a20 con 0x7f40bc101290 2023-10-26T06:45:34.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.810+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40bc110a30 con 0x7f40bc101290 2023-10-26T06:45:34.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.811+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f40b0005470 con 0x7f40bc101290 2023-10-26T06:45:34.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.812+0000 7f40ab7fe700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f40a40a15e0 0x7f40a40a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:34.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.813+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f40b00cdf40 con 0x7f40bc101290 2023-10-26T06:45:34.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.813+0000 7f40ba59c700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f40a40a15e0 0x7f40a40a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:34.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.813+0000 7f40ab7fe700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] conn(0x7f40a40aaf50 0x7f40a40ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:34.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.813+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 --> [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40a40ada20 con 0x7f40a40aaf50 2023-10-26T06:45:34.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.813+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b00d05c0 con 0x7f40bc101290 2023-10-26T06:45:34.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.813+0000 7f40bad9d700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] conn(0x7f40a40aaf50 0x7f40a40ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:34.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.814+0000 7f40ba59c700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f40a40a15e0 0x7f40a40a3aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009c10 tx=0x7f40ac00b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.814+0000 7f40bad9d700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] conn(0x7f40a40aaf50 0x7f40a40ad370 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.815+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== osd.1 v2:172.21.15.135:6800/77037565 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f40a40ada20 con 0x7f40a40aaf50 2023-10-26T06:45:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.836+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 --> [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f40bc060e70 con 0x7f40a40aaf50 2023-10-26T06:45:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.837+0000 7f40ab7fe700 1 -- 172.21.15.89:0/1301706900 <== osd.1 v2:172.21.15.135:6800/77037565 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f40bc060e70 con 0x7f40a40aaf50 2023-10-26T06:45:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] conn(0x7f40a40aaf50 msgr2=0x7f40a40ad370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:34.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] conn(0x7f40a40aaf50 0x7f40a40ad370 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f40a40a15e0 msgr2=0x7f40a40a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f40a40a15e0 0x7f40a40a3aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009c10 tx=0x7f40ac00b040 comp rx=0 tx=0).stop 2023-10-26T06:45:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40bc101290 msgr2=0x7f40bc19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 --2- 172.21.15.89:0/1301706900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40bc101290 0x7f40bc19ceb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000c00 tx=0x7f40b000bfb0 comp rx=0 tx=0).stop 2023-10-26T06:45:34.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40bad9d700 1 -- 172.21.15.89:0/1301706900 reap_dead start 2023-10-26T06:45:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 shutdown_connections 2023-10-26T06:45:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 >> 172.21.15.89:0/1301706900 conn(0x7f40bc0778d0 msgr2=0x7f40bc0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 shutdown_connections 2023-10-26T06:45:34.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:34.838+0000 7f40c0c1d700 1 -- 172.21.15.89:0/1301706900 wait complete. 2023-10-26T06:45:35.333 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:35 smithi089 ceph-mon[95031]: pgmap v268: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-10-26T06:45:35.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:35 smithi192 ceph-mon[131661]: pgmap v268: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-10-26T06:45:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:35 smithi135 ceph-mon[131403]: pgmap v268: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2023-10-26T06:45:36.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.873+0000 7f6523e76700 1 -- 172.21.15.89:0/285764067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1057c0 msgr2=0x7f651c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.873+0000 7f6523e76700 1 --2- 172.21.15.89:0/285764067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1057c0 0x7f651c105ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6518009990 tx=0x7f6518009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.874+0000 7f6523e76700 1 -- 172.21.15.89:0/285764067 shutdown_connections 2023-10-26T06:45:36.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.874+0000 7f6523e76700 1 --2- 172.21.15.89:0/285764067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1057c0 0x7f651c105ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.874+0000 7f6523e76700 1 --2- 172.21.15.89:0/285764067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f651c101260 0x7f651c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.874+0000 7f6523e76700 1 --2- 172.21.15.89:0/285764067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f651c1008a0 0x7f651c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.874+0000 7f6523e76700 1 -- 172.21.15.89:0/285764067 >> 172.21.15.89:0/285764067 conn(0x7f651c0779e0 msgr2=0x7f651c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:36.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.874+0000 7f6523e76700 1 -- 172.21.15.89:0/285764067 shutdown_connections 2023-10-26T06:45:36.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.874+0000 7f6523e76700 1 -- 172.21.15.89:0/285764067 wait complete. 2023-10-26T06:45:36.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 Processor -- start 2023-10-26T06:45:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 -- start start 2023-10-26T06:45:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1008a0 0x7f651c198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f651c101260 0x7f651c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f651c1057c0 0x7f651c19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c19de10 con 0x7f651c1008a0 2023-10-26T06:45:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c19df80 con 0x7f651c1057c0 2023-10-26T06:45:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.875+0000 7f6523e76700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c19e140 con 0x7f651c101260 2023-10-26T06:45:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1008a0 0x7f651c198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6522413700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f651c1057c0 0x7f651c19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1008a0 0x7f651c198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46272/0 (socket says 172.21.15.89:46272) 2023-10-26T06:45:36.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 -- 172.21.15.89:0/2494716172 learned_addr learned my addr 172.21.15.89:0/2494716172 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521411700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f651c101260 0x7f651c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 -- 172.21.15.89:0/2494716172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f651c101260 msgr2=0x7f651c1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f651c101260 0x7f651c1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 -- 172.21.15.89:0/2494716172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f651c1057c0 msgr2=0x7f651c19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f651c1057c0 0x7f651c19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 -- 172.21.15.89:0/2494716172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180095f0 con 0x7f651c1008a0 2023-10-26T06:45:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.876+0000 7f6521c12700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1008a0 0x7f651c198f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6518000c00 tx=0x7f6518005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:36.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.877+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651801d070 con 0x7f651c1008a0 2023-10-26T06:45:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.877+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c19e3c0 con 0x7f651c1008a0 2023-10-26T06:45:36.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.877+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6518022470 con 0x7f651c1008a0 2023-10-26T06:45:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.877+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518017650 con 0x7f651c1008a0 2023-10-26T06:45:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.877+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c1a5c80 con 0x7f651c1008a0 2023-10-26T06:45:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.878+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f651c110a20 con 0x7f651c1008a0 2023-10-26T06:45:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.881+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f651800fad0 con 0x7f651c1008a0 2023-10-26T06:45:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.883+0000 7f6512ffd700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65080a1560 0x7f65080a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.883+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f65180cec90 con 0x7f651c1008a0 2023-10-26T06:45:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.883+0000 7f6521411700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65080a1560 0x7f65080a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.883+0000 7f6512ffd700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] conn(0x7f65080aaed0 0x7f65080ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.883+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 --> [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65080ad9a0 con 0x7f65080aaed0 2023-10-26T06:45:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.883+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65180cf000 con 0x7f651c1008a0 2023-10-26T06:45:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.884+0000 7f6522413700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] conn(0x7f65080aaed0 0x7f65080ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.884+0000 7f6521411700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65080a1560 0x7f65080a3a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f651c19a470 tx=0x7f650c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.884+0000 7f6522413700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] conn(0x7f65080aaed0 0x7f65080ad2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.885+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== osd.3 v2:172.21.15.89:6810/1365528390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f65080ad9a0 con 0x7f65080aaed0 2023-10-26T06:45:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.904+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 --> [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f651c101f30 con 0x7f65080aaed0 2023-10-26T06:45:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.907+0000 7f6512ffd700 1 -- 172.21.15.89:0/2494716172 <== osd.3 v2:172.21.15.89:6810/1365528390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f651c101f30 con 0x7f65080aaed0 2023-10-26T06:45:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.907+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] conn(0x7f65080aaed0 msgr2=0x7f65080ad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.907+0000 7f6523e76700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] conn(0x7f65080aaed0 0x7f65080ad2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.907+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65080a1560 msgr2=0x7f65080a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.907+0000 7f6523e76700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65080a1560 0x7f65080a3a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f651c19a470 tx=0x7f650c009260 comp rx=0 tx=0).stop 2023-10-26T06:45:36.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.908+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1008a0 msgr2=0x7f651c198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.908+0000 7f6523e76700 1 --2- 172.21.15.89:0/2494716172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f651c1008a0 0x7f651c198f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6518000c00 tx=0x7f6518005740 comp rx=0 tx=0).stop 2023-10-26T06:45:36.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.908+0000 7f6522413700 1 -- 172.21.15.89:0/2494716172 reap_dead start 2023-10-26T06:45:36.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.908+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 shutdown_connections 2023-10-26T06:45:36.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.908+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 >> 172.21.15.89:0/2494716172 conn(0x7f651c0779e0 msgr2=0x7f651c10bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:36.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.908+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 shutdown_connections 2023-10-26T06:45:36.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.908+0000 7f6523e76700 1 -- 172.21.15.89:0/2494716172 wait complete. 2023-10-26T06:45:36.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 -- 172.21.15.89:0/1125996698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5794106680 msgr2=0x7f5794106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 --2- 172.21.15.89:0/1125996698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5794106680 0x7f5794106a60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5790009990 tx=0x7f5790009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 -- 172.21.15.89:0/1125996698 shutdown_connections 2023-10-26T06:45:36.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 --2- 172.21.15.89:0/1125996698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5794106680 0x7f5794106a60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 --2- 172.21.15.89:0/1125996698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5794100fb0 0x7f579410d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 --2- 172.21.15.89:0/1125996698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57941005f0 0x7f5794100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 -- 172.21.15.89:0/1125996698 >> 172.21.15.89:0/1125996698 conn(0x7f57940fc120 msgr2=0x7f57940fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 -- 172.21.15.89:0/1125996698 shutdown_connections 2023-10-26T06:45:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.285+0000 7f579b38f700 1 -- 172.21.15.89:0/1125996698 wait complete. 2023-10-26T06:45:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.286+0000 7f579b38f700 1 Processor -- start 2023-10-26T06:45:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579b38f700 1 -- start start 2023-10-26T06:45:36.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579b38f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57941005f0 0x7f579419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579b38f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5794100fb0 0x7f579419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579b38f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5794106680 0x7f5794198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579b38f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941990b0 con 0x7f57941005f0 2023-10-26T06:45:36.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579b38f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5794199220 con 0x7f5794100fb0 2023-10-26T06:45:36.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579b38f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57941993e0 con 0x7f5794106680 2023-10-26T06:45:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579a38d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57941005f0 0x7f579419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579a38d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57941005f0 0x7f579419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46252/0 (socket says 172.21.15.89:46252) 2023-10-26T06:45:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579a38d700 1 -- 172.21.15.89:0/2061507731 learned_addr learned my addr 172.21.15.89:0/2061507731 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f579ab8e700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5794106680 0x7f5794198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.287+0000 7f5799b8c700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5794100fb0 0x7f579419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579a38d700 1 -- 172.21.15.89:0/2061507731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5794106680 msgr2=0x7f5794198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579a38d700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5794106680 0x7f5794198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579a38d700 1 -- 172.21.15.89:0/2061507731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5794100fb0 msgr2=0x7f579419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579a38d700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5794100fb0 0x7f579419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579a38d700 1 -- 172.21.15.89:0/2061507731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900095f0 con 0x7f57941005f0 2023-10-26T06:45:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579a38d700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57941005f0 0x7f579419e910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5790000c00 tx=0x7f5790005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579001d070 con 0x7f57941005f0 2023-10-26T06:45:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5790022470 con 0x7f57941005f0 2023-10-26T06:45:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5794199660 con 0x7f57941005f0 2023-10-26T06:45:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790017650 con 0x7f57941005f0 2023-10-26T06:45:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.288+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57941a5900 con 0x7f57941005f0 2023-10-26T06:45:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.289+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5794110830 con 0x7f57941005f0 2023-10-26T06:45:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.291+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f579000fad0 con 0x7f57941005f0 2023-10-26T06:45:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.292+0000 7f578b7fe700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f57800a1560 0x7f57800a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.293+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f57900cec90 con 0x7f57941005f0 2023-10-26T06:45:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.293+0000 7f5799b8c700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f57800a1560 0x7f57800a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.293+0000 7f578b7fe700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] conn(0x7f57800aaed0 0x7f57800ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.293+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 --> [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57800ad9a0 con 0x7f57800aaed0 2023-10-26T06:45:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.293+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57900cf000 con 0x7f57941005f0 2023-10-26T06:45:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.293+0000 7f579ab8e700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] conn(0x7f57800aaed0 0x7f57800ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.293+0000 7f5799b8c700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f57800a1560 0x7f57800a3a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5794199c10 tx=0x7f578400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:36.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.294+0000 7f579ab8e700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] conn(0x7f57800aaed0 0x7f57800ad2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:36.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.294+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== osd.0 v2:172.21.15.89:6802/2463649651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f57800ad9a0 con 0x7f57800aaed0 2023-10-26T06:45:36.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.311+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 --> [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57940fdf90 con 0x7f57800aaed0 2023-10-26T06:45:36.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.311+0000 7f578b7fe700 1 -- 172.21.15.89:0/2061507731 <== osd.0 v2:172.21.15.89:6802/2463649651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57940fdf90 con 0x7f57800aaed0 2023-10-26T06:45:36.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] conn(0x7f57800aaed0 msgr2=0x7f57800ad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] conn(0x7f57800aaed0 0x7f57800ad2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:36.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f57800a1560 msgr2=0x7f57800a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f57800a1560 0x7f57800a3a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5794199c10 tx=0x7f578400b410 comp rx=0 tx=0).stop 2023-10-26T06:45:36.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57941005f0 msgr2=0x7f579419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:36.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 --2- 172.21.15.89:0/2061507731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57941005f0 0x7f579419e910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5790000c00 tx=0x7f5790005740 comp rx=0 tx=0).stop 2023-10-26T06:45:36.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579ab8e700 1 -- 172.21.15.89:0/2061507731 reap_dead start 2023-10-26T06:45:36.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 shutdown_connections 2023-10-26T06:45:36.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 >> 172.21.15.89:0/2061507731 conn(0x7f57940fc120 msgr2=0x7f57940fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:36.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 shutdown_connections 2023-10-26T06:45:36.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:35.312+0000 7f579b38f700 1 -- 172.21.15.89:0/2061507731 wait complete. 2023-10-26T06:45:37.338 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:37 smithi089 ceph-mon[95031]: pgmap v269: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-10-26T06:45:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 -- 172.21.15.89:0/799543791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9860100fb0 msgr2=0x7f986010d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 --2- 172.21.15.89:0/799543791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9860100fb0 0x7f986010d4c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f985800b0a0 tx=0x7f985800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 -- 172.21.15.89:0/799543791 shutdown_connections 2023-10-26T06:45:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 --2- 172.21.15.89:0/799543791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9860100fb0 0x7f986010d4c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 --2- 172.21.15.89:0/799543791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9860106680 0x7f9860106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 --2- 172.21.15.89:0/799543791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98601005f0 0x7f9860100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 -- 172.21.15.89:0/799543791 >> 172.21.15.89:0/799543791 conn(0x7f98600fc120 msgr2=0x7f98600fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 -- 172.21.15.89:0/799543791 shutdown_connections 2023-10-26T06:45:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.011+0000 7f9865ddd700 1 -- 172.21.15.89:0/799543791 wait complete. 2023-10-26T06:45:37.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.012+0000 7f9865ddd700 1 Processor -- start 2023-10-26T06:45:37.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.012+0000 7f9865ddd700 1 -- start start 2023-10-26T06:45:37.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9865ddd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98601005f0 0x7f986019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9865ddd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9860100fb0 0x7f986019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9865ddd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9860106680 0x7f9860198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9865ddd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860199130 con 0x7f98601005f0 2023-10-26T06:45:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9865ddd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601992a0 con 0x7f9860100fb0 2023-10-26T06:45:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9865ddd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860199460 con 0x7f9860106680 2023-10-26T06:45:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98601005f0 0x7f986019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98601005f0 0x7f986019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46282/0 (socket says 172.21.15.89:46282) 2023-10-26T06:45:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 -- 172.21.15.89:0/4136520953 learned_addr learned my addr 172.21.15.89:0/4136520953 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f98655dc700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9860106680 0x7f9860198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f985ffff700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9860100fb0 0x7f986019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 -- 172.21.15.89:0/4136520953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9860106680 msgr2=0x7f9860198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9860106680 0x7f9860198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 -- 172.21.15.89:0/4136520953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9860100fb0 msgr2=0x7f986019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9860100fb0 0x7f986019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f9864ddb700 1 -- 172.21.15.89:0/4136520953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858009d00 con 0x7f98601005f0 2023-10-26T06:45:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.013+0000 7f985ffff700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9860100fb0 0x7f986019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.014+0000 7f9864ddb700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98601005f0 0x7f986019e990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f985400e970 tx=0x7f985400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:37.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.014+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854009790 con 0x7f98601005f0 2023-10-26T06:45:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.014+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f985400eec0 con 0x7f98601005f0 2023-10-26T06:45:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.014+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9860199740 con 0x7f98601005f0 2023-10-26T06:45:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.014+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854003cf0 con 0x7f98601005f0 2023-10-26T06:45:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.014+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601a5980 con 0x7f98601005f0 2023-10-26T06:45:37.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.015+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98601107f0 con 0x7f98601005f0 2023-10-26T06:45:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.017+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f985400ccd0 con 0x7f98601005f0 2023-10-26T06:45:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.018+0000 7f985dffb700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f98500a15e0 0x7f98500a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.018+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f9854014070 con 0x7f98601005f0 2023-10-26T06:45:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.018+0000 7f985ffff700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f98500a15e0 0x7f98500a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.019+0000 7f985dffb700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] conn(0x7f98500aaf50 0x7f98500ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.019+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 --> [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98500ada20 con 0x7f98500aaf50 2023-10-26T06:45:37.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.019+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98540cd5c0 con 0x7f98601005f0 2023-10-26T06:45:37.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.019+0000 7f98655dc700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] conn(0x7f98500aaf50 0x7f98500ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.019+0000 7f985ffff700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f98500a15e0 0x7f98500a3aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9860199c90 tx=0x7f984c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:37.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.019+0000 7f98655dc700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] conn(0x7f98500aaf50 0x7f98500ad370 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:37.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.020+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== osd.9 v2:172.21.15.89:6826/781872938 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f98500ada20 con 0x7f98500aaf50 2023-10-26T06:45:37.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.037+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 --> [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98600fdf50 con 0x7f98500aaf50 2023-10-26T06:45:37.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.037+0000 7f985dffb700 1 -- 172.21.15.89:0/4136520953 <== osd.9 v2:172.21.15.89:6826/781872938 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f98600fdf50 con 0x7f98500aaf50 2023-10-26T06:45:37.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] conn(0x7f98500aaf50 msgr2=0x7f98500ad370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] conn(0x7f98500aaf50 0x7f98500ad370 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f98500a15e0 msgr2=0x7f98500a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f98500a15e0 0x7f98500a3aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9860199c90 tx=0x7f984c009260 comp rx=0 tx=0).stop 2023-10-26T06:45:37.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98601005f0 msgr2=0x7f986019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 --2- 172.21.15.89:0/4136520953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98601005f0 0x7f986019e990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f985400e970 tx=0x7f985400ec80 comp rx=0 tx=0).stop 2023-10-26T06:45:37.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f98655dc700 1 -- 172.21.15.89:0/4136520953 reap_dead start 2023-10-26T06:45:37.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 shutdown_connections 2023-10-26T06:45:37.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 >> 172.21.15.89:0/4136520953 conn(0x7f98600fc120 msgr2=0x7f98600fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:37.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 shutdown_connections 2023-10-26T06:45:37.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.038+0000 7f9865ddd700 1 -- 172.21.15.89:0/4136520953 wait complete. 2023-10-26T06:45:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:37 smithi192 ceph-mon[131661]: pgmap v269: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-10-26T06:45:37.462 INFO:teuthology.orchestra.run.smithi089.stdout:111669149751 2023-10-26T06:45:37.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-26T06:45:37.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:37 smithi135 ceph-mon[131403]: pgmap v269: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-10-26T06:45:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 -- 172.21.15.89:0/3452702669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c101260 msgr2=0x7fdd2c10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 --2- 172.21.15.89:0/3452702669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c101260 0x7fdd2c10d650 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400b0a0 tx=0x7fdd2400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 -- 172.21.15.89:0/3452702669 shutdown_connections 2023-10-26T06:45:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 --2- 172.21.15.89:0/3452702669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c101260 0x7fdd2c10d650 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 --2- 172.21.15.89:0/3452702669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd2c1057c0 0x7fdd2c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 --2- 172.21.15.89:0/3452702669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd2c1008a0 0x7fdd2c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 -- 172.21.15.89:0/3452702669 >> 172.21.15.89:0/3452702669 conn(0x7fdd2c0779e0 msgr2=0x7fdd2c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.645+0000 7fdd3141f700 1 -- 172.21.15.89:0/3452702669 shutdown_connections 2023-10-26T06:45:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.646+0000 7fdd3141f700 1 -- 172.21.15.89:0/3452702669 wait complete. 2023-10-26T06:45:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.646+0000 7fdd3141f700 1 Processor -- start 2023-10-26T06:45:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd3141f700 1 -- start start 2023-10-26T06:45:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd3141f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c1008a0 0x7fdd2c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd3141f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd2c101260 0x7fdd2c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd3141f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd2c1057c0 0x7fdd2c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd3141f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c19bcb0 con 0x7fdd2c101260 2023-10-26T06:45:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd3141f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c19be20 con 0x7fdd2c1008a0 2023-10-26T06:45:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd3141f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c19bfe0 con 0x7fdd2c1057c0 2023-10-26T06:45:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd2a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd2c101260 0x7fdd2c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd2affd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c1008a0 0x7fdd2c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd2a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd2c101260 0x7fdd2c1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46292/0 (socket says 172.21.15.89:46292) 2023-10-26T06:45:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd2a7fc700 1 -- 172.21.15.89:0/685726601 learned_addr learned my addr 172.21.15.89:0/685726601 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.647+0000 7fdd2b7fe700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd2c1057c0 0x7fdd2c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2a7fc700 1 -- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd2c1057c0 msgr2=0x7fdd2c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2a7fc700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd2c1057c0 0x7fdd2c19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2a7fc700 1 -- 172.21.15.89:0/685726601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c1008a0 msgr2=0x7fdd2c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2a7fc700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c1008a0 0x7fdd2c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2a7fc700 1 -- 172.21.15.89:0/685726601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd20009520 con 0x7fdd2c101260 2023-10-26T06:45:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2b7fe700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd2c1057c0 0x7fdd2c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2affd700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd2c1008a0 0x7fdd2c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd2a7fc700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd2c101260 0x7fdd2c1972f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000eb30 tx=0x7fdd2000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20009880 con 0x7fdd2c101260 2023-10-26T06:45:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd200044e0 con 0x7fdd2c101260 2023-10-26T06:45:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd24009d00 con 0x7fdd2c101260 2023-10-26T06:45:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.648+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20003e90 con 0x7fdd2c101260 2023-10-26T06:45:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.649+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c072a20 con 0x7fdd2c101260 2023-10-26T06:45:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.650+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd2c110a40 con 0x7fdd2c101260 2023-10-26T06:45:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.652+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fdd20004000 con 0x7fdd2c101260 2023-10-26T06:45:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.654+0000 7fdd13fff700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd140a1560 0x7fdd140a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.654+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fdd20014070 con 0x7fdd2c101260 2023-10-26T06:45:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.654+0000 7fdd2affd700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd140a1560 0x7fdd140a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.655+0000 7fdd13fff700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] conn(0x7fdd140aad60 0x7fdd140ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.655+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 --> [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd140ad830 con 0x7fdd140aad60 2023-10-26T06:45:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.655+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd200cd820 con 0x7fdd2c101260 2023-10-26T06:45:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.655+0000 7fdd2affd700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd140a1560 0x7fdd140a3a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c101930 tx=0x7fdd1c012040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.655+0000 7fdd2b7fe700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] conn(0x7fdd140aad60 0x7fdd140ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:37.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.655+0000 7fdd2b7fe700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] conn(0x7fdd140aad60 0x7fdd140ad180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:37.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.656+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== osd.5 v2:172.21.15.192:6808/2176647372 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fdd140ad830 con 0x7fdd140aad60 2023-10-26T06:45:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.674+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 --> [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd2c060e70 con 0x7fdd140aad60 2023-10-26T06:45:37.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd13fff700 1 -- 172.21.15.89:0/685726601 <== osd.5 v2:172.21.15.192:6808/2176647372 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdd2c060e70 con 0x7fdd140aad60 2023-10-26T06:45:37.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] conn(0x7fdd140aad60 msgr2=0x7fdd140ad180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] conn(0x7fdd140aad60 0x7fdd140ad180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:37.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd140a1560 msgr2=0x7fdd140a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd140a1560 0x7fdd140a3a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c101930 tx=0x7fdd1c012040 comp rx=0 tx=0).stop 2023-10-26T06:45:37.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd2c101260 msgr2=0x7fdd2c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 --2- 172.21.15.89:0/685726601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd2c101260 0x7fdd2c1972f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000eb30 tx=0x7fdd2000ee40 comp rx=0 tx=0).stop 2023-10-26T06:45:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd2b7fe700 1 -- 172.21.15.89:0/685726601 reap_dead start 2023-10-26T06:45:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 shutdown_connections 2023-10-26T06:45:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.675+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 >> 172.21.15.89:0/685726601 conn(0x7fdd2c0779e0 msgr2=0x7fdd2c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.676+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 shutdown_connections 2023-10-26T06:45:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:37.676+0000 7fdd3141f700 1 -- 172.21.15.89:0/685726601 wait complete. 2023-10-26T06:45:38.446 INFO:teuthology.orchestra.run.smithi089.stdout:287762808859 2023-10-26T06:45:38.447 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-26T06:45:39.211 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:39 smithi089 ceph-mon[95031]: pgmap v270: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:39 smithi192 ceph-mon[131661]: pgmap v270: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:39 smithi135 ceph-mon[131403]: pgmap v270: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:39.611 INFO:teuthology.orchestra.run.smithi089.stdout:38654705733 2023-10-26T06:45:39.611 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-26T06:45:39.967 INFO:teuthology.orchestra.run.smithi089.stdout:167503724591 2023-10-26T06:45:39.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-26T06:45:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.072+0000 7fe3814d4700 1 -- 172.21.15.89:0/586748706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 msgr2=0x7fe37c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.072+0000 7fe3814d4700 1 --2- 172.21.15.89:0/586748706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 0x7fe37c100d20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe370009990 tx=0x7fe370009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.073+0000 7fe3814d4700 1 -- 172.21.15.89:0/586748706 shutdown_connections 2023-10-26T06:45:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.073+0000 7fe3814d4700 1 --2- 172.21.15.89:0/586748706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c101260 0x7fe37c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.073+0000 7fe3814d4700 1 --2- 172.21.15.89:0/586748706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe37c1057c0 0x7fe37c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.073+0000 7fe3814d4700 1 --2- 172.21.15.89:0/586748706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 0x7fe37c100d20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.073+0000 7fe3814d4700 1 -- 172.21.15.89:0/586748706 >> 172.21.15.89:0/586748706 conn(0x7fe37c0779e0 msgr2=0x7fe37c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.073+0000 7fe3814d4700 1 -- 172.21.15.89:0/586748706 shutdown_connections 2023-10-26T06:45:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.073+0000 7fe3814d4700 1 -- 172.21.15.89:0/586748706 wait complete. 2023-10-26T06:45:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.074+0000 7fe3814d4700 1 Processor -- start 2023-10-26T06:45:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.074+0000 7fe3814d4700 1 -- start start 2023-10-26T06:45:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe3814d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 0x7fe37c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe3814d4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c101260 0x7fe37c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe3814d4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe37c1057c0 0x7fe37c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe3814d4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1a4f40 con 0x7fe37c1008a0 2023-10-26T06:45:40.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe3814d4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1a50b0 con 0x7fe37c101260 2023-10-26T06:45:40.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe3814d4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1a5270 con 0x7fe37c1057c0 2023-10-26T06:45:40.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe37a7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c101260 0x7fe37c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe37affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 0x7fe37c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe37a7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c101260 0x7fe37c073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37358/0 (socket says 172.21.15.89:37358) 2023-10-26T06:45:40.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe37a7fc700 1 -- 172.21.15.89:0/476268146 learned_addr learned my addr 172.21.15.89:0/476268146 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:40.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.075+0000 7fe37b7fe700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe37c1057c0 0x7fe37c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe37a7fc700 1 -- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe37c1057c0 msgr2=0x7fe37c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe37a7fc700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe37c1057c0 0x7fe37c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe37a7fc700 1 -- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 msgr2=0x7fe37c072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe37a7fc700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 0x7fe37c072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe37a7fc700 1 -- 172.21.15.89:0/476268146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3700095f0 con 0x7fe37c101260 2023-10-26T06:45:40.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe37affd700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe37c1008a0 0x7fe37c072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:40.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe37a7fc700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c101260 0x7fe37c073030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe37000b2e0 tx=0x7fe37000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37001d070 con 0x7fe37c101260 2023-10-26T06:45:40.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c1a54f0 con 0x7fe37c101260 2023-10-26T06:45:40.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.076+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c1a5960 con 0x7fe37c101260 2023-10-26T06:45:40.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.077+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe370021410 con 0x7fe37c101260 2023-10-26T06:45:40.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.077+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370017650 con 0x7fe37c101260 2023-10-26T06:45:40.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.077+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe368000ff0 con 0x7fe37c101260 2023-10-26T06:45:40.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.080+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe37000fb10 con 0x7fe37c101260 2023-10-26T06:45:40.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.081+0000 7fe363fff700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe3640a15e0 0x7fe3640a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.082+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fe3700ce060 con 0x7fe37c101260 2023-10-26T06:45:40.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.082+0000 7fe37affd700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe3640a15e0 0x7fe3640a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.082+0000 7fe363fff700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] conn(0x7fe3640aaf50 0x7fe3640ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.082+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 --> [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3640ada20 con 0x7fe3640aaf50 2023-10-26T06:45:40.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.082+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3700ce3d0 con 0x7fe37c101260 2023-10-26T06:45:40.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.082+0000 7fe37affd700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe3640a15e0 0x7fe3640a3aa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe37c101930 tx=0x7fe36c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.082+0000 7fe37b7fe700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] conn(0x7fe3640aaf50 0x7fe3640ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.083+0000 7fe37b7fe700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] conn(0x7fe3640aaf50 0x7fe3640ad370 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.084+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== osd.8 v2:172.21.15.192:6816/87624122 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe3640ada20 con 0x7fe3640aaf50 2023-10-26T06:45:40.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.100+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 --> [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe368002d70 con 0x7fe3640aaf50 2023-10-26T06:45:40.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.101+0000 7fe363fff700 1 -- 172.21.15.89:0/476268146 <== osd.8 v2:172.21.15.192:6816/87624122 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe368002d70 con 0x7fe3640aaf50 2023-10-26T06:45:40.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.101+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] conn(0x7fe3640aaf50 msgr2=0x7fe3640ad370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.101+0000 7fe3814d4700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] conn(0x7fe3640aaf50 0x7fe3640ad370 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe3640a15e0 msgr2=0x7fe3640a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe3640a15e0 0x7fe3640a3aa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe37c101930 tx=0x7fe36c009260 comp rx=0 tx=0).stop 2023-10-26T06:45:40.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c101260 msgr2=0x7fe37c073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 --2- 172.21.15.89:0/476268146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe37c101260 0x7fe37c073030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe37000b2e0 tx=0x7fe37000bfd0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe37b7fe700 1 -- 172.21.15.89:0/476268146 reap_dead start 2023-10-26T06:45:40.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 shutdown_connections 2023-10-26T06:45:40.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 >> 172.21.15.89:0/476268146 conn(0x7fe37c0779e0 msgr2=0x7fe37c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:40.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 shutdown_connections 2023-10-26T06:45:40.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.102+0000 7fe3814d4700 1 -- 172.21.15.89:0/476268146 wait complete. 2023-10-26T06:45:40.333 INFO:teuthology.orchestra.run.smithi089.stdout:60129542208 2023-10-26T06:45:40.334 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-26T06:45:40.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 -- 172.21.15.89:0/2910394359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de81057c0 msgr2=0x7f0de8105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 --2- 172.21.15.89:0/2910394359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de81057c0 0x7f0de8105ba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009990 tx=0x7f0de4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 -- 172.21.15.89:0/2910394359 shutdown_connections 2023-10-26T06:45:40.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 --2- 172.21.15.89:0/2910394359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de8101260 0x7f0de810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 --2- 172.21.15.89:0/2910394359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de81057c0 0x7f0de8105ba0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 --2- 172.21.15.89:0/2910394359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0de81008a0 0x7f0de8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 -- 172.21.15.89:0/2910394359 >> 172.21.15.89:0/2910394359 conn(0x7f0de80779e0 msgr2=0x7f0de8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:40.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 -- 172.21.15.89:0/2910394359 shutdown_connections 2023-10-26T06:45:40.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.333+0000 7f0df0321700 1 -- 172.21.15.89:0/2910394359 wait complete. 2023-10-26T06:45:40.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.334+0000 7f0df0321700 1 Processor -- start 2023-10-26T06:45:40.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0df0321700 1 -- start start 2023-10-26T06:45:40.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0df0321700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de81008a0 0x7f0de819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0df0321700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0de8101260 0x7f0de819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0df0321700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de81057c0 0x7f0de8196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0df0321700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81971a0 con 0x7f0de81057c0 2023-10-26T06:45:40.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0df0321700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81972e0 con 0x7f0de8101260 2023-10-26T06:45:40.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0df0321700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81974a0 con 0x7f0de81008a0 2023-10-26T06:45:40.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0ded8bc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0de8101260 0x7f0de819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0dee8be700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de81057c0 0x7f0de8196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0ded8bc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0de8101260 0x7f0de819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37376/0 (socket says 172.21.15.89:37376) 2023-10-26T06:45:40.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.335+0000 7f0ded8bc700 1 -- 172.21.15.89:0/123619795 learned_addr learned my addr 172.21.15.89:0/123619795 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:40.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0dee0bd700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de81008a0 0x7f0de819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ded8bc700 1 -- 172.21.15.89:0/123619795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de81008a0 msgr2=0x7f0de819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ded8bc700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de81008a0 0x7f0de819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ded8bc700 1 -- 172.21.15.89:0/123619795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de81057c0 msgr2=0x7f0de8196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ded8bc700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0de81057c0 0x7f0de8196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ded8bc700 1 -- 172.21.15.89:0/123619795 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40095f0 con 0x7f0de8101260 2023-10-26T06:45:40.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ded8bc700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0de8101260 0x7f0de819cee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800e9a0 tx=0x7f0dd800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8009790 con 0x7f0de8101260 2023-10-26T06:45:40.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de8197780 con 0x7f0de8101260 2023-10-26T06:45:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dd8010430 con 0x7f0de8101260 2023-10-26T06:45:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.336+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8018680 con 0x7f0de8101260 2023-10-26T06:45:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.337+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de81a3a40 con 0x7f0de8101260 2023-10-26T06:45:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.338+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0de8110a20 con 0x7f0de8101260 2023-10-26T06:45:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.341+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f0dd80105a0 con 0x7f0de8101260 2023-10-26T06:45:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.342+0000 7f0ddf7fe700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0dd40a1560 0x7f0dd40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.342+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f0dd8014070 con 0x7f0de8101260 2023-10-26T06:45:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.343+0000 7f0ddf7fe700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] conn(0x7f0dd40aaed0 0x7f0dd40ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.343+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 --> [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0dd40ad9a0 con 0x7f0dd40aaed0 2023-10-26T06:45:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.343+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd80cd270 con 0x7f0de8101260 2023-10-26T06:45:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.343+0000 7f0dee0bd700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0dd40a1560 0x7f0dd40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.343+0000 7f0dee8be700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] conn(0x7f0dd40aaed0 0x7f0dd40ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.344+0000 7f0dee8be700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] conn(0x7f0dd40aaed0 0x7f0dd40ad2f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.344+0000 7f0dee0bd700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0dd40a1560 0x7f0dd40a3a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0de400b2e0 tx=0x7f0de4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.345+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== osd.10 v2:172.21.15.135:6824/3703256177 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0dd40ad9a0 con 0x7f0dd40aaed0 2023-10-26T06:45:40.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.377+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 --> [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0de8060e70 con 0x7f0dd40aaed0 2023-10-26T06:45:40.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.377+0000 7f0ddf7fe700 1 -- 172.21.15.89:0/123619795 <== osd.10 v2:172.21.15.135:6824/3703256177 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0de8060e70 con 0x7f0dd40aaed0 2023-10-26T06:45:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] conn(0x7f0dd40aaed0 msgr2=0x7f0dd40ad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] conn(0x7f0dd40aaed0 0x7f0dd40ad2f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0dd40a1560 msgr2=0x7f0dd40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0dd40a1560 0x7f0dd40a3a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0de400b2e0 tx=0x7f0de4005fd0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0de8101260 msgr2=0x7f0de819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 --2- 172.21.15.89:0/123619795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0de8101260 0x7f0de819cee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800e9a0 tx=0x7f0dd800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0dee8be700 1 -- 172.21.15.89:0/123619795 reap_dead start 2023-10-26T06:45:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 shutdown_connections 2023-10-26T06:45:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.378+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 >> 172.21.15.89:0/123619795 conn(0x7f0de80779e0 msgr2=0x7f0de80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:40.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.379+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 shutdown_connections 2023-10-26T06:45:40.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.379+0000 7f0df0321700 1 -- 172.21.15.89:0/123619795 wait complete. 2023-10-26T06:45:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 -- 172.21.15.89:0/2905172580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90301057c0 msgr2=0x7f9030105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 --2- 172.21.15.89:0/2905172580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90301057c0 0x7f9030105ba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9020009990 tx=0x7f9020009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 -- 172.21.15.89:0/2905172580 shutdown_connections 2023-10-26T06:45:40.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 --2- 172.21.15.89:0/2905172580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90301057c0 0x7f9030105ba0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 --2- 172.21.15.89:0/2905172580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9030101260 0x7f903010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 --2- 172.21.15.89:0/2905172580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90301008a0 0x7f9030100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 -- 172.21.15.89:0/2905172580 >> 172.21.15.89:0/2905172580 conn(0x7f90300779e0 msgr2=0x7f9030077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 -- 172.21.15.89:0/2905172580 shutdown_connections 2023-10-26T06:45:40.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.682+0000 7f9035972700 1 -- 172.21.15.89:0/2905172580 wait complete. 2023-10-26T06:45:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.683+0000 7f9035972700 1 Processor -- start 2023-10-26T06:45:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.683+0000 7f9035972700 1 -- start start 2023-10-26T06:45:40.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f9035972700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90301008a0 0x7f9030198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f9035972700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9030101260 0x7f9030199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f9035972700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90301057c0 0x7f903019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f9035972700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903019de00 con 0x7f9030101260 2023-10-26T06:45:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f9035972700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903019df70 con 0x7f90301057c0 2023-10-26T06:45:40.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f9035972700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903019e130 con 0x7f90301008a0 2023-10-26T06:45:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f902effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90301008a0 0x7f9030198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f902e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9030101260 0x7f9030199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f902effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90301008a0 0x7f9030198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50804/0 (socket says 172.21.15.89:50804) 2023-10-26T06:45:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f902e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9030101260 0x7f9030199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38442/0 (socket says 172.21.15.89:38442) 2023-10-26T06:45:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f902effd700 1 -- 172.21.15.89:0/2739785585 learned_addr learned my addr 172.21.15.89:0/2739785585 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.684+0000 7f902f7fe700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90301057c0 0x7f903019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902f7fe700 1 -- 172.21.15.89:0/2739785585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90301008a0 msgr2=0x7f9030198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902f7fe700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90301008a0 0x7f9030198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902f7fe700 1 -- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9030101260 msgr2=0x7f9030199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902f7fe700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9030101260 0x7f9030199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902f7fe700 1 -- 172.21.15.89:0/2739785585 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90200095f0 con 0x7f90301057c0 2023-10-26T06:45:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902effd700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90301008a0 0x7f9030198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902e7fc700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9030101260 0x7f9030199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f902f7fe700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90301057c0 0x7f903019d7f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9028007f10 tx=0x7f902800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028004db0 con 0x7f90301057c0 2023-10-26T06:45:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903019e410 con 0x7f90301057c0 2023-10-26T06:45:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f902800f040 con 0x7f90301057c0 2023-10-26T06:45:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028005830 con 0x7f90301057c0 2023-10-26T06:45:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.685+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90301a5c70 con 0x7f90301057c0 2023-10-26T06:45:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.687+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90301109e0 con 0x7f90301057c0 2023-10-26T06:45:40.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.689+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f902800db20 con 0x7f90301057c0 2023-10-26T06:45:40.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.690+0000 7f9034970700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90180a15e0 0x7f90180a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.690+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f90280cc550 con 0x7f90301057c0 2023-10-26T06:45:40.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.691+0000 7f902effd700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90180a15e0 0x7f90180a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.691+0000 7f9034970700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] conn(0x7f90180aaf50 0x7f90180ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:40.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.691+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 --> [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90180ada20 con 0x7f90180aaf50 2023-10-26T06:45:40.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.691+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90280ceba0 con 0x7f90301057c0 2023-10-26T06:45:40.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.691+0000 7f902e7fc700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] conn(0x7f90180aaf50 0x7f90180ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:40.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.692+0000 7f902effd700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90180a15e0 0x7f90180a3aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9020006010 tx=0x7f902000b560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.692+0000 7f902e7fc700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] conn(0x7f90180aaf50 0x7f90180ad370 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:40.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.693+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== osd.6 v2:172.21.15.89:6818/2393166048 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f90180ada20 con 0x7f90180aaf50 2023-10-26T06:45:40.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.713+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 --> [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9030101f30 con 0x7f90180aaf50 2023-10-26T06:45:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.715+0000 7f9034970700 1 -- 172.21.15.89:0/2739785585 <== osd.6 v2:172.21.15.89:6818/2393166048 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9030101f30 con 0x7f90180aaf50 2023-10-26T06:45:40.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] conn(0x7f90180aaf50 msgr2=0x7f90180ad370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] conn(0x7f90180aaf50 0x7f90180ad370 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90180a15e0 msgr2=0x7f90180a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90180a15e0 0x7f90180a3aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9020006010 tx=0x7f902000b560 comp rx=0 tx=0).stop 2023-10-26T06:45:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90301057c0 msgr2=0x7f903019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 --2- 172.21.15.89:0/2739785585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90301057c0 0x7f903019d7f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9028007f10 tx=0x7f902800d3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f902f7fe700 1 -- 172.21.15.89:0/2739785585 reap_dead start 2023-10-26T06:45:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 shutdown_connections 2023-10-26T06:45:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.716+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 >> 172.21.15.89:0/2739785585 conn(0x7f90300779e0 msgr2=0x7f903010ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.717+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 shutdown_connections 2023-10-26T06:45:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:40.717+0000 7f9035972700 1 -- 172.21.15.89:0/2739785585 wait complete. 2023-10-26T06:45:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 -- 172.21.15.89:0/1605390999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44111570 msgr2=0x7f1e441139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 --2- 172.21.15.89:0/1605390999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44111570 0x7f1e441139a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00b0a0 tx=0x7f1e3c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 -- 172.21.15.89:0/1605390999 shutdown_connections 2023-10-26T06:45:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 --2- 172.21.15.89:0/1605390999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44111570 0x7f1e441139a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 --2- 172.21.15.89:0/1605390999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e44074770 0x7f1e44072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 --2- 172.21.15.89:0/1605390999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e44072f60 0x7f1e440733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 -- 172.21.15.89:0/1605390999 >> 172.21.15.89:0/1605390999 conn(0x7f1e440779e0 msgr2=0x7f1e44077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 -- 172.21.15.89:0/1605390999 shutdown_connections 2023-10-26T06:45:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.231+0000 7f1e48ed8700 1 -- 172.21.15.89:0/1605390999 wait complete. 2023-10-26T06:45:41.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.232+0000 7f1e48ed8700 1 Processor -- start 2023-10-26T06:45:41.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.232+0000 7f1e48ed8700 1 -- start start 2023-10-26T06:45:41.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e48ed8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e44072f60 0x7f1e4419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e48ed8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44074770 0x7f1e4419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e48ed8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e44111570 0x7f1e441a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e48ed8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441a22b0 con 0x7f1e44074770 2023-10-26T06:45:41.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e48ed8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441a2420 con 0x7f1e44111570 2023-10-26T06:45:41.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e48ed8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441a25e0 con 0x7f1e44072f60 2023-10-26T06:45:41.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e41d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44074770 0x7f1e4419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e41d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44074770 0x7f1e4419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38456/0 (socket says 172.21.15.89:38456) 2023-10-26T06:45:41.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e41d9b700 1 -- 172.21.15.89:0/3156643676 learned_addr learned my addr 172.21.15.89:0/3156643676 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:41.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e4259c700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e44072f60 0x7f1e4419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.233+0000 7f1e42d9d700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e44111570 0x7f1e441a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e4259c700 1 -- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e44111570 msgr2=0x7f1e441a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e4259c700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e44111570 0x7f1e441a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e4259c700 1 -- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44074770 msgr2=0x7f1e4419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e4259c700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44074770 0x7f1e4419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e4259c700 1 -- 172.21.15.89:0/3156643676 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38009520 con 0x7f1e44072f60 2023-10-26T06:45:41.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e41d9b700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e44074770 0x7f1e4419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:41.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e4259c700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e44072f60 0x7f1e4419d400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400e9f0 tx=0x7f1e3400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:41.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400cb70 con 0x7f1e44072f60 2023-10-26T06:45:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e3400ee90 con 0x7f1e44072f60 2023-10-26T06:45:41.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e34009520 con 0x7f1e44072f60 2023-10-26T06:45:41.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.234+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34003cf0 con 0x7f1e44072f60 2023-10-26T06:45:41.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.235+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c009d00 con 0x7f1e44072f60 2023-10-26T06:45:41.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.235+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e440740b0 con 0x7f1e44072f60 2023-10-26T06:45:41.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.237+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1e34004020 con 0x7f1e44072f60 2023-10-26T06:45:41.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.239+0000 7f1e337fe700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1e2c0a1560 0x7f1e2c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.239+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f1e34014070 con 0x7f1e44072f60 2023-10-26T06:45:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.239+0000 7f1e41d9b700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1e2c0a1560 0x7f1e2c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.239+0000 7f1e337fe700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] conn(0x7f1e2c0aaed0 0x7f1e2c0ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.239+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 --> [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e2c0ad9a0 con 0x7f1e2c0aaed0 2023-10-26T06:45:41.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.239+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e340cd810 con 0x7f1e44072f60 2023-10-26T06:45:41.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.239+0000 7f1e42d9d700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] conn(0x7f1e2c0aaed0 0x7f1e2c0ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.240+0000 7f1e41d9b700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1e2c0a1560 0x7f1e2c0a3a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009fd0 tx=0x7f1e38009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:41.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.240+0000 7f1e42d9d700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] conn(0x7f1e2c0aaed0 0x7f1e2c0ad2f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:41.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.241+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== osd.7 v2:172.21.15.135:6816/1440944869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f1e2c0ad9a0 con 0x7f1e2c0aaed0 2023-10-26T06:45:41.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.258+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 --> [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e44117090 con 0x7f1e2c0aaed0 2023-10-26T06:45:41.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.258+0000 7f1e337fe700 1 -- 172.21.15.89:0/3156643676 <== osd.7 v2:172.21.15.135:6816/1440944869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1e44117090 con 0x7f1e2c0aaed0 2023-10-26T06:45:41.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] conn(0x7f1e2c0aaed0 msgr2=0x7f1e2c0ad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] conn(0x7f1e2c0aaed0 0x7f1e2c0ad2f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1e2c0a1560 msgr2=0x7f1e2c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1e2c0a1560 0x7f1e2c0a3a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009fd0 tx=0x7f1e38009260 comp rx=0 tx=0).stop 2023-10-26T06:45:41.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e44072f60 msgr2=0x7f1e4419d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 --2- 172.21.15.89:0/3156643676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e44072f60 0x7f1e4419d400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400e9f0 tx=0x7f1e3400ed00 comp rx=0 tx=0).stop 2023-10-26T06:45:41.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e42d9d700 1 -- 172.21.15.89:0/3156643676 reap_dead start 2023-10-26T06:45:41.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 shutdown_connections 2023-10-26T06:45:41.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 >> 172.21.15.89:0/3156643676 conn(0x7f1e440779e0 msgr2=0x7f1e44110140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:41.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 shutdown_connections 2023-10-26T06:45:41.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.259+0000 7f1e48ed8700 1 -- 172.21.15.89:0/3156643676 wait complete. 2023-10-26T06:45:41.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 -- 172.21.15.89:0/3192927323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d501029c0 msgr2=0x7f6d50102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 --2- 172.21.15.89:0/3192927323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d501029c0 0x7f6d50102e40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009990 tx=0x7f6d44009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 -- 172.21.15.89:0/3192927323 shutdown_connections 2023-10-26T06:45:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 --2- 172.21.15.89:0/3192927323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d501089c0 0x7f6d50108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 --2- 172.21.15.89:0/3192927323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d50103380 0x7f6d5010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 --2- 172.21.15.89:0/3192927323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d501029c0 0x7f6d50102e40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 -- 172.21.15.89:0/3192927323 >> 172.21.15.89:0/3192927323 conn(0x7f6d500fe4b0 msgr2=0x7f6d501008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:41.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 -- 172.21.15.89:0/3192927323 shutdown_connections 2023-10-26T06:45:41.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.326+0000 7f6d55e33700 1 -- 172.21.15.89:0/3192927323 wait complete. 2023-10-26T06:45:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.327+0000 7f6d55e33700 1 Processor -- start 2023-10-26T06:45:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.327+0000 7f6d55e33700 1 -- start start 2023-10-26T06:45:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d55e33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d501029c0 0x7f6d50198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d55e33700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d50103380 0x7f6d50199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d55e33700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d501089c0 0x7f6d5019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d55e33700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019dcc0 con 0x7f6d501029c0 2023-10-26T06:45:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d55e33700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019de30 con 0x7f6d501089c0 2023-10-26T06:45:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d55e33700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019dff0 con 0x7f6d50103380 2023-10-26T06:45:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d501029c0 0x7f6d50198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d4effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d50103380 0x7f6d50199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d501029c0 0x7f6d50198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38466/0 (socket says 172.21.15.89:38466) 2023-10-26T06:45:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d4f7fe700 1 -- 172.21.15.89:0/2950479241 learned_addr learned my addr 172.21.15.89:0/2950479241 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.328+0000 7f6d4ffff700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d501089c0 0x7f6d5019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4f7fe700 1 -- 172.21.15.89:0/2950479241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d50103380 msgr2=0x7f6d50199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4f7fe700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d50103380 0x7f6d50199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4f7fe700 1 -- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d501089c0 msgr2=0x7f6d5019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4f7fe700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d501089c0 0x7f6d5019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4f7fe700 1 -- 172.21.15.89:0/2950479241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40009520 con 0x7f6d501029c0 2023-10-26T06:45:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4effd700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d50103380 0x7f6d50199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:41.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4ffff700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d501089c0 0x7f6d5019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4f7fe700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d501029c0 0x7f6d50198dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000eb30 tx=0x7f6d4000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40009880 con 0x7f6d501029c0 2023-10-26T06:45:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d400044e0 con 0x7f6d501029c0 2023-10-26T06:45:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d440095f0 con 0x7f6d501029c0 2023-10-26T06:45:41.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40003d80 con 0x7f6d501029c0 2023-10-26T06:45:41.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.329+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d501a5960 con 0x7f6d501029c0 2023-10-26T06:45:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.331+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d50112c10 con 0x7f6d501029c0 2023-10-26T06:45:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.333+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f6d40003f00 con 0x7f6d501029c0 2023-10-26T06:45:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.334+0000 7f6d4cff9700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6d380a15e0 0x7f6d380a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.334+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f6d40014070 con 0x7f6d501029c0 2023-10-26T06:45:41.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.335+0000 7f6d4effd700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6d380a15e0 0x7f6d380a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.335+0000 7f6d4cff9700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] conn(0x7f6d380aaf50 0x7f6d380ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.335+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 --> [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d380ada20 con 0x7f6d380aaf50 2023-10-26T06:45:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.335+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d400cf7a0 con 0x7f6d501029c0 2023-10-26T06:45:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.335+0000 7f6d4ffff700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] conn(0x7f6d380aaf50 0x7f6d380ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:41.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.336+0000 7f6d4effd700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6d380a15e0 0x7f6d380a3aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b2e0 tx=0x7f6d44005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:41.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.336+0000 7f6d4ffff700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] conn(0x7f6d380aaf50 0x7f6d380ad370 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:41.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.337+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== osd.4 v2:172.21.15.135:6808/3181790377 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f6d380ada20 con 0x7f6d380aaf50 2023-10-26T06:45:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.357+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 --> [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d50104100 con 0x7f6d380aaf50 2023-10-26T06:45:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d4cff9700 1 -- 172.21.15.89:0/2950479241 <== osd.4 v2:172.21.15.135:6808/3181790377 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6d50104100 con 0x7f6d380aaf50 2023-10-26T06:45:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] conn(0x7f6d380aaf50 msgr2=0x7f6d380ad370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d55e33700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] conn(0x7f6d380aaf50 0x7f6d380ad370 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6d380a15e0 msgr2=0x7f6d380a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d55e33700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6d380a15e0 0x7f6d380a3aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b2e0 tx=0x7f6d44005960 comp rx=0 tx=0).stop 2023-10-26T06:45:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d501029c0 msgr2=0x7f6d50198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d55e33700 1 --2- 172.21.15.89:0/2950479241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d501029c0 0x7f6d50198dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000eb30 tx=0x7f6d4000ee40 comp rx=0 tx=0).stop 2023-10-26T06:45:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.358+0000 7f6d4ffff700 1 -- 172.21.15.89:0/2950479241 reap_dead start 2023-10-26T06:45:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.359+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 shutdown_connections 2023-10-26T06:45:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.359+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 >> 172.21.15.89:0/2950479241 conn(0x7f6d500fe4b0 msgr2=0x7f6d500fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.359+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 shutdown_connections 2023-10-26T06:45:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:41.359+0000 7f6d55e33700 1 -- 172.21.15.89:0/2950479241 wait complete. 2023-10-26T06:45:41.395 INFO:teuthology.orchestra.run.smithi089.stdout:253403070497 2023-10-26T06:45:41.395 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-26T06:45:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:41 smithi135 ceph-mon[131403]: pgmap v271: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-10-26T06:45:41.497 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:41 smithi089 ceph-mon[95031]: pgmap v271: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-10-26T06:45:41.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:41 smithi192 ceph-mon[131661]: pgmap v271: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2023-10-26T06:45:42.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.064+0000 7f6f5a983700 1 -- 172.21.15.89:0/3307953928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541008a0 msgr2=0x7f6f54100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:42.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.064+0000 7f6f5a983700 1 --2- 172.21.15.89:0/3307953928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541008a0 0x7f6f54100d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009990 tx=0x7f6f48009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.065+0000 7f6f5a983700 1 -- 172.21.15.89:0/3307953928 shutdown_connections 2023-10-26T06:45:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.065+0000 7f6f5a983700 1 --2- 172.21.15.89:0/3307953928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f54101260 0x7f6f5410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.065+0000 7f6f5a983700 1 --2- 172.21.15.89:0/3307953928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f541057c0 0x7f6f54105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.065+0000 7f6f5a983700 1 --2- 172.21.15.89:0/3307953928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541008a0 0x7f6f54100d20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.065+0000 7f6f5a983700 1 -- 172.21.15.89:0/3307953928 >> 172.21.15.89:0/3307953928 conn(0x7f6f540779e0 msgr2=0x7f6f54077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.065+0000 7f6f5a983700 1 -- 172.21.15.89:0/3307953928 shutdown_connections 2023-10-26T06:45:42.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.065+0000 7f6f5a983700 1 -- 172.21.15.89:0/3307953928 wait complete. 2023-10-26T06:45:42.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.066+0000 7f6f5a983700 1 Processor -- start 2023-10-26T06:45:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.066+0000 7f6f5a983700 1 -- start start 2023-10-26T06:45:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f5a983700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f541008a0 0x7f6f54198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f5a983700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f54101260 0x7f6f54199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:42.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f5a983700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541057c0 0x7f6f5419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f5a983700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5419dd50 con 0x7f6f541008a0 2023-10-26T06:45:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f5a983700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5419dec0 con 0x7f6f541057c0 2023-10-26T06:45:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f5a983700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5419e080 con 0x7f6f54101260 2023-10-26T06:45:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f53fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f541008a0 0x7f6f54198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:42.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f53fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f541008a0 0x7f6f54198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38486/0 (socket says 172.21.15.89:38486) 2023-10-26T06:45:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f53fff700 1 -- 172.21.15.89:0/132188575 learned_addr learned my addr 172.21.15.89:0/132188575 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f537fe700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f54101260 0x7f6f54199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.067+0000 7f6f58f20700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541057c0 0x7f6f5419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.068+0000 7f6f53fff700 1 -- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f54101260 msgr2=0x7f6f54199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:42.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.068+0000 7f6f53fff700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f54101260 0x7f6f54199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.068+0000 7f6f53fff700 1 -- 172.21.15.89:0/132188575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541057c0 msgr2=0x7f6f5419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.068+0000 7f6f53fff700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541057c0 0x7f6f5419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.068+0000 7f6f53fff700 1 -- 172.21.15.89:0/132188575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c009c40 con 0x7f6f541008a0 2023-10-26T06:45:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.068+0000 7f6f58f20700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f541057c0 0x7f6f5419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.068+0000 7f6f53fff700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f541008a0 0x7f6f54198e50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400e970 tx=0x7f6f4400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:42.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.069+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44009790 con 0x7f6f541008a0 2023-10-26T06:45:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.069+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4400eec0 con 0x7f6f541008a0 2023-10-26T06:45:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.069+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f480095f0 con 0x7f6f541008a0 2023-10-26T06:45:42.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.069+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44003cf0 con 0x7f6f541008a0 2023-10-26T06:45:42.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.069+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f541a59f0 con 0x7f6f541008a0 2023-10-26T06:45:42.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.070+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f54110a60 con 0x7f6f541008a0 2023-10-26T06:45:42.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.072+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f6f4400ccd0 con 0x7f6f541008a0 2023-10-26T06:45:42.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.074+0000 7f6f517fa700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6f3c0a1560 0x7f6f3c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:42.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.074+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f6f44014070 con 0x7f6f541008a0 2023-10-26T06:45:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.074+0000 7f6f537fe700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6f3c0a1560 0x7f6f3c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.074+0000 7f6f517fa700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] conn(0x7f6f3c0aaed0 0x7f6f3c0ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.074+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 --> [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f3c0ad9a0 con 0x7f6f3c0aaed0 2023-10-26T06:45:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.074+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f440cf550 con 0x7f6f541008a0 2023-10-26T06:45:42.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.075+0000 7f6f58f20700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] conn(0x7f6f3c0aaed0 0x7f6f3c0ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.075+0000 7f6f537fe700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6f3c0a1560 0x7f6f3c0a3a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6f5419a3b0 tx=0x7f6f48005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.075+0000 7f6f58f20700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] conn(0x7f6f3c0aaed0 0x7f6f3c0ad2f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:42.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.076+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== osd.11 v2:172.21.15.192:6824/2369975497 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f6f3c0ad9a0 con 0x7f6f3c0aaed0 2023-10-26T06:45:42.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.094+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 --> [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f54101f30 con 0x7f6f3c0aaed0 2023-10-26T06:45:42.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.094+0000 7f6f517fa700 1 -- 172.21.15.89:0/132188575 <== osd.11 v2:172.21.15.192:6824/2369975497 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6f54101f30 con 0x7f6f3c0aaed0 2023-10-26T06:45:42.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] conn(0x7f6f3c0aaed0 msgr2=0x7f6f3c0ad2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:42.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] conn(0x7f6f3c0aaed0 0x7f6f3c0ad2f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6f3c0a1560 msgr2=0x7f6f3c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:42.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6f3c0a1560 0x7f6f3c0a3a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6f5419a3b0 tx=0x7f6f48005fd0 comp rx=0 tx=0).stop 2023-10-26T06:45:42.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f541008a0 msgr2=0x7f6f54198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 --2- 172.21.15.89:0/132188575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f541008a0 0x7f6f54198e50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400e970 tx=0x7f6f4400ec80 comp rx=0 tx=0).stop 2023-10-26T06:45:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f58f20700 1 -- 172.21.15.89:0/132188575 reap_dead start 2023-10-26T06:45:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 shutdown_connections 2023-10-26T06:45:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 >> 172.21.15.89:0/132188575 conn(0x7f6f540779e0 msgr2=0x7f6f5410ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:42.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 shutdown_connections 2023-10-26T06:45:42.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:42.095+0000 7f6f5a983700 1 -- 172.21.15.89:0/132188575 wait complete. 2023-10-26T06:45:42.312 INFO:teuthology.orchestra.run.smithi089.stdout:146028888115 2023-10-26T06:45:42.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-26T06:45:42.524 INFO:teuthology.orchestra.run.smithi089.stdout:223338299429 2023-10-26T06:45:42.525 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-26T06:45:42.830 INFO:teuthology.orchestra.run.smithi089.stdout:201863462953 2023-10-26T06:45:42.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-26T06:45:42.996 INFO:teuthology.orchestra.run.smithi089.stdout:339302416403 2023-10-26T06:45:42.996 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-26T06:45:43.253 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:43.272 INFO:teuthology.orchestra.run.smithi089.stdout:309237645336 2023-10-26T06:45:43.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-26T06:45:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:43.386 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:43 smithi089 ceph-mon[95031]: pgmap v272: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:43 smithi135 ceph-mon[131403]: pgmap v272: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:43.483 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:43.491 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:43.492 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:43.530 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:43.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:43 smithi192 ceph-mon[131661]: pgmap v272: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:44.615 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:45.279 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:45 smithi089 ceph-mon[95031]: pgmap v273: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:45.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:45 smithi135 ceph-mon[131403]: pgmap v273: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:45 smithi192 ceph-mon[131661]: pgmap v273: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:45:47.314 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:47 smithi089 ceph-mon[95031]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:47 smithi135 ceph-mon[131403]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:47 smithi192 ceph-mon[131661]: pgmap v274: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:47.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 -- 172.21.15.89:0/928450619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881008a0 msgr2=0x7f4f88100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 --2- 172.21.15.89:0/928450619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881008a0 0x7f4f88100d20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009990 tx=0x7f4f7c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 -- 172.21.15.89:0/928450619 shutdown_connections 2023-10-26T06:45:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 --2- 172.21.15.89:0/928450619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f88101260 0x7f4f8810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 --2- 172.21.15.89:0/928450619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f881057c0 0x7f4f88105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 --2- 172.21.15.89:0/928450619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881008a0 0x7f4f88100d20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 -- 172.21.15.89:0/928450619 >> 172.21.15.89:0/928450619 conn(0x7f4f880779e0 msgr2=0x7f4f88077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 -- 172.21.15.89:0/928450619 shutdown_connections 2023-10-26T06:45:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.744+0000 7f4f8c881700 1 -- 172.21.15.89:0/928450619 wait complete. 2023-10-26T06:45:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.745+0000 7f4f8c881700 1 Processor -- start 2023-10-26T06:45:47.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8c881700 1 -- start start 2023-10-26T06:45:47.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8c881700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f881008a0 0x7f4f88198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:47.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8c881700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f88101260 0x7f4f88199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:47.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8c881700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881057c0 0x7f4f8819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:47.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8c881700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8819dd20 con 0x7f4f881057c0 2023-10-26T06:45:47.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8c881700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8819de90 con 0x7f4f88101260 2023-10-26T06:45:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8c881700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8819e050 con 0x7f4f881008a0 2023-10-26T06:45:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f86d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881057c0 0x7f4f8819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f86d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881057c0 0x7f4f8819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38504/0 (socket says 172.21.15.89:38504) 2023-10-26T06:45:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f86d9d700 1 -- 172.21.15.89:0/3388817478 learned_addr learned my addr 172.21.15.89:0/3388817478 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:47.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f8659c700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f881008a0 0x7f4f88198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.746+0000 7f4f85d9b700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f88101260 0x7f4f88199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f86d9d700 1 -- 172.21.15.89:0/3388817478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f881008a0 msgr2=0x7f4f88198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f86d9d700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f881008a0 0x7f4f88198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f86d9d700 1 -- 172.21.15.89:0/3388817478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f88101260 msgr2=0x7f4f88199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:47.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f86d9d700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f88101260 0x7f4f88199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f86d9d700 1 -- 172.21.15.89:0/3388817478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c0095f0 con 0x7f4f881057c0 2023-10-26T06:45:47.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f8659c700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f881008a0 0x7f4f88198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:47.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f85d9b700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f88101260 0x7f4f88199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:47.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f86d9d700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881057c0 0x7f4f8819d710 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000efc0 tx=0x7f4f8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:47.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f777fe700 1 -- 172.21.15.89:0/3388817478 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80018070 con 0x7f4f881057c0 2023-10-26T06:45:47.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8819e2d0 con 0x7f4f881057c0 2023-10-26T06:45:47.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f777fe700 1 -- 172.21.15.89:0/3388817478 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8000f040 con 0x7f4f881057c0 2023-10-26T06:45:47.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f777fe700 1 -- 172.21.15.89:0/3388817478 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000cbc0 con 0x7f4f881057c0 2023-10-26T06:45:47.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.747+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f881a5b90 con 0x7f4f881057c0 2023-10-26T06:45:47.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.748+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f88101f30 con 0x7f4f881057c0 2023-10-26T06:45:47.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.750+0000 7f4f777fe700 1 -- 172.21.15.89:0/3388817478 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f4f8000cd60 con 0x7f4f881057c0 2023-10-26T06:45:47.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.751+0000 7f4f777fe700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f700a1560 0x7f4f700a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:47.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.751+0000 7f4f777fe700 1 -- 172.21.15.89:0/3388817478 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f4f800cc910 con 0x7f4f881057c0 2023-10-26T06:45:47.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.752+0000 7f4f8659c700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f700a1560 0x7f4f700a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:47.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.752+0000 7f4f8659c700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f700a1560 0x7f4f700a3a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f88101930 tx=0x7f4f78009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:47.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.756+0000 7f4f777fe700 1 -- 172.21.15.89:0/3388817478 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4f80095e60 con 0x7f4f881057c0 2023-10-26T06:45:47.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.976+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4f8804e4c0 con 0x7f4f881057c0 2023-10-26T06:45:47.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.976+0000 7f4f777fe700 1 -- 172.21.15.89:0/3388817478 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f80099480 con 0x7f4f881057c0 2023-10-26T06:45:47.978 INFO:teuthology.orchestra.run.smithi089.stdout:38654705735 2023-10-26T06:45:47.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f700a1560 msgr2=0x7f4f700a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:47.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f700a1560 0x7f4f700a3a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f88101930 tx=0x7f4f78009260 comp rx=0 tx=0).stop 2023-10-26T06:45:47.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881057c0 msgr2=0x7f4f8819d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:47.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881057c0 0x7f4f8819d710 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000efc0 tx=0x7f4f8000c3a0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 shutdown_connections 2023-10-26T06:45:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f700a1560 0x7f4f700a3a20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f881057c0 0x7f4f8819d710 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f881008a0 0x7f4f88198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 --2- 172.21.15.89:0/3388817478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f88101260 0x7f4f88199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:47.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 >> 172.21.15.89:0/3388817478 conn(0x7f4f880779e0 msgr2=0x7f4f880fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:47.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 shutdown_connections 2023-10-26T06:45:47.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.979+0000 7f4f8c881700 1 -- 172.21.15.89:0/3388817478 wait complete. 2023-10-26T06:45:48.270 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:48 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3388817478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-26T06:45:48.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:48 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3388817478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-26T06:45:48.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.523+0000 7f8673a33700 1 -- 172.21.15.89:0/3222721728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f866c1008a0 msgr2=0x7f866c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.523+0000 7f8673a33700 1 --2- 172.21.15.89:0/3222721728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f866c1008a0 0x7f866c100d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f865c0098f0 tx=0x7f865c009c00 comp rx=0 tx=0).stop 2023-10-26T06:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.523+0000 7f8673a33700 1 -- 172.21.15.89:0/3222721728 shutdown_connections 2023-10-26T06:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.523+0000 7f8673a33700 1 --2- 172.21.15.89:0/3222721728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c101260 0x7f866c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.523+0000 7f8673a33700 1 --2- 172.21.15.89:0/3222721728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f866c1057c0 0x7f866c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.523+0000 7f8673a33700 1 --2- 172.21.15.89:0/3222721728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f866c1008a0 0x7f866c100d20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.523+0000 7f8673a33700 1 -- 172.21.15.89:0/3222721728 >> 172.21.15.89:0/3222721728 conn(0x7f866c0779e0 msgr2=0x7f866c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.526+0000 7f8673a33700 1 -- 172.21.15.89:0/3222721728 shutdown_connections 2023-10-26T06:45:48.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.526+0000 7f8673a33700 1 -- 172.21.15.89:0/3222721728 wait complete. 2023-10-26T06:45:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.527+0000 7f8673a33700 1 Processor -- start 2023-10-26T06:45:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.528+0000 7f8673a33700 1 -- start start 2023-10-26T06:45:48.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.528+0000 7f8673a33700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c1008a0 0x7f866c114a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.528+0000 7f8673a33700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f866c101260 0x7f866c11a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.528+0000 7f8673a33700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f866c1057c0 0x7f866c1150a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.528+0000 7f8673a33700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c10fd60 con 0x7f866c1057c0 2023-10-26T06:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.528+0000 7f8673a33700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c10fed0 con 0x7f866c101260 2023-10-26T06:45:48.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.528+0000 7f8673a33700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c110090 con 0x7f866c1008a0 2023-10-26T06:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c1008a0 0x7f866c114a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c1008a0 0x7f866c114a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50898/0 (socket says 172.21.15.89:50898) 2023-10-26T06:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 -- 172.21.15.89:0/2674393635 learned_addr learned my addr 172.21.15.89:0/2674393635 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 -- 172.21.15.89:0/2674393635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f866c101260 msgr2=0x7f866c11a790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-26T06:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f866c101260 0x7f866c11a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 -- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f866c1057c0 msgr2=0x7f866c1150a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T06:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f866c1057c0 0x7f866c1150a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.529+0000 7f86717cf700 1 -- 172.21.15.89:0/2674393635 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680095f0 con 0x7f866c1008a0 2023-10-26T06:45:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.530+0000 7f86717cf700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c1008a0 0x7f866c114a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f866800ebf0 tx=0x7f866800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.530+0000 7f86627fc700 1 -- 172.21.15.89:0/2674393635 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866800ce10 con 0x7f866c1008a0 2023-10-26T06:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.530+0000 7f86627fc700 1 -- 172.21.15.89:0/2674393635 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86680108f0 con 0x7f866c1008a0 2023-10-26T06:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.530+0000 7f86627fc700 1 -- 172.21.15.89:0/2674393635 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668018910 con 0x7f866c1008a0 2023-10-26T06:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.530+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c009520 con 0x7f866c1008a0 2023-10-26T06:45:48.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.530+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c1105d0 con 0x7f866c1008a0 2023-10-26T06:45:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.536+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c108da0 con 0x7f866c1008a0 2023-10-26T06:45:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.537+0000 7f86627fc700 1 -- 172.21.15.89:0/2674393635 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f8668010a60 con 0x7f866c1008a0 2023-10-26T06:45:48.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.540+0000 7f86627fc700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f86580a15e0 0x7f86580a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:48.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.540+0000 7f86627fc700 1 -- 172.21.15.89:0/2674393635 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f8668014070 con 0x7f866c1008a0 2023-10-26T06:45:48.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.540+0000 7f8670fce700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f86580a15e0 0x7f86580a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:48.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.541+0000 7f8670fce700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f86580a15e0 0x7f86580a3aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f865c009230 tx=0x7f865c009f50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:48.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.544+0000 7f86627fc700 1 -- 172.21.15.89:0/2674393635 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f86680965c0 con 0x7f866c1008a0 2023-10-26T06:45:48.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:48 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3388817478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-26T06:45:48.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.773+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f866c101f30 con 0x7f866c1008a0 2023-10-26T06:45:48.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.774+0000 7f86627fc700 1 -- 172.21.15.89:0/2674393635 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8668099be0 con 0x7f866c1008a0 2023-10-26T06:45:48.775 INFO:teuthology.orchestra.run.smithi089.stdout:167503724593 2023-10-26T06:45:48.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f86580a15e0 msgr2=0x7f86580a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:48.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f86580a15e0 0x7f86580a3aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f865c009230 tx=0x7f865c009f50 comp rx=0 tx=0).stop 2023-10-26T06:45:48.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c1008a0 msgr2=0x7f866c114a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:48.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c1008a0 0x7f866c114a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f866800ebf0 tx=0x7f866800c2d0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 shutdown_connections 2023-10-26T06:45:48.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f86580a15e0 0x7f86580a3aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f866c1057c0 0x7f866c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f866c1008a0 0x7f866c114a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 --2- 172.21.15.89:0/2674393635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f866c101260 0x7f866c11a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:48.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 >> 172.21.15.89:0/2674393635 conn(0x7f866c0779e0 msgr2=0x7f866c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:48.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 shutdown_connections 2023-10-26T06:45:48.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.776+0000 7f8673a33700 1 -- 172.21.15.89:0/2674393635 wait complete. 2023-10-26T06:45:49.050 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705733 got 38654705735 for osd.0 2023-10-26T06:45:49.050 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:49.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.246+0000 7efeb89e9700 1 -- 172.21.15.89:0/319805225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb41008a0 msgr2=0x7efeb4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.246+0000 7efeb89e9700 1 --2- 172.21.15.89:0/319805225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb41008a0 0x7efeb4100d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efea8009990 tx=0x7efea8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.246+0000 7efeb89e9700 1 -- 172.21.15.89:0/319805225 shutdown_connections 2023-10-26T06:45:49.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.246+0000 7efeb89e9700 1 --2- 172.21.15.89:0/319805225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeb4101260 0x7efeb410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.246+0000 7efeb89e9700 1 --2- 172.21.15.89:0/319805225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb41057c0 0x7efeb4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.246+0000 7efeb89e9700 1 --2- 172.21.15.89:0/319805225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb41008a0 0x7efeb4100d20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.246+0000 7efeb89e9700 1 -- 172.21.15.89:0/319805225 >> 172.21.15.89:0/319805225 conn(0x7efeb40779e0 msgr2=0x7efeb4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.247+0000 7efeb89e9700 1 -- 172.21.15.89:0/319805225 shutdown_connections 2023-10-26T06:45:49.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.247+0000 7efeb89e9700 1 -- 172.21.15.89:0/319805225 wait complete. 2023-10-26T06:45:49.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.248+0000 7efeb89e9700 1 Processor -- start 2023-10-26T06:45:49.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.248+0000 7efeb89e9700 1 -- start start 2023-10-26T06:45:49.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb89e9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb41008a0 0x7efeb4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb89e9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb4101260 0x7efeb4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb89e9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeb41057c0 0x7efeb419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb89e9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb419dd20 con 0x7efeb4101260 2023-10-26T06:45:49.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb89e9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb419de90 con 0x7efeb41008a0 2023-10-26T06:45:49.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb89e9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb419e050 con 0x7efeb41057c0 2023-10-26T06:45:49.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb4101260 0x7efeb4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeb41057c0 0x7efeb419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeb41057c0 0x7efeb419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50904/0 (socket says 172.21.15.89:50904) 2023-10-26T06:45:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb2d9d700 1 -- 172.21.15.89:0/1082023676 learned_addr learned my addr 172.21.15.89:0/1082023676 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.249+0000 7efeb259c700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb41008a0 0x7efeb4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.250+0000 7efeb1d9b700 1 -- 172.21.15.89:0/1082023676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeb41057c0 msgr2=0x7efeb419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.250+0000 7efeb1d9b700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeb41057c0 0x7efeb419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.250+0000 7efeb1d9b700 1 -- 172.21.15.89:0/1082023676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb41008a0 msgr2=0x7efeb4198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.250+0000 7efeb1d9b700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb41008a0 0x7efeb4198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.250+0000 7efeb1d9b700 1 -- 172.21.15.89:0/1082023676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea80095f0 con 0x7efeb4101260 2023-10-26T06:45:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.250+0000 7efeb1d9b700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb4101260 0x7efeb4199360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efea80098e0 tx=0x7efea8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.250+0000 7efea37fe700 1 -- 172.21.15.89:0/1082023676 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea801d070 con 0x7efeb4101260 2023-10-26T06:45:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.251+0000 7efea37fe700 1 -- 172.21.15.89:0/1082023676 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea8022470 con 0x7efeb4101260 2023-10-26T06:45:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.251+0000 7efea37fe700 1 -- 172.21.15.89:0/1082023676 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8017650 con 0x7efeb4101260 2023-10-26T06:45:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.252+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb419e2d0 con 0x7efeb4101260 2023-10-26T06:45:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.252+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb41a59c0 con 0x7efeb4101260 2023-10-26T06:45:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.253+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb4101f30 con 0x7efeb4101260 2023-10-26T06:45:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.255+0000 7efea37fe700 1 -- 172.21.15.89:0/1082023676 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7efea80225e0 con 0x7efeb4101260 2023-10-26T06:45:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.258+0000 7efea37fe700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe9c0a15e0 0x7efe9c0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.258+0000 7efea37fe700 1 -- 172.21.15.89:0/1082023676 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7efea80ce760 con 0x7efeb4101260 2023-10-26T06:45:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.258+0000 7efeb259c700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe9c0a15e0 0x7efe9c0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.259+0000 7efeb259c700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe9c0a15e0 0x7efe9c0a3aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efeb4101930 tx=0x7efea4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.261+0000 7efea37fe700 1 -- 172.21.15.89:0/1082023676 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efea8097cb0 con 0x7efeb4101260 2023-10-26T06:45:49.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 -- 172.21.15.89:0/3201702204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 msgr2=0x7fd4880ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 --2- 172.21.15.89:0/3201702204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 0x7fd4880ffbd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009990 tx=0x7fd47c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 -- 172.21.15.89:0/3201702204 shutdown_connections 2023-10-26T06:45:49.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 --2- 172.21.15.89:0/3201702204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd488100110 0x7fd48810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 --2- 172.21.15.89:0/3201702204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4881056b0 0x7fd488105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 --2- 172.21.15.89:0/3201702204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 0x7fd4880ffbd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 -- 172.21.15.89:0/3201702204 >> 172.21.15.89:0/3201702204 conn(0x7fd488075790 msgr2=0x7fd488075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 -- 172.21.15.89:0/3201702204 shutdown_connections 2023-10-26T06:45:49.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.385+0000 7fd48d982700 1 -- 172.21.15.89:0/3201702204 wait complete. 2023-10-26T06:45:49.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.386+0000 7fd48d982700 1 Processor -- start 2023-10-26T06:45:49.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.386+0000 7fd48d982700 1 -- start start 2023-10-26T06:45:49.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd48d982700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 0x7fd48819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd48d982700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd488100110 0x7fd48819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd48d982700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4881056b0 0x7fd488196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd48d982700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488197100 con 0x7fd4880ff750 2023-10-26T06:45:49.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd48d982700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488197270 con 0x7fd488100110 2023-10-26T06:45:49.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd48d982700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488197430 con 0x7fd4881056b0 2023-10-26T06:45:49.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd488100110 0x7fd48819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 0x7fd48819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd488100110 0x7fd48819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37574/0 (socket says 172.21.15.89:37574) 2023-10-26T06:45:49.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd4867fc700 1 -- 172.21.15.89:0/2489260046 learned_addr learned my addr 172.21.15.89:0/2489260046 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:49.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.387+0000 7fd4877fe700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4881056b0 0x7fd488196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd4867fc700 1 -- 172.21.15.89:0/2489260046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4881056b0 msgr2=0x7fd488196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd4867fc700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4881056b0 0x7fd488196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd4867fc700 1 -- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 msgr2=0x7fd48819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd4867fc700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 0x7fd48819c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd4867fc700 1 -- 172.21.15.89:0/2489260046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0095f0 con 0x7fd488100110 2023-10-26T06:45:49.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd4877fe700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4881056b0 0x7fd488196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:49.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd486ffd700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 0x7fd48819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:49.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd4867fc700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd488100110 0x7fd48819ce50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b2e0 tx=0x7fd47c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd48c980700 1 -- 172.21.15.89:0/2489260046 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c01d070 con 0x7fd488100110 2023-10-26T06:45:49.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4881976b0 con 0x7fd488100110 2023-10-26T06:45:49.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd48c980700 1 -- 172.21.15.89:0/2489260046 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47c021410 con 0x7fd488100110 2023-10-26T06:45:49.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd48c980700 1 -- 172.21.15.89:0/2489260046 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c017650 con 0x7fd488100110 2023-10-26T06:45:49.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.388+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4881a3900 con 0x7fd488100110 2023-10-26T06:45:49.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.389+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488107cd0 con 0x7fd488100110 2023-10-26T06:45:49.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.391+0000 7fd48c980700 1 -- 172.21.15.89:0/2489260046 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd47c021580 con 0x7fd488100110 2023-10-26T06:45:49.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.394+0000 7fd48c980700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4700a1560 0x7fd4700a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.394+0000 7fd48c980700 1 -- 172.21.15.89:0/2489260046 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fd47c0ce070 con 0x7fd488100110 2023-10-26T06:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.394+0000 7fd486ffd700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4700a1560 0x7fd4700a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.395+0000 7fd486ffd700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4700a1560 0x7fd4700a3a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd478009fd0 tx=0x7fd478009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.397+0000 7fd48c980700 1 -- 172.21.15.89:0/2489260046 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd47c097730 con 0x7fd488100110 2023-10-26T06:45:49.428 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:49 smithi089 ceph-mon[95031]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:45:49.429 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:49 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/974387196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-26T06:45:49.429 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:49 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2674393635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-26T06:45:49.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 -- 172.21.15.89:0/410379691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c10d250 msgr2=0x7f057c10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 --2- 172.21.15.89:0/410379691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c10d250 0x7f057c10f640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f057400b0a0 tx=0x7f057400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 -- 172.21.15.89:0/410379691 shutdown_connections 2023-10-26T06:45:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 --2- 172.21.15.89:0/410379691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c10d250 0x7f057c10f640 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 --2- 172.21.15.89:0/410379691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f057c072a20 0x7f057c072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 --2- 172.21.15.89:0/410379691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f057c073340 0x7f057c10cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 -- 172.21.15.89:0/410379691 >> 172.21.15.89:0/410379691 conn(0x7f057c0779e0 msgr2=0x7f057c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 -- 172.21.15.89:0/410379691 shutdown_connections 2023-10-26T06:45:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.943+0000 7f058294b700 1 -- 172.21.15.89:0/410379691 wait complete. 2023-10-26T06:45:49.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.944+0000 7f058294b700 1 Processor -- start 2023-10-26T06:45:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.944+0000 7f058294b700 1 -- start start 2023-10-26T06:45:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f058294b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f057c072a20 0x7f057c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f058294b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c073340 0x7f057c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f058294b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f057c10d250 0x7f057c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f058294b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c19def0 con 0x7f057c073340 2023-10-26T06:45:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f058294b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c19e060 con 0x7f057c10d250 2023-10-26T06:45:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f058294b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c19e220 con 0x7f057c072a20 2023-10-26T06:45:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c073340 0x7f057c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c073340 0x7f057c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38526/0 (socket says 172.21.15.89:38526) 2023-10-26T06:45:49.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 -- 172.21.15.89:0/974387196 learned_addr learned my addr 172.21.15.89:0/974387196 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057bfff700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f057c072a20 0x7f057c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f0580ee8700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f057c10d250 0x7f057c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 -- 172.21.15.89:0/974387196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f057c072a20 msgr2=0x7f057c199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f057c072a20 0x7f057c199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 -- 172.21.15.89:0/974387196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f057c10d250 msgr2=0x7f057c19d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f057c10d250 0x7f057c19d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.945+0000 7f057b7fe700 1 -- 172.21.15.89:0/974387196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0574009d00 con 0x7f057c073340 2023-10-26T06:45:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.946+0000 7f057b7fe700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c073340 0x7f057c199580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f057000e9a0 tx=0x7f057000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.946+0000 7f05797fa700 1 -- 172.21.15.89:0/974387196 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570009790 con 0x7f057c073340 2023-10-26T06:45:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.946+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c19e500 con 0x7f057c073340 2023-10-26T06:45:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.946+0000 7f05797fa700 1 -- 172.21.15.89:0/974387196 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057000cbd0 con 0x7f057c073340 2023-10-26T06:45:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.946+0000 7f05797fa700 1 -- 172.21.15.89:0/974387196 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570018680 con 0x7f057c073340 2023-10-26T06:45:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.946+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c1a5d60 con 0x7f057c073340 2023-10-26T06:45:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.948+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c108460 con 0x7f057c073340 2023-10-26T06:45:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.949+0000 7f05797fa700 1 -- 172.21.15.89:0/974387196 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f057000cd40 con 0x7f057c073340 2023-10-26T06:45:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.951+0000 7f05797fa700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05640a1560 0x7f05640a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.951+0000 7f05797fa700 1 -- 172.21.15.89:0/974387196 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f0570014070 con 0x7f057c073340 2023-10-26T06:45:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.951+0000 7f057bfff700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05640a1560 0x7f05640a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.952+0000 7f057bfff700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05640a1560 0x7f05640a3a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f056c009bf0 tx=0x7f056c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:47.955+0000 7f05797fa700 1 -- 172.21.15.89:0/974387196 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0570096710 con 0x7f057c073340 2023-10-26T06:45:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.230+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f057c0740b0 con 0x7f057c073340 2023-10-26T06:45:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.231+0000 7f05797fa700 1 -- 172.21.15.89:0/974387196 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0570099d30 con 0x7f057c073340 2023-10-26T06:45:49.466 INFO:teuthology.orchestra.run.smithi089.stdout:81604378686 2023-10-26T06:45:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.233+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05640a1560 msgr2=0x7f05640a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.233+0000 7f058294b700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05640a1560 0x7f05640a3a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f056c009bf0 tx=0x7f056c009260 comp rx=0 tx=0).stop 2023-10-26T06:45:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.233+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c073340 msgr2=0x7f057c199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.233+0000 7f058294b700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c073340 0x7f057c199580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f057000e9a0 tx=0x7f057000ed60 comp rx=0 tx=0).stop 2023-10-26T06:45:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 shutdown_connections 2023-10-26T06:45:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f05640a1560 0x7f05640a3a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f057c10d250 0x7f057c19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f057c072a20 0x7f057c199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 --2- 172.21.15.89:0/974387196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f057c073340 0x7f057c199580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 >> 172.21.15.89:0/974387196 conn(0x7f057c0779e0 msgr2=0x7f057c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 shutdown_connections 2023-10-26T06:45:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:48.234+0000 7f058294b700 1 -- 172.21.15.89:0/974387196 wait complete. 2023-10-26T06:45:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:49.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:49 smithi135 ceph-mon[131403]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:45:49.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:49 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/974387196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-26T06:45:49.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:49 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2674393635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-26T06:45:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.585+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7efeb404e4c0 con 0x7efeb4101260 2023-10-26T06:45:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.585+0000 7efea37fe700 1 -- 172.21.15.89:0/1082023676 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efea809b2d0 con 0x7efeb4101260 2023-10-26T06:45:49.587 INFO:teuthology.orchestra.run.smithi089.stdout:146028888116 2023-10-26T06:45:49.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe9c0a15e0 msgr2=0x7efe9c0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe9c0a15e0 0x7efe9c0a3aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efeb4101930 tx=0x7efea4009260 comp rx=0 tx=0).stop 2023-10-26T06:45:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb4101260 msgr2=0x7efeb4199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb4101260 0x7efeb4199360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efea80098e0 tx=0x7efea8005740 comp rx=0 tx=0).stop 2023-10-26T06:45:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 shutdown_connections 2023-10-26T06:45:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe9c0a15e0 0x7efe9c0a3aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efeb41057c0 0x7efeb419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb41008a0 0x7efeb4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 --2- 172.21.15.89:0/1082023676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efeb4101260 0x7efeb4199360 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.589+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 >> 172.21.15.89:0/1082023676 conn(0x7efeb40779e0 msgr2=0x7efeb40fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.590+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 shutdown_connections 2023-10-26T06:45:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.590+0000 7efeb89e9700 1 -- 172.21.15.89:0/1082023676 wait complete. 2023-10-26T06:45:49.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 -- 172.21.15.89:0/3821987054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c073340 msgr2=0x7f655c10cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 --2- 172.21.15.89:0/3821987054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c073340 0x7f655c10cd10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6550009990 tx=0x7f6550009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 -- 172.21.15.89:0/3821987054 shutdown_connections 2023-10-26T06:45:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 --2- 172.21.15.89:0/3821987054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c072a20 0x7f655c072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 --2- 172.21.15.89:0/3821987054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 0x7f655c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 --2- 172.21.15.89:0/3821987054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c073340 0x7f655c10cd10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 -- 172.21.15.89:0/3821987054 >> 172.21.15.89:0/3821987054 conn(0x7f655c0779e0 msgr2=0x7f655c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 -- 172.21.15.89:0/3821987054 shutdown_connections 2023-10-26T06:45:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.603+0000 7f6561cba700 1 -- 172.21.15.89:0/3821987054 wait complete. 2023-10-26T06:45:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.605+0000 7f6561cba700 1 Processor -- start 2023-10-26T06:45:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.605+0000 7f6561cba700 1 -- start start 2023-10-26T06:45:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.605+0000 7f6561cba700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c072a20 0x7f655c198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f6561cba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c073340 0x7f655c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f6561cba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 0x7f655c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f6561cba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c19dde0 con 0x7f655c073340 2023-10-26T06:45:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f6561cba700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c19df50 con 0x7f655c072a20 2023-10-26T06:45:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f6561cba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c19e110 con 0x7f655c10d250 2023-10-26T06:45:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c073340 0x7f655c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 0x7f655c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 0x7f655c19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50924/0 (socket says 172.21.15.89:50924) 2023-10-26T06:45:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c073340 0x7f655c199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38596/0 (socket says 172.21.15.89:38596) 2023-10-26T06:45:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655bfff700 1 -- 172.21.15.89:0/3976506667 learned_addr learned my addr 172.21.15.89:0/3976506667 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655b7fe700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c072a20 0x7f655c198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655bfff700 1 -- 172.21.15.89:0/3976506667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c072a20 msgr2=0x7f655c198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.606+0000 7f655bfff700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c072a20 0x7f655c198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f655bfff700 1 -- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c073340 msgr2=0x7f655c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f655bfff700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c073340 0x7f655c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f655bfff700 1 -- 172.21.15.89:0/3976506667 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500095f0 con 0x7f655c10d250 2023-10-26T06:45:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f655bfff700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 0x7f655c19d7d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6554007ee0 tx=0x7f655400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f6558ff9700 1 -- 172.21.15.89:0/3976506667 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554004db0 con 0x7f655c10d250 2023-10-26T06:45:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f6558ff9700 1 -- 172.21.15.89:0/3976506667 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f655400f040 con 0x7f655c10d250 2023-10-26T06:45:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f6558ff9700 1 -- 172.21.15.89:0/3976506667 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65540058a0 con 0x7f655c10d250 2023-10-26T06:45:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c19e3f0 con 0x7f655c10d250 2023-10-26T06:45:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.607+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c1a5c50 con 0x7f655c10d250 2023-10-26T06:45:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.609+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655c108460 con 0x7f655c10d250 2023-10-26T06:45:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.611+0000 7f6558ff9700 1 -- 172.21.15.89:0/3976506667 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f6554015400 con 0x7f655c10d250 2023-10-26T06:45:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.612+0000 7f6558ff9700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65440a15e0 0x7f65440a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.612+0000 7f6558ff9700 1 -- 172.21.15.89:0/3976506667 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f65540cc6e0 con 0x7f655c10d250 2023-10-26T06:45:49.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.612+0000 7f655b7fe700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65440a15e0 0x7f65440a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.613+0000 7f655b7fe700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65440a15e0 0x7f65440a3aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f654c009c60 tx=0x7f654c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.616+0000 7f6558ff9700 1 -- 172.21.15.89:0/3976506667 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f655404f100 con 0x7f655c10d250 2023-10-26T06:45:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:49 smithi192 ceph-mon[131661]: pgmap v275: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:45:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:49 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/974387196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-26T06:45:49.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:49 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2674393635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-26T06:45:49.646 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:49.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.731+0000 7f5914156700 1 -- 172.21.15.89:0/666744467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c101260 msgr2=0x7f590c10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.731+0000 7f5914156700 1 --2- 172.21.15.89:0/666744467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c101260 0x7f590c10d650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f590400b0a0 tx=0x7f590400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.732+0000 7f5914156700 1 -- 172.21.15.89:0/666744467 shutdown_connections 2023-10-26T06:45:49.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.732+0000 7f5914156700 1 --2- 172.21.15.89:0/666744467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f590c1057c0 0x7f590c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.732+0000 7f5914156700 1 --2- 172.21.15.89:0/666744467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c101260 0x7f590c10d650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.732+0000 7f5914156700 1 --2- 172.21.15.89:0/666744467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f590c1008a0 0x7f590c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.732+0000 7f5914156700 1 -- 172.21.15.89:0/666744467 >> 172.21.15.89:0/666744467 conn(0x7f590c0779e0 msgr2=0x7f590c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.732+0000 7f5914156700 1 -- 172.21.15.89:0/666744467 shutdown_connections 2023-10-26T06:45:49.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.732+0000 7f5914156700 1 -- 172.21.15.89:0/666744467 wait complete. 2023-10-26T06:45:49.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.733+0000 7f5914156700 1 Processor -- start 2023-10-26T06:45:49.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.734+0000 7f5914156700 1 -- start start 2023-10-26T06:45:49.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5914156700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c1008a0 0x7f590c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5914156700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f590c101260 0x7f590c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5914156700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f590c1057c0 0x7f590c194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5914156700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c194e30 con 0x7f590c101260 2023-10-26T06:45:49.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5911ef2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c1008a0 0x7f590c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5911ef2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c1008a0 0x7f590c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37622/0 (socket says 172.21.15.89:37622) 2023-10-26T06:45:49.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5911ef2700 1 -- 172.21.15.89:0/3040336170 learned_addr learned my addr 172.21.15.89:0/3040336170 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:49.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f59116f1700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f590c101260 0x7f590c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c194fa0 con 0x7f590c1008a0 2023-10-26T06:45:49.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.735+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c195160 con 0x7f590c1057c0 2023-10-26T06:45:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.736+0000 7f59126f3700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f590c1057c0 0x7f590c194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.736+0000 7f59116f1700 1 -- 172.21.15.89:0/3040336170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f590c1057c0 msgr2=0x7f590c194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.736+0000 7f59116f1700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f590c1057c0 0x7f590c194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.736+0000 7f59116f1700 1 -- 172.21.15.89:0/3040336170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c1008a0 msgr2=0x7f590c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.736+0000 7f59116f1700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c1008a0 0x7f590c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.736+0000 7f59116f1700 1 -- 172.21.15.89:0/3040336170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5904009d00 con 0x7f590c101260 2023-10-26T06:45:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.736+0000 7f59116f1700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f590c101260 0x7f590c19ab60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00e9f0 tx=0x7f58fc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.737+0000 7f5902ffd700 1 -- 172.21.15.89:0/3040336170 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00c9c0 con 0x7f590c101260 2023-10-26T06:45:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.737+0000 7f5902ffd700 1 -- 172.21.15.89:0/3040336170 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58fc010430 con 0x7f590c101260 2023-10-26T06:45:49.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.737+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c195440 con 0x7f590c101260 2023-10-26T06:45:49.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.737+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c1a1650 con 0x7f590c101260 2023-10-26T06:45:49.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.738+0000 7f5902ffd700 1 -- 172.21.15.89:0/3040336170 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc018680 con 0x7f590c101260 2023-10-26T06:45:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.739+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c108da0 con 0x7f590c101260 2023-10-26T06:45:49.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.742+0000 7f5902ffd700 1 -- 172.21.15.89:0/3040336170 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f58fc010860 con 0x7f590c101260 2023-10-26T06:45:49.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.744+0000 7f5902ffd700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58f80a5970 0x7f58f80a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:49.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.744+0000 7f5902ffd700 1 -- 172.21.15.89:0/3040336170 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f58fc014070 con 0x7f590c101260 2023-10-26T06:45:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.744+0000 7f5911ef2700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58f80a5970 0x7f58f80a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:49.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.745+0000 7f5911ef2700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58f80a5970 0x7f58f80a7e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5908007bf0 tx=0x7f5908007480 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:49.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.748+0000 7f5902ffd700 1 -- 172.21.15.89:0/3040336170 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f58fc096510 con 0x7f590c101260 2023-10-26T06:45:49.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.772+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd488100df0 con 0x7fd488100110 2023-10-26T06:45:49.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.772+0000 7fd48c980700 1 -- 172.21.15.89:0/2489260046 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd47c09ad50 con 0x7fd488100110 2023-10-26T06:45:49.774 INFO:teuthology.orchestra.run.smithi089.stdout:253403070498 2023-10-26T06:45:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4700a1560 msgr2=0x7fd4700a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4700a1560 0x7fd4700a3a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd478009fd0 tx=0x7fd478009260 comp rx=0 tx=0).stop 2023-10-26T06:45:49.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd488100110 msgr2=0x7fd48819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd488100110 0x7fd48819ce50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b2e0 tx=0x7fd47c00bfd0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 shutdown_connections 2023-10-26T06:45:49.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4700a1560 0x7fd4700a3a20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4880ff750 0x7fd48819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4881056b0 0x7fd488196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 --2- 172.21.15.89:0/2489260046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd488100110 0x7fd48819ce50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 >> 172.21.15.89:0/2489260046 conn(0x7fd488075790 msgr2=0x7fd4880fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 shutdown_connections 2023-10-26T06:45:49.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.775+0000 7fd48d982700 1 -- 172.21.15.89:0/2489260046 wait complete. 2023-10-26T06:45:49.817 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.901+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f655c0740b0 con 0x7f655c10d250 2023-10-26T06:45:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.902+0000 7f6558ff9700 1 -- 172.21.15.89:0/3976506667 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6554095ba0 con 0x7f655c10d250 2023-10-26T06:45:49.905 INFO:teuthology.orchestra.run.smithi089.stdout:111669149754 2023-10-26T06:45:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65440a15e0 msgr2=0x7f65440a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65440a15e0 0x7f65440a3aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f654c009c60 tx=0x7f654c009260 comp rx=0 tx=0).stop 2023-10-26T06:45:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 msgr2=0x7f655c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 0x7f655c19d7d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6554007ee0 tx=0x7f655400d3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 shutdown_connections 2023-10-26T06:45:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f65440a15e0 0x7f65440a3aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f655c10d250 0x7f655c19d7d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f655c072a20 0x7f655c198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 --2- 172.21.15.89:0/3976506667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c073340 0x7f655c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.905+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 >> 172.21.15.89:0/3976506667 conn(0x7f655c0779e0 msgr2=0x7f655c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.907+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 shutdown_connections 2023-10-26T06:45:49.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.907+0000 7f6561cba700 1 -- 172.21.15.89:0/3976506667 wait complete. 2023-10-26T06:45:49.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.985+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f590c195c60 con 0x7f590c101260 2023-10-26T06:45:49.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.985+0000 7f5902ffd700 1 -- 172.21.15.89:0/3040336170 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58fc099b30 con 0x7f590c101260 2023-10-26T06:45:49.987 INFO:teuthology.orchestra.run.smithi089.stdout:287762808861 2023-10-26T06:45:49.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58f80a5970 msgr2=0x7f58f80a7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58f80a5970 0x7f58f80a7e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5908007bf0 tx=0x7f5908007480 comp rx=0 tx=0).stop 2023-10-26T06:45:49.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f590c101260 msgr2=0x7f590c19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:49.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f590c101260 0x7f590c19ab60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00e9f0 tx=0x7f58fc00ed00 comp rx=0 tx=0).stop 2023-10-26T06:45:49.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 shutdown_connections 2023-10-26T06:45:49.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58f80a5970 0x7f58f80a7e30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f590c1057c0 0x7f590c194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f590c1008a0 0x7f590c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 --2- 172.21.15.89:0/3040336170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f590c101260 0x7f590c19ab60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:49.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 >> 172.21.15.89:0/3040336170 conn(0x7f590c0779e0 msgr2=0x7f590c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:49.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 shutdown_connections 2023-10-26T06:45:49.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:49.988+0000 7f5914156700 1 -- 172.21.15.89:0/3040336170 wait complete. 2023-10-26T06:45:50.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:50 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1082023676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-26T06:45:50.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:50 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2489260046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-26T06:45:50.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:50 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3976506667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-26T06:45:50.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:50 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3040336170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-26T06:45:50.232 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724591 got 167503724593 for osd.5 2023-10-26T06:45:50.233 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:50.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:50 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1082023676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-26T06:45:50.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:50 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2489260046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-26T06:45:50.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:50 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3976506667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-26T06:45:50.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:50 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3040336170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-26T06:45:50.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:50 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1082023676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-26T06:45:50.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:50 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2489260046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-26T06:45:50.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:50 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3976506667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-26T06:45:50.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:50 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3040336170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-26T06:45:50.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149751 got 111669149754 for osd.3 2023-10-26T06:45:50.994 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:50.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378683 got 81604378686 for osd.2 2023-10-26T06:45:50.995 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:51.442 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:51 smithi089 ceph-mon[95031]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:51.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:51 smithi135 ceph-mon[131403]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:51.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070497 got 253403070498 for osd.8 2023-10-26T06:45:51.512 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:51 smithi192 ceph-mon[131661]: pgmap v276: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:51.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 -- 172.21.15.89:0/1875504386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c113940 msgr2=0x7f533c115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:51.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 --2- 172.21.15.89:0/1875504386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c113940 0x7f533c115d30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5334009fc0 tx=0x7f533400b330 comp rx=0 tx=0).stop 2023-10-26T06:45:51.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 -- 172.21.15.89:0/1875504386 shutdown_connections 2023-10-26T06:45:51.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 --2- 172.21.15.89:0/1875504386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c113940 0x7f533c115d30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 --2- 172.21.15.89:0/1875504386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533c1077f0 0x7f533c107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 --2- 172.21.15.89:0/1875504386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f533c108110 0x7f533c113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 -- 172.21.15.89:0/1875504386 >> 172.21.15.89:0/1875504386 conn(0x7f533c06b8c0 msgr2=0x7f533c06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 -- 172.21.15.89:0/1875504386 shutdown_connections 2023-10-26T06:45:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.990+0000 7f534250b700 1 -- 172.21.15.89:0/1875504386 wait complete. 2023-10-26T06:45:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.991+0000 7f534250b700 1 Processor -- start 2023-10-26T06:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.991+0000 7f534250b700 1 -- start start 2023-10-26T06:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f534250b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533c1077f0 0x7f533c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f534250b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c108110 0x7f533c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f534250b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f533c113940 0x7f533c1a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:51.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f534250b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1a44c0 con 0x7f533c108110 2023-10-26T06:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f534250b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1a4630 con 0x7f533c113940 2023-10-26T06:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f534250b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1a47f0 con 0x7f533c1077f0 2023-10-26T06:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f533b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c108110 0x7f533c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f533b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c108110 0x7f533c19fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37562/0 (socket says 172.21.15.89:37562) 2023-10-26T06:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f533b7fe700 1 -- 172.21.15.89:0/1181265088 learned_addr learned my addr 172.21.15.89:0/1181265088 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f533bfff700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533c1077f0 0x7f533c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.992+0000 7f5340aa8700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f533c113940 0x7f533c1a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f533b7fe700 1 -- 172.21.15.89:0/1181265088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533c1077f0 msgr2=0x7f533c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:51.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f533b7fe700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533c1077f0 0x7f533c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f533b7fe700 1 -- 172.21.15.89:0/1181265088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f533c113940 msgr2=0x7f533c1a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f533b7fe700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f533c113940 0x7f533c1a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f533b7fe700 1 -- 172.21.15.89:0/1181265088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334009c40 con 0x7f533c108110 2023-10-26T06:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f533bfff700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533c1077f0 0x7f533c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:51.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f5340aa8700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f533c113940 0x7f533c1a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f533b7fe700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c108110 0x7f533c19fb00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f533000e9a0 tx=0x7f533000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f53397fa700 1 -- 172.21.15.89:0/1181265088 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330009790 con 0x7f533c108110 2023-10-26T06:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f53397fa700 1 -- 172.21.15.89:0/1181265088 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f533000cbd0 con 0x7f533c108110 2023-10-26T06:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c1a4ad0 con 0x7f533c108110 2023-10-26T06:45:51.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f53397fa700 1 -- 172.21.15.89:0/1181265088 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330018680 con 0x7f533c108110 2023-10-26T06:45:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.993+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1ac330 con 0x7f533c108110 2023-10-26T06:45:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.994+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c10eb50 con 0x7f533c108110 2023-10-26T06:45:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.996+0000 7f53397fa700 1 -- 172.21.15.89:0/1181265088 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f533000cd40 con 0x7f533c108110 2023-10-26T06:45:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.997+0000 7f53397fa700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53240a15e0 0x7f53240a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:51.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.998+0000 7f53397fa700 1 -- 172.21.15.89:0/1181265088 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f5330014070 con 0x7f533c108110 2023-10-26T06:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.998+0000 7f533bfff700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53240a15e0 0x7f53240a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:50.998+0000 7f533bfff700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53240a15e0 0x7f53240a3aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f532c005fd0 tx=0x7f532c009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.002+0000 7f53397fa700 1 -- 172.21.15.89:0/1181265088 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5330052100 con 0x7f533c108110 2023-10-26T06:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.218+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f533c1a08d0 con 0x7f533c108110 2023-10-26T06:45:51.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.219+0000 7f53397fa700 1 -- 172.21.15.89:0/1181265088 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f533009a100 con 0x7f533c108110 2023-10-26T06:45:51.809 INFO:teuthology.orchestra.run.smithi089.stdout:60129542211 2023-10-26T06:45:51.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53240a15e0 msgr2=0x7f53240a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:51.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53240a15e0 0x7f53240a3aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f532c005fd0 tx=0x7f532c009310 comp rx=0 tx=0).stop 2023-10-26T06:45:51.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c108110 msgr2=0x7f533c19fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:51.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c108110 0x7f533c19fb00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f533000e9a0 tx=0x7f533000ed60 comp rx=0 tx=0).stop 2023-10-26T06:45:51.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 shutdown_connections 2023-10-26T06:45:51.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53240a15e0 0x7f53240a3aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f533c113940 0x7f533c1a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533c1077f0 0x7f533c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 --2- 172.21.15.89:0/1181265088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f533c108110 0x7f533c19fb00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:51.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.221+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 >> 172.21.15.89:0/1181265088 conn(0x7f533c06b8c0 msgr2=0x7f533c070940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:51.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.222+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 shutdown_connections 2023-10-26T06:45:51.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:51.222+0000 7f534250b700 1 -- 172.21.15.89:0/1181265088 wait complete. 2023-10-26T06:45:51.922 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808859 got 287762808861 for osd.9 2023-10-26T06:45:51.922 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:52.387 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:52 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1181265088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-26T06:45:52.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:52 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1181265088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-26T06:45:52.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:52 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1181265088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-26T06:45:53.017 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:53.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542208 got 60129542211 for osd.1 2023-10-26T06:45:53.077 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:53.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:53 smithi089 ceph-mon[95031]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:53.360 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888115 got 146028888116 for osd.4 2023-10-26T06:45:53.360 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:53.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:53 smithi135 ceph-mon[131403]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:53 smithi192 ceph-mon[131661]: pgmap v277: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:54.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.042+0000 7f3a4c319700 1 -- 172.21.15.89:0/2269436388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441067d0 msgr2=0x7f3a44106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.042+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2269436388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441067d0 0x7f3a44106bb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009990 tx=0x7f3a40009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.043+0000 7f3a4c319700 1 -- 172.21.15.89:0/2269436388 shutdown_connections 2023-10-26T06:45:54.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.043+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2269436388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a44101290 0x7f3a4410d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.043+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2269436388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441067d0 0x7f3a44106bb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.043+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2269436388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a441008d0 0x7f3a44100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.043+0000 7f3a4c319700 1 -- 172.21.15.89:0/2269436388 >> 172.21.15.89:0/2269436388 conn(0x7f3a440778d0 msgr2=0x7f3a44077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:54.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.044+0000 7f3a4c319700 1 -- 172.21.15.89:0/2269436388 shutdown_connections 2023-10-26T06:45:54.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.044+0000 7f3a4c319700 1 -- 172.21.15.89:0/2269436388 wait complete. 2023-10-26T06:45:54.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.046+0000 7f3a4c319700 1 Processor -- start 2023-10-26T06:45:54.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.046+0000 7f3a4c319700 1 -- start start 2023-10-26T06:45:54.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4c319700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441008d0 0x7f3a44072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4c319700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a44101290 0x7f3a44073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4c319700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a441067d0 0x7f3a44073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4c319700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a44074ab0 con 0x7f3a441067d0 2023-10-26T06:45:54.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4c319700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a441a9600 con 0x7f3a44101290 2023-10-26T06:45:54.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4c319700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a441a97c0 con 0x7f3a441008d0 2023-10-26T06:45:54.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a0b5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441008d0 0x7f3a44072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a8b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a441067d0 0x7f3a44073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a0b5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441008d0 0x7f3a44072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50372/0 (socket says 172.21.15.89:50372) 2023-10-26T06:45:54.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a8b6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a441067d0 0x7f3a44073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37594/0 (socket says 172.21.15.89:37594) 2023-10-26T06:45:54.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a0b5700 1 -- 172.21.15.89:0/2832767309 learned_addr learned my addr 172.21.15.89:0/2832767309 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:54.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a498b4700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a44101290 0x7f3a44073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a0b5700 1 -- 172.21.15.89:0/2832767309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a44101290 msgr2=0x7f3a44073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a0b5700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a44101290 0x7f3a44073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a0b5700 1 -- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a441067d0 msgr2=0x7f3a44073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.047+0000 7f3a4a0b5700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a441067d0 0x7f3a44073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a4a0b5700 1 -- 172.21.15.89:0/2832767309 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c009c40 con 0x7f3a441008d0 2023-10-26T06:45:54.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a4a8b6700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a441067d0 0x7f3a44073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:45:54.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a4a0b5700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441008d0 0x7f3a44072af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a40004730 tx=0x7f3a40004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:54.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a3b7fe700 1 -- 172.21.15.89:0/2832767309 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4001d070 con 0x7f3a441008d0 2023-10-26T06:45:54.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a3b7fe700 1 -- 172.21.15.89:0/2832767309 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a40022470 con 0x7f3a441008d0 2023-10-26T06:45:54.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a400095f0 con 0x7f3a441008d0 2023-10-26T06:45:54.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a3b7fe700 1 -- 172.21.15.89:0/2832767309 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40017650 con 0x7f3a441008d0 2023-10-26T06:45:54.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.048+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a441a9da0 con 0x7f3a441008d0 2023-10-26T06:45:54.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.049+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a44108d10 con 0x7f3a441008d0 2023-10-26T06:45:54.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.051+0000 7f3a3b7fe700 1 -- 172.21.15.89:0/2832767309 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3a400225e0 con 0x7f3a441008d0 2023-10-26T06:45:54.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.052+0000 7f3a3b7fe700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3a300a15e0 0x7f3a300a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.053+0000 7f3a3b7fe700 1 -- 172.21.15.89:0/2832767309 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f3a400ce860 con 0x7f3a441008d0 2023-10-26T06:45:54.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.053+0000 7f3a498b4700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3a300a15e0 0x7f3a300a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.054+0000 7f3a498b4700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3a300a15e0 0x7f3a300a3aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a44074220 tx=0x7f3a34009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:54.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.055+0000 7f3a3b7fe700 1 -- 172.21.15.89:0/2832767309 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3a40097db0 con 0x7f3a441008d0 2023-10-26T06:45:54.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.293+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3a44101f00 con 0x7f3a441008d0 2023-10-26T06:45:54.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.294+0000 7f3a3b7fe700 1 -- 172.21.15.89:0/2832767309 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a4009b3d0 con 0x7f3a441008d0 2023-10-26T06:45:54.297 INFO:teuthology.orchestra.run.smithi089.stdout:223338299432 2023-10-26T06:45:54.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3a300a15e0 msgr2=0x7f3a300a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3a300a15e0 0x7f3a300a3aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a44074220 tx=0x7f3a34009260 comp rx=0 tx=0).stop 2023-10-26T06:45:54.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441008d0 msgr2=0x7f3a44072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441008d0 0x7f3a44072af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a40004730 tx=0x7f3a40004760 comp rx=0 tx=0).stop 2023-10-26T06:45:54.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 shutdown_connections 2023-10-26T06:45:54.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3a300a15e0 0x7f3a300a3aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a441067d0 0x7f3a44073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a441008d0 0x7f3a44072af0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 --2- 172.21.15.89:0/2832767309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a44101290 0x7f3a44073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 >> 172.21.15.89:0/2832767309 conn(0x7f3a440778d0 msgr2=0x7f3a440ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:54.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 shutdown_connections 2023-10-26T06:45:54.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.297+0000 7f3a4c319700 1 -- 172.21.15.89:0/2832767309 wait complete. 2023-10-26T06:45:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 -- 172.21.15.89:0/1334776626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9441008a0 msgr2=0x7ff944100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 --2- 172.21.15.89:0/1334776626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9441008a0 0x7ff944100d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff934009990 tx=0x7ff934009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 -- 172.21.15.89:0/1334776626 shutdown_connections 2023-10-26T06:45:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 --2- 172.21.15.89:0/1334776626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff944101260 0x7ff94410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 --2- 172.21.15.89:0/1334776626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9441057c0 0x7ff944105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 --2- 172.21.15.89:0/1334776626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9441008a0 0x7ff944100d20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 -- 172.21.15.89:0/1334776626 >> 172.21.15.89:0/1334776626 conn(0x7ff9440779e0 msgr2=0x7ff944077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:54.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 -- 172.21.15.89:0/1334776626 shutdown_connections 2023-10-26T06:45:54.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.355+0000 7ff94b359700 1 -- 172.21.15.89:0/1334776626 wait complete. 2023-10-26T06:45:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 Processor -- start 2023-10-26T06:45:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 -- start start 2023-10-26T06:45:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9441008a0 0x7ff94419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff944101260 0x7ff94419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9441057c0 0x7ff944196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9441971f0 con 0x7ff944101260 2023-10-26T06:45:54.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944197360 con 0x7ff9441057c0 2023-10-26T06:45:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.357+0000 7ff94b359700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944197520 con 0x7ff9441008a0 2023-10-26T06:45:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff944101260 0x7ff94419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff944101260 0x7ff94419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37614/0 (socket says 172.21.15.89:37614) 2023-10-26T06:45:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 -- 172.21.15.89:0/1699784941 learned_addr learned my addr 172.21.15.89:0/1699784941 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:54.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9490f5700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9441008a0 0x7ff94419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 -- 172.21.15.89:0/1699784941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9441008a0 msgr2=0x7ff94419c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9441008a0 0x7ff94419c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 -- 172.21.15.89:0/1699784941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9441057c0 msgr2=0x7ff944196a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T06:45:54.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9441057c0 0x7ff944196a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.358+0000 7ff9488f4700 1 -- 172.21.15.89:0/1699784941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940009520 con 0x7ff944101260 2023-10-26T06:45:54.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.359+0000 7ff9488f4700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff944101260 0x7ff94419ced0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff934005910 tx=0x7ff934004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:54.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.359+0000 7ff93a7fc700 1 -- 172.21.15.89:0/1699784941 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93401d070 con 0x7ff944101260 2023-10-26T06:45:54.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.359+0000 7ff93a7fc700 1 -- 172.21.15.89:0/1699784941 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff934022470 con 0x7ff944101260 2023-10-26T06:45:54.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.359+0000 7ff93a7fc700 1 -- 172.21.15.89:0/1699784941 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934017650 con 0x7ff944101260 2023-10-26T06:45:54.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.359+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9340095f0 con 0x7ff944101260 2023-10-26T06:45:54.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.359+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff944072a20 con 0x7ff944101260 2023-10-26T06:45:54.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.361+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff944197970 con 0x7ff944101260 2023-10-26T06:45:54.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.364+0000 7ff93a7fc700 1 -- 172.21.15.89:0/1699784941 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7ff93400fac0 con 0x7ff944101260 2023-10-26T06:45:54.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.365+0000 7ff93a7fc700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9300a15e0 0x7ff9300a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.366+0000 7ff93a7fc700 1 -- 172.21.15.89:0/1699784941 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7ff9340ce160 con 0x7ff944101260 2023-10-26T06:45:54.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.366+0000 7ff9490f5700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9300a15e0 0x7ff9300a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.366+0000 7ff9490f5700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9300a15e0 0x7ff9300a3aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff944101930 tx=0x7ff940014040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:54.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.368+0000 7ff93a7fc700 1 -- 172.21.15.89:0/1699784941 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff934097710 con 0x7ff944101260 2023-10-26T06:45:54.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.581+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff944197f20 con 0x7ff944101260 2023-10-26T06:45:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.581+0000 7ff93a7fc700 1 -- 172.21.15.89:0/1699784941 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff93409ad30 con 0x7ff944101260 2023-10-26T06:45:54.584 INFO:teuthology.orchestra.run.smithi089.stdout:201863462956 2023-10-26T06:45:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9300a15e0 msgr2=0x7ff9300a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9300a15e0 0x7ff9300a3aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff944101930 tx=0x7ff940014040 comp rx=0 tx=0).stop 2023-10-26T06:45:54.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff944101260 msgr2=0x7ff94419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff944101260 0x7ff94419ced0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff934005910 tx=0x7ff934004730 comp rx=0 tx=0).stop 2023-10-26T06:45:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 shutdown_connections 2023-10-26T06:45:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9300a15e0 0x7ff9300a3aa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9441057c0 0x7ff944196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9441008a0 0x7ff94419c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 --2- 172.21.15.89:0/1699784941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff944101260 0x7ff94419ced0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.583+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 >> 172.21.15.89:0/1699784941 conn(0x7ff9440779e0 msgr2=0x7ff9440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:54.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.584+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 shutdown_connections 2023-10-26T06:45:54.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.584+0000 7ff94b359700 1 -- 172.21.15.89:0/1699784941 wait complete. 2023-10-26T06:45:54.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.828+0000 7f172b445700 1 -- 172.21.15.89:0/3931067570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 msgr2=0x7f1724072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.828+0000 7f172b445700 1 --2- 172.21.15.89:0/3931067570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 0x7f1724072a20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f17200098f0 tx=0x7f1720009c00 comp rx=0 tx=0).stop 2023-10-26T06:45:54.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.828+0000 7f172b445700 1 -- 172.21.15.89:0/3931067570 shutdown_connections 2023-10-26T06:45:54.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.828+0000 7f172b445700 1 --2- 172.21.15.89:0/3931067570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1724111570 0x7f17241139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.828+0000 7f172b445700 1 --2- 172.21.15.89:0/3931067570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 0x7f1724072a20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.828+0000 7f172b445700 1 --2- 172.21.15.89:0/3931067570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 0x7f17240733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.828+0000 7f172b445700 1 -- 172.21.15.89:0/3931067570 >> 172.21.15.89:0/3931067570 conn(0x7f17240779e0 msgr2=0x7f1724077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.829+0000 7f172b445700 1 -- 172.21.15.89:0/3931067570 shutdown_connections 2023-10-26T06:45:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.829+0000 7f172b445700 1 -- 172.21.15.89:0/3931067570 wait complete. 2023-10-26T06:45:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.830+0000 7f172b445700 1 Processor -- start 2023-10-26T06:45:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.830+0000 7f172b445700 1 -- start start 2023-10-26T06:45:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f172b445700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 0x7f172419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f172b445700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 0x7f172419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f172b445700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1724111570 0x7f17241a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f172b445700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241a2210 con 0x7f1724072f60 2023-10-26T06:45:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f172b445700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241a2380 con 0x7f1724111570 2023-10-26T06:45:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f172b445700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241a2540 con 0x7f1724074770 2023-10-26T06:45:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f17291e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 0x7f172419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f17291e1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 0x7f172419d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37632/0 (socket says 172.21.15.89:37632) 2023-10-26T06:45:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f17291e1700 1 -- 172.21.15.89:0/1574175244 learned_addr learned my addr 172.21.15.89:0/1574175244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f17289e0700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 0x7f172419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.831+0000 7f17299e2700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1724111570 0x7f17241a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f17291e1700 1 -- 172.21.15.89:0/1574175244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 msgr2=0x7f172419d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f17291e1700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 0x7f172419d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f17291e1700 1 -- 172.21.15.89:0/1574175244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1724111570 msgr2=0x7f17241a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f17291e1700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1724111570 0x7f17241a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f17291e1700 1 -- 172.21.15.89:0/1574175244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1720009520 con 0x7f1724072f60 2023-10-26T06:45:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f17291e1700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 0x7f172419d310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1720005950 tx=0x7f172000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f171a7fc700 1 -- 172.21.15.89:0/1574175244 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172001c070 con 0x7f1724072f60 2023-10-26T06:45:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17241a27c0 con 0x7f1724072f60 2023-10-26T06:45:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f171a7fc700 1 -- 172.21.15.89:0/1574175244 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f172000bb20 con 0x7f1724072f60 2023-10-26T06:45:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f171a7fc700 1 -- 172.21.15.89:0/1574175244 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17200038e0 con 0x7f1724072f60 2023-10-26T06:45:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.832+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17241aa080 con 0x7f1724072f60 2023-10-26T06:45:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.833+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17240740b0 con 0x7f1724072f60 2023-10-26T06:45:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.835+0000 7f171a7fc700 1 -- 172.21.15.89:0/1574175244 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1720020030 con 0x7f1724072f60 2023-10-26T06:45:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.837+0000 7f171a7fc700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f17100a1560 0x7f17100a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.837+0000 7f171a7fc700 1 -- 172.21.15.89:0/1574175244 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f17200cdc90 con 0x7f1724072f60 2023-10-26T06:45:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.837+0000 7f17289e0700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f17100a1560 0x7f17100a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.837+0000 7f17289e0700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f17100a1560 0x7f17100a3a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f172419e870 tx=0x7f1714009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:54.840+0000 7f171a7fc700 1 -- 172.21.15.89:0/1574175244 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f17200d3020 con 0x7f1724072f60 2023-10-26T06:45:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.071+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f172419e620 con 0x7f1724072f60 2023-10-26T06:45:55.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.071+0000 7f171a7fc700 1 -- 172.21.15.89:0/1574175244 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1720097240 con 0x7f1724072f60 2023-10-26T06:45:55.074 INFO:teuthology.orchestra.run.smithi089.stdout:309237645339 2023-10-26T06:45:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f17100a1560 msgr2=0x7f17100a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f17100a1560 0x7f17100a3a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f172419e870 tx=0x7f1714009310 comp rx=0 tx=0).stop 2023-10-26T06:45:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 msgr2=0x7f172419d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 0x7f172419d310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1720005950 tx=0x7f172000fa00 comp rx=0 tx=0).stop 2023-10-26T06:45:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 shutdown_connections 2023-10-26T06:45:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f17100a1560 0x7f17100a3a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1724072f60 0x7f172419d310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1724111570 0x7f17241a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 --2- 172.21.15.89:0/1574175244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724074770 0x7f172419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 >> 172.21.15.89:0/1574175244 conn(0x7f17240779e0 msgr2=0x7f1724110200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:55.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 shutdown_connections 2023-10-26T06:45:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.073+0000 7f172b445700 1 -- 172.21.15.89:0/1574175244 wait complete. 2023-10-26T06:45:55.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:55 smithi089 ceph-mon[95031]: pgmap v278: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:45:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:55 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2832767309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-26T06:45:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:55 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1699784941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-26T06:45:55.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:55 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1574175244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-26T06:45:55.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462953 got 201863462956 for osd.6 2023-10-26T06:45:55.464 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:55 smithi135 ceph-mon[131403]: pgmap v278: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:45:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:55 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2832767309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-26T06:45:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:55 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1699784941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-26T06:45:55.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:55 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1574175244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-26T06:45:55.468 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299429 got 223338299432 for osd.7 2023-10-26T06:45:55.469 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:55 smithi192 ceph-mon[131661]: pgmap v278: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:45:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:55 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2832767309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-26T06:45:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:55 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1699784941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-26T06:45:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:55 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1574175244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-26T06:45:55.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645336 got 309237645339 for osd.10 2023-10-26T06:45:55.775 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:55.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 -- 172.21.15.89:0/2109264919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae78073340 msgr2=0x7fae7810cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 --2- 172.21.15.89:0/2109264919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae78073340 0x7fae7810cd10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009990 tx=0x7fae6c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 -- 172.21.15.89:0/2109264919 shutdown_connections 2023-10-26T06:45:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 --2- 172.21.15.89:0/2109264919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae7810d250 0x7fae7810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 --2- 172.21.15.89:0/2109264919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 0x7fae78072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 --2- 172.21.15.89:0/2109264919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae78073340 0x7fae7810cd10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 -- 172.21.15.89:0/2109264919 >> 172.21.15.89:0/2109264919 conn(0x7fae780779e0 msgr2=0x7fae78077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:55.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 -- 172.21.15.89:0/2109264919 shutdown_connections 2023-10-26T06:45:55.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.894+0000 7fae7ca93700 1 -- 172.21.15.89:0/2109264919 wait complete. 2023-10-26T06:45:55.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 Processor -- start 2023-10-26T06:45:55.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 -- start start 2023-10-26T06:45:55.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 0x7fae78198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:55.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78073340 0x7fae78199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:55.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7810d250 0x7fae7819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7819ddb0 con 0x7fae7810d250 2023-10-26T06:45:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7819df20 con 0x7fae78073340 2023-10-26T06:45:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.895+0000 7fae7ca93700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7819e0e0 con 0x7fae78072a20 2023-10-26T06:45:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 0x7fae78198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae76d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7810d250 0x7fae7819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:55.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 0x7fae78198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50412/0 (socket says 172.21.15.89:50412) 2023-10-26T06:45:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 -- 172.21.15.89:0/963918859 learned_addr learned my addr 172.21.15.89:0/963918859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae75d9b700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78073340 0x7fae78199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 -- 172.21.15.89:0/963918859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78073340 msgr2=0x7fae78199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78073340 0x7fae78199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 -- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7810d250 msgr2=0x7fae7819d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7810d250 0x7fae7819d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 -- 172.21.15.89:0/963918859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c0095f0 con 0x7fae78072a20 2023-10-26T06:45:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.896+0000 7fae7659c700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 0x7fae78198f00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e9a0 tx=0x7fae6800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.897+0000 7fae677fe700 1 -- 172.21.15.89:0/963918859 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68009790 con 0x7fae78072a20 2023-10-26T06:45:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.897+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7819e3c0 con 0x7fae78072a20 2023-10-26T06:45:55.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.897+0000 7fae677fe700 1 -- 172.21.15.89:0/963918859 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae6800cbd0 con 0x7fae78072a20 2023-10-26T06:45:55.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.897+0000 7fae677fe700 1 -- 172.21.15.89:0/963918859 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68018680 con 0x7fae78072a20 2023-10-26T06:45:55.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.897+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae781a5c20 con 0x7fae78072a20 2023-10-26T06:45:55.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.898+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae78108460 con 0x7fae78072a20 2023-10-26T06:45:55.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.900+0000 7fae677fe700 1 -- 172.21.15.89:0/963918859 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fae6800cd40 con 0x7fae78072a20 2023-10-26T06:45:55.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.901+0000 7fae677fe700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae600a1560 0x7fae600a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:55.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.901+0000 7fae677fe700 1 -- 172.21.15.89:0/963918859 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fae68014070 con 0x7fae78072a20 2023-10-26T06:45:55.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.901+0000 7fae75d9b700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae600a1560 0x7fae600a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:55.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.902+0000 7fae75d9b700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae600a1560 0x7fae600a3a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae7819a410 tx=0x7fae6c00b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:55.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:55.903+0000 7fae677fe700 1 -- 172.21.15.89:0/963918859 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae68096940 con 0x7fae78072a20 2023-10-26T06:45:56.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.102+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fae7804e4f0 con 0x7fae78072a20 2023-10-26T06:45:56.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.102+0000 7fae677fe700 1 -- 172.21.15.89:0/963918859 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fae68099f60 con 0x7fae78072a20 2023-10-26T06:45:56.105 INFO:teuthology.orchestra.run.smithi089.stdout:339302416405 2023-10-26T06:45:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae600a1560 msgr2=0x7fae600a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae600a1560 0x7fae600a3a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae7819a410 tx=0x7fae6c00b260 comp rx=0 tx=0).stop 2023-10-26T06:45:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 msgr2=0x7fae78198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:56.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 0x7fae78198f00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e9a0 tx=0x7fae6800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:45:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 shutdown_connections 2023-10-26T06:45:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fae600a1560 0x7fae600a3a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7810d250 0x7fae7819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:56.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78072a20 0x7fae78198f00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:56.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 --2- 172.21.15.89:0/963918859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae78073340 0x7fae78199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:56.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 >> 172.21.15.89:0/963918859 conn(0x7fae780779e0 msgr2=0x7fae780fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:56.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 shutdown_connections 2023-10-26T06:45:56.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:56.105+0000 7fae7ca93700 1 -- 172.21.15.89:0/963918859 wait complete. 2023-10-26T06:45:56.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:56 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/963918859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-26T06:45:56.529 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:56 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/963918859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-26T06:45:56.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416403 got 339302416405 for osd.11 2023-10-26T06:45:56.624 DEBUG:teuthology.parallel:result is None 2023-10-26T06:45:56.624 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-26T06:45:56.625 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph pg dump --format=json 2023-10-26T06:45:56.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:56 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/963918859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-26T06:45:57.381 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:45:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:57 smithi135 ceph-mon[131403]: pgmap v279: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:57 smithi089 ceph-mon[95031]: pgmap v279: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:57 smithi192 ceph-mon[131661]: pgmap v279: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:45:58.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/1178812792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f98101260 msgr2=0x7f3f9810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:58.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/1178812792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f98101260 0x7f3f9810d650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000b0a0 tx=0x7f3f9000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:45:58.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/1178812792 shutdown_connections 2023-10-26T06:45:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/1178812792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f98101260 0x7f3f9810d650 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/1178812792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f981057c0 0x7f3f98105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/1178812792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f981008a0 0x7f3f98100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:58.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/1178812792 >> 172.21.15.89:0/1178812792 conn(0x7f3f980779e0 msgr2=0x7f3f98077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:58.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/1178812792 shutdown_connections 2023-10-26T06:45:58.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.923+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/1178812792 wait complete. 2023-10-26T06:45:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.924+0000 7f3f9c8c3700 1 Processor -- start 2023-10-26T06:45:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.924+0000 7f3f9c8c3700 1 -- start start 2023-10-26T06:45:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9c8c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f981008a0 0x7f3f98072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9c8c3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f98101260 0x7f3f98073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9c8c3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f981057c0 0x7f3f98073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9c8c3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f981a5050 con 0x7f3f981008a0 2023-10-26T06:45:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9c8c3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f981a51c0 con 0x7f3f981057c0 2023-10-26T06:45:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9c8c3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f981a5380 con 0x7f3f98101260 2023-10-26T06:45:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f981008a0 0x7f3f98072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f981008a0 0x7f3f98072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37680/0 (socket says 172.21.15.89:37680) 2023-10-26T06:45:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 -- 172.21.15.89:0/4285466921 learned_addr learned my addr 172.21.15.89:0/4285466921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:45:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f95d9b700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f98101260 0x7f3f98073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f96d9d700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f981057c0 0x7f3f98073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 -- 172.21.15.89:0/4285466921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f98101260 msgr2=0x7f3f98073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f98101260 0x7f3f98073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 -- 172.21.15.89:0/4285466921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f981057c0 msgr2=0x7f3f98073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f981057c0 0x7f3f98073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f9659c700 1 -- 172.21.15.89:0/4285466921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f90009d00 con 0x7f3f981008a0 2023-10-26T06:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.925+0000 7f3f96d9d700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f981057c0 0x7f3f98073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:45:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.926+0000 7f3f9659c700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f981008a0 0x7f3f98072af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e970 tx=0x7f3f8800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.926+0000 7f3f877fe700 1 -- 172.21.15.89:0/4285466921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88009790 con 0x7f3f981008a0 2023-10-26T06:45:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.926+0000 7f3f877fe700 1 -- 172.21.15.89:0/4285466921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f8800eec0 con 0x7f3f981008a0 2023-10-26T06:45:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.926+0000 7f3f877fe700 1 -- 172.21.15.89:0/4285466921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88003cf0 con 0x7f3f981008a0 2023-10-26T06:45:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.926+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f981a5660 con 0x7f3f981008a0 2023-10-26T06:45:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.926+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f981a5b60 con 0x7f3f981008a0 2023-10-26T06:45:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.927+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f98101f30 con 0x7f3f981008a0 2023-10-26T06:45:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.929+0000 7f3f877fe700 1 -- 172.21.15.89:0/4285466921 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3f8800ccd0 con 0x7f3f981008a0 2023-10-26T06:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.931+0000 7f3f877fe700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f800a15e0 0x7f3f800a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.931+0000 7f3f877fe700 1 -- 172.21.15.89:0/4285466921 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7f3f88014070 con 0x7f3f981008a0 2023-10-26T06:45:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.932+0000 7f3f95d9b700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f800a15e0 0x7f3f800a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:45:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.932+0000 7f3f95d9b700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f800a15e0 0x7f3f800a3aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f98074220 tx=0x7f3f8c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:45:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:58.936+0000 7f3f877fe700 1 -- 172.21.15.89:0/4285466921 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f88096540 con 0x7f3f981008a0 2023-10-26T06:45:59.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.131+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f981107f0 con 0x7f3f800a15e0 2023-10-26T06:45:59.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.142+0000 7f3f877fe700 1 -- 172.21.15.89:0/4285466921 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340726 (secure 0 0 0) 0x7f3f981107f0 con 0x7f3f800a15e0 2023-10-26T06:45:59.150 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:45:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f800a15e0 msgr2=0x7f3f800a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f800a15e0 0x7f3f800a3aa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f98074220 tx=0x7f3f8c009260 comp rx=0 tx=0).stop 2023-10-26T06:45:59.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f981008a0 msgr2=0x7f3f98072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:45:59.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f981008a0 0x7f3f98072af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e970 tx=0x7f3f8800ec80 comp rx=0 tx=0).stop 2023-10-26T06:45:59.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 shutdown_connections 2023-10-26T06:45:59.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f800a15e0 0x7f3f800a3aa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:59.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f981008a0 0x7f3f98072af0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f981057c0 0x7f3f98073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.146+0000 7f3f9c8c3700 1 --2- 172.21.15.89:0/4285466921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f98101260 0x7f3f98073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:45:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.147+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 >> 172.21.15.89:0/4285466921 conn(0x7f3f980779e0 msgr2=0x7f3f980fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:45:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.147+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 shutdown_connections 2023-10-26T06:45:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:45:59.147+0000 7f3f9c8c3700 1 -- 172.21.15.89:0/4285466921 wait complete. 2023-10-26T06:45:59.155 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-26T06:45:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:45:59 smithi135 ceph-mon[131403]: pgmap v280: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:45:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:45:59 smithi089 ceph-mon[95031]: pgmap v280: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:45:59.564 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-10-26T06:45:57.726489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649987,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":247,"num_read_kb":233,"num_write":251,"num_write_kb":3017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336832,"kb_used_data":12624,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604992,"statfs":{"total":1151940427776,"available":1151595511808,"internally_reserved":0,"allocated":12926976,"data_stored":7283686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003979"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600110+0000","last_change":"2023-10-26T06:45:05.903170+0000","last_active":"2023-10-26T06:45:08.600110+0000","last_peered":"2023-10-26T06:45:08.600110+0000","last_clean":"2023-10-26T06:45:08.600110+0000","last_became_active":"2023-10-26T06:45:05.902574+0000","last_became_peered":"2023-10-26T06:45:05.902574+0000","last_unstale":"2023-10-26T06:45:08.600110+0000","last_undegraded":"2023-10-26T06:45:08.600110+0000","last_fullsized":"2023-10-26T06:45:08.600110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:33.065828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505541+0000","last_change":"2023-10-26T06:45:05.911249+0000","last_active":"2023-10-26T06:45:07.505541+0000","last_peered":"2023-10-26T06:45:07.505541+0000","last_clean":"2023-10-26T06:45:07.505541+0000","last_became_active":"2023-10-26T06:45:05.910859+0000","last_became_peered":"2023-10-26T06:45:05.910859+0000","last_unstale":"2023-10-26T06:45:07.505541+0000","last_undegraded":"2023-10-26T06:45:07.505541+0000","last_fullsized":"2023-10-26T06:45:07.505541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:07:13.362460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505391+0000","last_change":"2023-10-26T06:45:05.909219+0000","last_active":"2023-10-26T06:45:07.505391+0000","last_peered":"2023-10-26T06:45:07.505391+0000","last_clean":"2023-10-26T06:45:07.505391+0000","last_became_active":"2023-10-26T06:45:05.906721+0000","last_became_peered":"2023-10-26T06:45:05.906721+0000","last_unstale":"2023-10-26T06:45:07.505391+0000","last_undegraded":"2023-10-26T06:45:07.505391+0000","last_fullsized":"2023-10-26T06:45:07.505391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:41.461357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823998+0000","last_change":"2023-10-26T06:45:05.897514+0000","last_active":"2023-10-26T06:45:07.823998+0000","last_peered":"2023-10-26T06:45:07.823998+0000","last_clean":"2023-10-26T06:45:07.823998+0000","last_became_active":"2023-10-26T06:45:05.897201+0000","last_became_peered":"2023-10-26T06:45:05.897201+0000","last_unstale":"2023-10-26T06:45:07.823998+0000","last_undegraded":"2023-10-26T06:45:07.823998+0000","last_fullsized":"2023-10-26T06:45:07.823998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:39.508654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689213+0000","last_change":"2023-10-26T06:45:05.900475+0000","last_active":"2023-10-26T06:45:07.689213+0000","last_peered":"2023-10-26T06:45:07.689213+0000","last_clean":"2023-10-26T06:45:07.689213+0000","last_became_active":"2023-10-26T06:45:05.900313+0000","last_became_peered":"2023-10-26T06:45:05.900313+0000","last_unstale":"2023-10-26T06:45:07.689213+0000","last_undegraded":"2023-10-26T06:45:07.689213+0000","last_fullsized":"2023-10-26T06:45:07.689213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:51.438741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823465+0000","last_change":"2023-10-26T06:45:05.909040+0000","last_active":"2023-10-26T06:45:07.823465+0000","last_peered":"2023-10-26T06:45:07.823465+0000","last_clean":"2023-10-26T06:45:07.823465+0000","last_became_active":"2023-10-26T06:45:05.908696+0000","last_became_peered":"2023-10-26T06:45:05.908696+0000","last_unstale":"2023-10-26T06:45:07.823465+0000","last_undegraded":"2023-10-26T06:45:07.823465+0000","last_fullsized":"2023-10-26T06:45:07.823465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:00.147712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823545+0000","last_change":"2023-10-26T06:45:05.898356+0000","last_active":"2023-10-26T06:45:07.823545+0000","last_peered":"2023-10-26T06:45:07.823545+0000","last_clean":"2023-10-26T06:45:07.823545+0000","last_became_active":"2023-10-26T06:45:05.898008+0000","last_became_peered":"2023-10-26T06:45:05.898008+0000","last_unstale":"2023-10-26T06:45:07.823545+0000","last_undegraded":"2023-10-26T06:45:07.823545+0000","last_fullsized":"2023-10-26T06:45:07.823545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:23.209987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137846+0000","last_change":"2023-10-26T06:45:05.911301+0000","last_active":"2023-10-26T06:45:07.137846+0000","last_peered":"2023-10-26T06:45:07.137846+0000","last_clean":"2023-10-26T06:45:07.137846+0000","last_became_active":"2023-10-26T06:45:05.910667+0000","last_became_peered":"2023-10-26T06:45:05.910667+0000","last_unstale":"2023-10-26T06:45:07.137846+0000","last_undegraded":"2023-10-26T06:45:07.137846+0000","last_fullsized":"2023-10-26T06:45:07.137846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:03:36.856248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885616+0000","last_change":"2023-10-26T06:45:05.904456+0000","last_active":"2023-10-26T06:45:06.885616+0000","last_peered":"2023-10-26T06:45:06.885616+0000","last_clean":"2023-10-26T06:45:06.885616+0000","last_became_active":"2023-10-26T06:45:05.904165+0000","last_became_peered":"2023-10-26T06:45:05.904165+0000","last_unstale":"2023-10-26T06:45:06.885616+0000","last_undegraded":"2023-10-26T06:45:06.885616+0000","last_fullsized":"2023-10-26T06:45:06.885616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:51:01.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824115+0000","last_change":"2023-10-26T06:45:05.909544+0000","last_active":"2023-10-26T06:45:07.824115+0000","last_peered":"2023-10-26T06:45:07.824115+0000","last_clean":"2023-10-26T06:45:07.824115+0000","last_became_active":"2023-10-26T06:45:05.909103+0000","last_became_peered":"2023-10-26T06:45:05.909103+0000","last_unstale":"2023-10-26T06:45:07.824115+0000","last_undegraded":"2023-10-26T06:45:07.824115+0000","last_fullsized":"2023-10-26T06:45:07.824115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:18.069186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823939+0000","last_change":"2023-10-26T06:45:05.906549+0000","last_active":"2023-10-26T06:45:07.823939+0000","last_peered":"2023-10-26T06:45:07.823939+0000","last_clean":"2023-10-26T06:45:07.823939+0000","last_became_active":"2023-10-26T06:45:05.905883+0000","last_became_peered":"2023-10-26T06:45:05.905883+0000","last_unstale":"2023-10-26T06:45:07.823939+0000","last_undegraded":"2023-10-26T06:45:07.823939+0000","last_fullsized":"2023-10-26T06:45:07.823939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:54.771031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823855+0000","last_change":"2023-10-26T06:45:05.909460+0000","last_active":"2023-10-26T06:45:07.823855+0000","last_peered":"2023-10-26T06:45:07.823855+0000","last_clean":"2023-10-26T06:45:07.823855+0000","last_became_active":"2023-10-26T06:45:05.909007+0000","last_became_peered":"2023-10-26T06:45:05.909007+0000","last_unstale":"2023-10-26T06:45:07.823855+0000","last_undegraded":"2023-10-26T06:45:07.823855+0000","last_fullsized":"2023-10-26T06:45:07.823855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:45:55.817606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822132+0000","last_change":"2023-10-26T06:45:05.908305+0000","last_active":"2023-10-26T06:45:07.822132+0000","last_peered":"2023-10-26T06:45:07.822132+0000","last_clean":"2023-10-26T06:45:07.822132+0000","last_became_active":"2023-10-26T06:45:05.907931+0000","last_became_peered":"2023-10-26T06:45:05.907931+0000","last_unstale":"2023-10-26T06:45:07.822132+0000","last_undegraded":"2023-10-26T06:45:07.822132+0000","last_fullsized":"2023-10-26T06:45:07.822132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:00:43.605324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505175+0000","last_change":"2023-10-26T06:45:05.909737+0000","last_active":"2023-10-26T06:45:07.505175+0000","last_peered":"2023-10-26T06:45:07.505175+0000","last_clean":"2023-10-26T06:45:07.505175+0000","last_became_active":"2023-10-26T06:45:05.909445+0000","last_became_peered":"2023-10-26T06:45:05.909445+0000","last_unstale":"2023-10-26T06:45:07.505175+0000","last_undegraded":"2023-10-26T06:45:07.505175+0000","last_fullsized":"2023-10-26T06:45:07.505175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:38.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.504905+0000","last_change":"2023-10-26T06:45:05.907722+0000","last_active":"2023-10-26T06:45:07.504905+0000","last_peered":"2023-10-26T06:45:07.504905+0000","last_clean":"2023-10-26T06:45:07.504905+0000","last_became_active":"2023-10-26T06:45:05.905622+0000","last_became_peered":"2023-10-26T06:45:05.905622+0000","last_unstale":"2023-10-26T06:45:07.504905+0000","last_undegraded":"2023-10-26T06:45:07.504905+0000","last_fullsized":"2023-10-26T06:45:07.504905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:30:01.048032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885013+0000","last_change":"2023-10-26T06:45:05.907801+0000","last_active":"2023-10-26T06:45:06.885013+0000","last_peered":"2023-10-26T06:45:06.885013+0000","last_clean":"2023-10-26T06:45:06.885013+0000","last_became_active":"2023-10-26T06:45:05.907485+0000","last_became_peered":"2023-10-26T06:45:05.907485+0000","last_unstale":"2023-10-26T06:45:06.885013+0000","last_undegraded":"2023-10-26T06:45:06.885013+0000","last_fullsized":"2023-10-26T06:45:06.885013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:16.850992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600141+0000","last_change":"2023-10-26T06:45:05.902859+0000","last_active":"2023-10-26T06:45:08.600141+0000","last_peered":"2023-10-26T06:45:08.600141+0000","last_clean":"2023-10-26T06:45:08.600141+0000","last_became_active":"2023-10-26T06:45:05.902218+0000","last_became_peered":"2023-10-26T06:45:05.902218+0000","last_unstale":"2023-10-26T06:45:08.600141+0000","last_undegraded":"2023-10-26T06:45:08.600141+0000","last_fullsized":"2023-10-26T06:45:08.600141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:47.717227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884920+0000","last_change":"2023-10-26T06:45:05.910445+0000","last_active":"2023-10-26T06:45:06.884920+0000","last_peered":"2023-10-26T06:45:06.884920+0000","last_clean":"2023-10-26T06:45:06.884920+0000","last_became_active":"2023-10-26T06:45:05.910043+0000","last_became_peered":"2023-10-26T06:45:05.910043+0000","last_unstale":"2023-10-26T06:45:06.884920+0000","last_undegraded":"2023-10-26T06:45:06.884920+0000","last_fullsized":"2023-10-26T06:45:06.884920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:33.751134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884907+0000","last_change":"2023-10-26T06:45:05.905340+0000","last_active":"2023-10-26T06:45:06.884907+0000","last_peered":"2023-10-26T06:45:06.884907+0000","last_clean":"2023-10-26T06:45:06.884907+0000","last_became_active":"2023-10-26T06:45:05.905030+0000","last_became_peered":"2023-10-26T06:45:05.905030+0000","last_unstale":"2023-10-26T06:45:06.884907+0000","last_undegraded":"2023-10-26T06:45:06.884907+0000","last_fullsized":"2023-10-26T06:45:06.884907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.303210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884956+0000","last_change":"2023-10-26T06:45:04.919128+0000","last_active":"2023-10-26T06:45:06.884956+0000","last_peered":"2023-10-26T06:45:06.884956+0000","last_clean":"2023-10-26T06:45:06.884956+0000","last_became_active":"2023-10-26T06:45:04.913034+0000","last_became_peered":"2023-10-26T06:45:04.913034+0000","last_unstale":"2023-10-26T06:45:06.884956+0000","last_undegraded":"2023-10-26T06:45:06.884956+0000","last_fullsized":"2023-10-26T06:45:06.884956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:50.835037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138323+0000","last_change":"2023-10-26T06:45:05.905315+0000","last_active":"2023-10-26T06:45:07.138323+0000","last_peered":"2023-10-26T06:45:07.138323+0000","last_clean":"2023-10-26T06:45:07.138323+0000","last_became_active":"2023-10-26T06:45:05.904260+0000","last_became_peered":"2023-10-26T06:45:05.904260+0000","last_unstale":"2023-10-26T06:45:07.138323+0000","last_undegraded":"2023-10-26T06:45:07.138323+0000","last_fullsized":"2023-10-26T06:45:07.138323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:02.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.080681+0000","last_change":"2023-10-26T06:45:04.920440+0000","last_active":"2023-10-26T06:45:07.080681+0000","last_peered":"2023-10-26T06:45:07.080681+0000","last_clean":"2023-10-26T06:45:07.080681+0000","last_became_active":"2023-10-26T06:45:04.920129+0000","last_became_peered":"2023-10-26T06:45:04.920129+0000","last_unstale":"2023-10-26T06:45:07.080681+0000","last_undegraded":"2023-10-26T06:45:07.080681+0000","last_fullsized":"2023-10-26T06:45:07.080681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:43.176830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505711+0000","last_change":"2023-10-26T06:45:05.902898+0000","last_active":"2023-10-26T06:45:07.505711+0000","last_peered":"2023-10-26T06:45:07.505711+0000","last_clean":"2023-10-26T06:45:07.505711+0000","last_became_active":"2023-10-26T06:45:05.902161+0000","last_became_peered":"2023-10-26T06:45:05.902161+0000","last_unstale":"2023-10-26T06:45:07.505711+0000","last_undegraded":"2023-10-26T06:45:07.505711+0000","last_fullsized":"2023-10-26T06:45:07.505711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:27.872225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063544+0000","last_change":"2023-10-26T06:45:04.911835+0000","last_active":"2023-10-26T06:45:06.063544+0000","last_peered":"2023-10-26T06:45:06.063544+0000","last_clean":"2023-10-26T06:45:06.063544+0000","last_became_active":"2023-10-26T06:45:04.911505+0000","last_became_peered":"2023-10-26T06:45:04.911505+0000","last_unstale":"2023-10-26T06:45:06.063544+0000","last_undegraded":"2023-10-26T06:45:06.063544+0000","last_fullsized":"2023-10-26T06:45:06.063544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:01:18.971381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885164+0000","last_change":"2023-10-26T06:45:05.909088+0000","last_active":"2023-10-26T06:45:06.885164+0000","last_peered":"2023-10-26T06:45:06.885164+0000","last_clean":"2023-10-26T06:45:06.885164+0000","last_became_active":"2023-10-26T06:45:05.908528+0000","last_became_peered":"2023-10-26T06:45:05.908528+0000","last_unstale":"2023-10-26T06:45:06.885164+0000","last_undegraded":"2023-10-26T06:45:06.885164+0000","last_fullsized":"2023-10-26T06:45:06.885164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:07.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063027+0000","last_change":"2023-10-26T06:45:04.913604+0000","last_active":"2023-10-26T06:45:06.063027+0000","last_peered":"2023-10-26T06:45:06.063027+0000","last_clean":"2023-10-26T06:45:06.063027+0000","last_became_active":"2023-10-26T06:45:04.913318+0000","last_became_peered":"2023-10-26T06:45:04.913318+0000","last_unstale":"2023-10-26T06:45:06.063027+0000","last_undegraded":"2023-10-26T06:45:06.063027+0000","last_fullsized":"2023-10-26T06:45:06.063027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:19:51.144044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822837+0000","last_change":"2023-10-26T06:45:05.898795+0000","last_active":"2023-10-26T06:45:07.822837+0000","last_peered":"2023-10-26T06:45:07.822837+0000","last_clean":"2023-10-26T06:45:07.822837+0000","last_became_active":"2023-10-26T06:45:05.898489+0000","last_became_peered":"2023-10-26T06:45:05.898489+0000","last_unstale":"2023-10-26T06:45:07.822837+0000","last_undegraded":"2023-10-26T06:45:07.822837+0000","last_fullsized":"2023-10-26T06:45:07.822837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:55.343736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062447+0000","last_change":"2023-10-26T06:45:04.908881+0000","last_active":"2023-10-26T06:45:06.062447+0000","last_peered":"2023-10-26T06:45:06.062447+0000","last_clean":"2023-10-26T06:45:06.062447+0000","last_became_active":"2023-10-26T06:45:04.908515+0000","last_became_peered":"2023-10-26T06:45:04.908515+0000","last_unstale":"2023-10-26T06:45:06.062447+0000","last_undegraded":"2023-10-26T06:45:06.062447+0000","last_fullsized":"2023-10-26T06:45:06.062447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:39.200294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689203+0000","last_change":"2023-10-26T06:45:05.901281+0000","last_active":"2023-10-26T06:45:07.689203+0000","last_peered":"2023-10-26T06:45:07.689203+0000","last_clean":"2023-10-26T06:45:07.689203+0000","last_became_active":"2023-10-26T06:45:05.900995+0000","last_became_peered":"2023-10-26T06:45:05.900995+0000","last_unstale":"2023-10-26T06:45:07.689203+0000","last_undegraded":"2023-10-26T06:45:07.689203+0000","last_fullsized":"2023-10-26T06:45:07.689203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:44:34.977571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.285852+0000","last_change":"2023-10-26T06:45:04.918480+0000","last_active":"2023-10-26T06:45:07.285852+0000","last_peered":"2023-10-26T06:45:07.285852+0000","last_clean":"2023-10-26T06:45:07.285852+0000","last_became_active":"2023-10-26T06:45:04.918185+0000","last_became_peered":"2023-10-26T06:45:04.918185+0000","last_unstale":"2023-10-26T06:45:07.285852+0000","last_undegraded":"2023-10-26T06:45:07.285852+0000","last_fullsized":"2023-10-26T06:45:07.285852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:07:58.263116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822711+0000","last_change":"2023-10-26T06:45:05.901825+0000","last_active":"2023-10-26T06:45:07.822711+0000","last_peered":"2023-10-26T06:45:07.822711+0000","last_clean":"2023-10-26T06:45:07.822711+0000","last_became_active":"2023-10-26T06:45:05.901509+0000","last_became_peered":"2023-10-26T06:45:05.901509+0000","last_unstale":"2023-10-26T06:45:07.822711+0000","last_undegraded":"2023-10-26T06:45:07.822711+0000","last_fullsized":"2023-10-26T06:45:07.822711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:16.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534260+0000","last_change":"2023-10-26T06:45:04.919813+0000","last_active":"2023-10-26T06:45:06.534260+0000","last_peered":"2023-10-26T06:45:06.534260+0000","last_clean":"2023-10-26T06:45:06.534260+0000","last_became_active":"2023-10-26T06:45:04.918016+0000","last_became_peered":"2023-10-26T06:45:04.918016+0000","last_unstale":"2023-10-26T06:45:06.534260+0000","last_undegraded":"2023-10-26T06:45:06.534260+0000","last_fullsized":"2023-10-26T06:45:06.534260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:32.336126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137929+0000","last_change":"2023-10-26T06:45:05.902182+0000","last_active":"2023-10-26T06:45:07.137929+0000","last_peered":"2023-10-26T06:45:07.137929+0000","last_clean":"2023-10-26T06:45:07.137929+0000","last_became_active":"2023-10-26T06:45:05.901428+0000","last_became_peered":"2023-10-26T06:45:05.901428+0000","last_unstale":"2023-10-26T06:45:07.137929+0000","last_undegraded":"2023-10-26T06:45:07.137929+0000","last_fullsized":"2023-10-26T06:45:07.137929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:12.722532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.890140+0000","last_change":"2023-10-26T06:45:04.909713+0000","last_active":"2023-10-26T06:45:05.890140+0000","last_peered":"2023-10-26T06:45:05.890140+0000","last_clean":"2023-10-26T06:45:05.890140+0000","last_became_active":"2023-10-26T06:45:04.909412+0000","last_became_peered":"2023-10-26T06:45:04.909412+0000","last_unstale":"2023-10-26T06:45:05.890140+0000","last_undegraded":"2023-10-26T06:45:05.890140+0000","last_fullsized":"2023-10-26T06:45:05.890140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:23.901745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884924+0000","last_change":"2023-10-26T06:45:05.908348+0000","last_active":"2023-10-26T06:45:06.884924+0000","last_peered":"2023-10-26T06:45:06.884924+0000","last_clean":"2023-10-26T06:45:06.884924+0000","last_became_active":"2023-10-26T06:45:05.908022+0000","last_became_peered":"2023-10-26T06:45:05.908022+0000","last_unstale":"2023-10-26T06:45:06.884924+0000","last_undegraded":"2023-10-26T06:45:06.884924+0000","last_fullsized":"2023-10-26T06:45:06.884924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:22:16.027732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.285938+0000","last_change":"2023-10-26T06:45:04.915491+0000","last_active":"2023-10-26T06:45:07.285938+0000","last_peered":"2023-10-26T06:45:07.285938+0000","last_clean":"2023-10-26T06:45:07.285938+0000","last_became_active":"2023-10-26T06:45:04.915191+0000","last_became_peered":"2023-10-26T06:45:04.915191+0000","last_unstale":"2023-10-26T06:45:07.285938+0000","last_undegraded":"2023-10-26T06:45:07.285938+0000","last_fullsized":"2023-10-26T06:45:07.285938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:57.758903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885545+0000","last_change":"2023-10-26T06:45:05.912676+0000","last_active":"2023-10-26T06:45:06.885545+0000","last_peered":"2023-10-26T06:45:06.885545+0000","last_clean":"2023-10-26T06:45:06.885545+0000","last_became_active":"2023-10-26T06:45:05.912372+0000","last_became_peered":"2023-10-26T06:45:05.912372+0000","last_unstale":"2023-10-26T06:45:06.885545+0000","last_undegraded":"2023-10-26T06:45:06.885545+0000","last_fullsized":"2023-10-26T06:45:06.885545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:33.299166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502215+0000","last_change":"2023-10-26T06:45:04.912123+0000","last_active":"2023-10-26T06:45:07.502215+0000","last_peered":"2023-10-26T06:45:07.502215+0000","last_clean":"2023-10-26T06:45:07.502215+0000","last_became_active":"2023-10-26T06:45:04.911893+0000","last_became_peered":"2023-10-26T06:45:04.911893+0000","last_unstale":"2023-10-26T06:45:07.502215+0000","last_undegraded":"2023-10-26T06:45:07.502215+0000","last_fullsized":"2023-10-26T06:45:07.502215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:32.907496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505559+0000","last_change":"2023-10-26T06:45:05.893434+0000","last_active":"2023-10-26T06:45:07.505559+0000","last_peered":"2023-10-26T06:45:07.505559+0000","last_clean":"2023-10-26T06:45:07.505559+0000","last_became_active":"2023-10-26T06:45:05.893100+0000","last_became_peered":"2023-10-26T06:45:05.893100+0000","last_unstale":"2023-10-26T06:45:07.505559+0000","last_undegraded":"2023-10-26T06:45:07.505559+0000","last_fullsized":"2023-10-26T06:45:07.505559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:22:48.317428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062642+0000","last_change":"2023-10-26T06:45:04.908910+0000","last_active":"2023-10-26T06:45:06.062642+0000","last_peered":"2023-10-26T06:45:06.062642+0000","last_clean":"2023-10-26T06:45:06.062642+0000","last_became_active":"2023-10-26T06:45:04.908592+0000","last_became_peered":"2023-10-26T06:45:04.908592+0000","last_unstale":"2023-10-26T06:45:06.062642+0000","last_undegraded":"2023-10-26T06:45:06.062642+0000","last_fullsized":"2023-10-26T06:45:06.062642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:40.266013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885272+0000","last_change":"2023-10-26T06:45:05.908887+0000","last_active":"2023-10-26T06:45:06.885272+0000","last_peered":"2023-10-26T06:45:06.885272+0000","last_clean":"2023-10-26T06:45:06.885272+0000","last_became_active":"2023-10-26T06:45:05.908163+0000","last_became_peered":"2023-10-26T06:45:05.908163+0000","last_unstale":"2023-10-26T06:45:06.885272+0000","last_undegraded":"2023-10-26T06:45:06.885272+0000","last_fullsized":"2023-10-26T06:45:06.885272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:09.339374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'25","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:57.165190+0000","last_change":"2023-10-26T06:45:04.905691+0000","last_active":"2023-10-26T06:45:57.165190+0000","last_peered":"2023-10-26T06:45:57.165190+0000","last_clean":"2023-10-26T06:45:57.165190+0000","last_became_active":"2023-10-26T06:45:04.905378+0000","last_became_peered":"2023-10-26T06:45:04.905378+0000","last_unstale":"2023-10-26T06:45:57.165190+0000","last_undegraded":"2023-10-26T06:45:57.165190+0000","last_fullsized":"2023-10-26T06:45:57.165190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:24:21.498141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58575,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":108,"num_write":25,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689388+0000","last_change":"2023-10-26T06:45:05.897366+0000","last_active":"2023-10-26T06:45:07.689388+0000","last_peered":"2023-10-26T06:45:07.689388+0000","last_clean":"2023-10-26T06:45:07.689388+0000","last_became_active":"2023-10-26T06:45:05.897091+0000","last_became_peered":"2023-10-26T06:45:05.897091+0000","last_unstale":"2023-10-26T06:45:07.689388+0000","last_undegraded":"2023-10-26T06:45:07.689388+0000","last_fullsized":"2023-10-26T06:45:07.689388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:11.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884895+0000","last_change":"2023-10-26T06:45:04.914949+0000","last_active":"2023-10-26T06:45:06.884895+0000","last_peered":"2023-10-26T06:45:06.884895+0000","last_clean":"2023-10-26T06:45:06.884895+0000","last_became_active":"2023-10-26T06:45:04.914610+0000","last_became_peered":"2023-10-26T06:45:04.914610+0000","last_unstale":"2023-10-26T06:45:06.884895+0000","last_undegraded":"2023-10-26T06:45:06.884895+0000","last_fullsized":"2023-10-26T06:45:06.884895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:16.683352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822001+0000","last_change":"2023-10-26T06:45:05.911067+0000","last_active":"2023-10-26T06:45:07.822001+0000","last_peered":"2023-10-26T06:45:07.822001+0000","last_clean":"2023-10-26T06:45:07.822001+0000","last_became_active":"2023-10-26T06:45:05.908460+0000","last_became_peered":"2023-10-26T06:45:05.908460+0000","last_unstale":"2023-10-26T06:45:07.822001+0000","last_undegraded":"2023-10-26T06:45:07.822001+0000","last_fullsized":"2023-10-26T06:45:07.822001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:17.461781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.895349+0000","last_change":"2023-10-26T06:45:04.919947+0000","last_active":"2023-10-26T06:45:05.895349+0000","last_peered":"2023-10-26T06:45:05.895349+0000","last_clean":"2023-10-26T06:45:05.895349+0000","last_became_active":"2023-10-26T06:45:04.919535+0000","last_became_peered":"2023-10-26T06:45:04.919535+0000","last_unstale":"2023-10-26T06:45:05.895349+0000","last_undegraded":"2023-10-26T06:45:05.895349+0000","last_fullsized":"2023-10-26T06:45:05.895349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:46:09.201139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823839+0000","last_change":"2023-10-26T06:45:05.906381+0000","last_active":"2023-10-26T06:45:07.823839+0000","last_peered":"2023-10-26T06:45:07.823839+0000","last_clean":"2023-10-26T06:45:07.823839+0000","last_became_active":"2023-10-26T06:45:05.904487+0000","last_became_peered":"2023-10-26T06:45:05.904487+0000","last_unstale":"2023-10-26T06:45:07.823839+0000","last_undegraded":"2023-10-26T06:45:07.823839+0000","last_fullsized":"2023-10-26T06:45:07.823839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:00.164936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887797+0000","last_change":"2023-10-26T06:45:04.908779+0000","last_active":"2023-10-26T06:45:05.887797+0000","last_peered":"2023-10-26T06:45:05.887797+0000","last_clean":"2023-10-26T06:45:05.887797+0000","last_became_active":"2023-10-26T06:45:04.908490+0000","last_became_peered":"2023-10-26T06:45:04.908490+0000","last_unstale":"2023-10-26T06:45:05.887797+0000","last_undegraded":"2023-10-26T06:45:05.887797+0000","last_fullsized":"2023-10-26T06:45:05.887797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:12:13.199863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823130+0000","last_change":"2023-10-26T06:45:05.909408+0000","last_active":"2023-10-26T06:45:07.823130+0000","last_peered":"2023-10-26T06:45:07.823130+0000","last_clean":"2023-10-26T06:45:07.823130+0000","last_became_active":"2023-10-26T06:45:05.909107+0000","last_became_peered":"2023-10-26T06:45:05.909107+0000","last_unstale":"2023-10-26T06:45:07.823130+0000","last_undegraded":"2023-10-26T06:45:07.823130+0000","last_fullsized":"2023-10-26T06:45:07.823130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:05.887472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063862+0000","last_change":"2023-10-26T06:45:04.918194+0000","last_active":"2023-10-26T06:45:06.063862+0000","last_peered":"2023-10-26T06:45:06.063862+0000","last_clean":"2023-10-26T06:45:06.063862+0000","last_became_active":"2023-10-26T06:45:04.917850+0000","last_became_peered":"2023-10-26T06:45:04.917850+0000","last_unstale":"2023-10-26T06:45:06.063862+0000","last_undegraded":"2023-10-26T06:45:06.063862+0000","last_fullsized":"2023-10-26T06:45:06.063862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:25.743939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822581+0000","last_change":"2023-10-26T06:45:05.902156+0000","last_active":"2023-10-26T06:45:07.822581+0000","last_peered":"2023-10-26T06:45:07.822581+0000","last_clean":"2023-10-26T06:45:07.822581+0000","last_became_active":"2023-10-26T06:45:05.901865+0000","last_became_peered":"2023-10-26T06:45:05.901865+0000","last_unstale":"2023-10-26T06:45:07.822581+0000","last_undegraded":"2023-10-26T06:45:07.822581+0000","last_fullsized":"2023-10-26T06:45:07.822581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:06:08.167280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689452+0000","last_change":"2023-10-26T06:45:04.913071+0000","last_active":"2023-10-26T06:45:07.689452+0000","last_peered":"2023-10-26T06:45:07.689452+0000","last_clean":"2023-10-26T06:45:07.689452+0000","last_became_active":"2023-10-26T06:45:04.912736+0000","last_became_peered":"2023-10-26T06:45:04.912736+0000","last_unstale":"2023-10-26T06:45:07.689452+0000","last_undegraded":"2023-10-26T06:45:07.689452+0000","last_fullsized":"2023-10-26T06:45:07.689452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:59.690355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600012+0000","last_change":"2023-10-26T06:45:05.891949+0000","last_active":"2023-10-26T06:45:08.600012+0000","last_peered":"2023-10-26T06:45:08.600012+0000","last_clean":"2023-10-26T06:45:08.600012+0000","last_became_active":"2023-10-26T06:45:05.891583+0000","last_became_peered":"2023-10-26T06:45:05.891583+0000","last_unstale":"2023-10-26T06:45:08.600012+0000","last_undegraded":"2023-10-26T06:45:08.600012+0000","last_fullsized":"2023-10-26T06:45:08.600012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:54:18.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.883535+0000","last_change":"2023-10-26T06:45:04.913969+0000","last_active":"2023-10-26T06:45:05.883535+0000","last_peered":"2023-10-26T06:45:05.883535+0000","last_clean":"2023-10-26T06:45:05.883535+0000","last_became_active":"2023-10-26T06:45:04.913533+0000","last_became_peered":"2023-10-26T06:45:04.913533+0000","last_unstale":"2023-10-26T06:45:05.883535+0000","last_undegraded":"2023-10-26T06:45:05.883535+0000","last_fullsized":"2023-10-26T06:45:05.883535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:47:38.755102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824112+0000","last_change":"2023-10-26T06:45:05.904832+0000","last_active":"2023-10-26T06:45:07.824112+0000","last_peered":"2023-10-26T06:45:07.824112+0000","last_clean":"2023-10-26T06:45:07.824112+0000","last_became_active":"2023-10-26T06:45:05.904527+0000","last_became_peered":"2023-10-26T06:45:05.904527+0000","last_unstale":"2023-10-26T06:45:07.824112+0000","last_undegraded":"2023-10-26T06:45:07.824112+0000","last_fullsized":"2023-10-26T06:45:07.824112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:30.546198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502122+0000","last_change":"2023-10-26T06:45:04.922832+0000","last_active":"2023-10-26T06:45:07.502122+0000","last_peered":"2023-10-26T06:45:07.502122+0000","last_clean":"2023-10-26T06:45:07.502122+0000","last_became_active":"2023-10-26T06:45:04.922349+0000","last_became_peered":"2023-10-26T06:45:04.922349+0000","last_unstale":"2023-10-26T06:45:07.502122+0000","last_undegraded":"2023-10-26T06:45:07.502122+0000","last_fullsized":"2023-10-26T06:45:07.502122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:37.839093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823836+0000","last_change":"2023-10-26T06:45:05.909946+0000","last_active":"2023-10-26T06:45:07.823836+0000","last_peered":"2023-10-26T06:45:07.823836+0000","last_clean":"2023-10-26T06:45:07.823836+0000","last_became_active":"2023-10-26T06:45:05.909650+0000","last_became_peered":"2023-10-26T06:45:05.909650+0000","last_unstale":"2023-10-26T06:45:07.823836+0000","last_undegraded":"2023-10-26T06:45:07.823836+0000","last_fullsized":"2023-10-26T06:45:07.823836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:41.813928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822476+0000","last_change":"2023-10-26T06:45:05.903400+0000","last_active":"2023-10-26T06:45:07.822476+0000","last_peered":"2023-10-26T06:45:07.822476+0000","last_clean":"2023-10-26T06:45:07.822476+0000","last_became_active":"2023-10-26T06:45:05.903101+0000","last_became_peered":"2023-10-26T06:45:05.903101+0000","last_unstale":"2023-10-26T06:45:07.822476+0000","last_undegraded":"2023-10-26T06:45:07.822476+0000","last_fullsized":"2023-10-26T06:45:07.822476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534251+0000","last_change":"2023-10-26T06:45:04.923960+0000","last_active":"2023-10-26T06:45:06.534251+0000","last_peered":"2023-10-26T06:45:06.534251+0000","last_clean":"2023-10-26T06:45:06.534251+0000","last_became_active":"2023-10-26T06:45:04.923613+0000","last_became_peered":"2023-10-26T06:45:04.923613+0000","last_unstale":"2023-10-26T06:45:06.534251+0000","last_undegraded":"2023-10-26T06:45:06.534251+0000","last_fullsized":"2023-10-26T06:45:06.534251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:53.342463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138101+0000","last_change":"2023-10-26T06:45:05.903966+0000","last_active":"2023-10-26T06:45:07.138101+0000","last_peered":"2023-10-26T06:45:07.138101+0000","last_clean":"2023-10-26T06:45:07.138101+0000","last_became_active":"2023-10-26T06:45:05.903583+0000","last_became_peered":"2023-10-26T06:45:05.903583+0000","last_unstale":"2023-10-26T06:45:07.138101+0000","last_undegraded":"2023-10-26T06:45:07.138101+0000","last_fullsized":"2023-10-26T06:45:07.138101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:18.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887855+0000","last_change":"2023-10-26T06:45:04.907838+0000","last_active":"2023-10-26T06:45:05.887855+0000","last_peered":"2023-10-26T06:45:05.887855+0000","last_clean":"2023-10-26T06:45:05.887855+0000","last_became_active":"2023-10-26T06:45:04.907508+0000","last_became_peered":"2023-10-26T06:45:04.907508+0000","last_unstale":"2023-10-26T06:45:05.887855+0000","last_undegraded":"2023-10-26T06:45:05.887855+0000","last_fullsized":"2023-10-26T06:45:05.887855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:34.852243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":319,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.892156+0000","last_change":"2023-10-26T06:43:59.765466+0000","last_active":"2023-10-26T06:45:05.892156+0000","last_peered":"2023-10-26T06:45:05.892156+0000","last_clean":"2023-10-26T06:45:05.892156+0000","last_became_active":"2023-10-26T06:43:59.765005+0000","last_became_peered":"2023-10-26T06:43:59.765005+0000","last_unstale":"2023-10-26T06:45:05.892156+0000","last_undegraded":"2023-10-26T06:45:05.892156+0000","last_fullsized":"2023-10-26T06:45:05.892156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_clean_scrub_stamp":"2023-10-26T06:40:50.556448+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:18.688941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.821877+0000","last_change":"2023-10-26T06:45:05.906909+0000","last_active":"2023-10-26T06:45:07.821877+0000","last_peered":"2023-10-26T06:45:07.821877+0000","last_clean":"2023-10-26T06:45:07.821877+0000","last_became_active":"2023-10-26T06:45:05.906531+0000","last_became_peered":"2023-10-26T06:45:05.906531+0000","last_unstale":"2023-10-26T06:45:07.821877+0000","last_undegraded":"2023-10-26T06:45:07.821877+0000","last_fullsized":"2023-10-26T06:45:07.821877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502092+0000","last_change":"2023-10-26T06:45:04.911635+0000","last_active":"2023-10-26T06:45:07.502092+0000","last_peered":"2023-10-26T06:45:07.502092+0000","last_clean":"2023-10-26T06:45:07.502092+0000","last_became_active":"2023-10-26T06:45:04.911289+0000","last_became_peered":"2023-10-26T06:45:04.911289+0000","last_unstale":"2023-10-26T06:45:07.502092+0000","last_undegraded":"2023-10-26T06:45:07.502092+0000","last_fullsized":"2023-10-26T06:45:07.502092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.115211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888625+0000","last_change":"2023-10-26T06:45:04.919853+0000","last_active":"2023-10-26T06:45:05.888625+0000","last_peered":"2023-10-26T06:45:05.888625+0000","last_clean":"2023-10-26T06:45:05.888625+0000","last_became_active":"2023-10-26T06:45:04.919488+0000","last_became_peered":"2023-10-26T06:45:04.919488+0000","last_unstale":"2023-10-26T06:45:05.888625+0000","last_undegraded":"2023-10-26T06:45:05.888625+0000","last_fullsized":"2023-10-26T06:45:05.888625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:06.555851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822290+0000","last_change":"2023-10-26T06:45:05.903975+0000","last_active":"2023-10-26T06:45:07.822290+0000","last_peered":"2023-10-26T06:45:07.822290+0000","last_clean":"2023-10-26T06:45:07.822290+0000","last_became_active":"2023-10-26T06:45:05.903539+0000","last_became_peered":"2023-10-26T06:45:05.903539+0000","last_unstale":"2023-10-26T06:45:07.822290+0000","last_undegraded":"2023-10-26T06:45:07.822290+0000","last_fullsized":"2023-10-26T06:45:07.822290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:30.648947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'3","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:13.944339+0000","last_change":"2023-10-26T06:45:04.919815+0000","last_active":"2023-10-26T06:45:13.944339+0000","last_peered":"2023-10-26T06:45:13.944339+0000","last_clean":"2023-10-26T06:45:13.944339+0000","last_became_active":"2023-10-26T06:45:04.918141+0000","last_became_peered":"2023-10-26T06:45:04.918141+0000","last_unstale":"2023-10-26T06:45:13.944339+0000","last_undegraded":"2023-10-26T06:45:13.944339+0000","last_fullsized":"2023-10-26T06:45:13.944339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:53.024534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138397+0000","last_change":"2023-10-26T06:45:05.898180+0000","last_active":"2023-10-26T06:45:07.138397+0000","last_peered":"2023-10-26T06:45:07.138397+0000","last_clean":"2023-10-26T06:45:07.138397+0000","last_became_active":"2023-10-26T06:45:05.896937+0000","last_became_peered":"2023-10-26T06:45:05.896937+0000","last_unstale":"2023-10-26T06:45:07.138397+0000","last_undegraded":"2023-10-26T06:45:07.138397+0000","last_fullsized":"2023-10-26T06:45:07.138397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:48.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.061365+0000","last_change":"2023-10-26T06:45:04.922456+0000","last_active":"2023-10-26T06:45:06.061365+0000","last_peered":"2023-10-26T06:45:06.061365+0000","last_clean":"2023-10-26T06:45:06.061365+0000","last_became_active":"2023-10-26T06:45:04.922112+0000","last_became_peered":"2023-10-26T06:45:04.922112+0000","last_unstale":"2023-10-26T06:45:06.061365+0000","last_undegraded":"2023-10-26T06:45:06.061365+0000","last_fullsized":"2023-10-26T06:45:06.061365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:44.055569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505613+0000","last_change":"2023-10-26T06:45:05.895440+0000","last_active":"2023-10-26T06:45:07.505613+0000","last_peered":"2023-10-26T06:45:07.505613+0000","last_clean":"2023-10-26T06:45:07.505613+0000","last_became_active":"2023-10-26T06:45:05.892605+0000","last_became_peered":"2023-10-26T06:45:05.892605+0000","last_unstale":"2023-10-26T06:45:07.505613+0000","last_undegraded":"2023-10-26T06:45:07.505613+0000","last_fullsized":"2023-10-26T06:45:07.505613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:09:27.392772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.899985+0000","last_change":"2023-10-26T06:45:04.916042+0000","last_active":"2023-10-26T06:45:05.899985+0000","last_peered":"2023-10-26T06:45:05.899985+0000","last_clean":"2023-10-26T06:45:05.899985+0000","last_became_active":"2023-10-26T06:45:04.915649+0000","last_became_peered":"2023-10-26T06:45:04.915649+0000","last_unstale":"2023-10-26T06:45:05.899985+0000","last_undegraded":"2023-10-26T06:45:05.899985+0000","last_fullsized":"2023-10-26T06:45:05.899985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:30.060074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689096+0000","last_change":"2023-10-26T06:45:05.908179+0000","last_active":"2023-10-26T06:45:07.689096+0000","last_peered":"2023-10-26T06:45:07.689096+0000","last_clean":"2023-10-26T06:45:07.689096+0000","last_became_active":"2023-10-26T06:45:05.907857+0000","last_became_peered":"2023-10-26T06:45:05.907857+0000","last_unstale":"2023-10-26T06:45:07.689096+0000","last_undegraded":"2023-10-26T06:45:07.689096+0000","last_fullsized":"2023-10-26T06:45:07.689096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:20.017194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.080625+0000","last_change":"2023-10-26T06:45:04.921016+0000","last_active":"2023-10-26T06:45:07.080625+0000","last_peered":"2023-10-26T06:45:07.080625+0000","last_clean":"2023-10-26T06:45:07.080625+0000","last_became_active":"2023-10-26T06:45:04.920741+0000","last_became_peered":"2023-10-26T06:45:04.920741+0000","last_unstale":"2023-10-26T06:45:07.080625+0000","last_undegraded":"2023-10-26T06:45:07.080625+0000","last_fullsized":"2023-10-26T06:45:07.080625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:05:06.218205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599772+0000","last_change":"2023-10-26T06:45:05.893562+0000","last_active":"2023-10-26T06:45:08.599772+0000","last_peered":"2023-10-26T06:45:08.599772+0000","last_clean":"2023-10-26T06:45:08.599772+0000","last_became_active":"2023-10-26T06:45:05.893111+0000","last_became_peered":"2023-10-26T06:45:05.893111+0000","last_unstale":"2023-10-26T06:45:08.599772+0000","last_undegraded":"2023-10-26T06:45:08.599772+0000","last_fullsized":"2023-10-26T06:45:08.599772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.305664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888454+0000","last_change":"2023-10-26T06:45:04.914936+0000","last_active":"2023-10-26T06:45:05.888454+0000","last_peered":"2023-10-26T06:45:05.888454+0000","last_clean":"2023-10-26T06:45:05.888454+0000","last_became_active":"2023-10-26T06:45:04.914617+0000","last_became_peered":"2023-10-26T06:45:04.914617+0000","last_unstale":"2023-10-26T06:45:05.888454+0000","last_undegraded":"2023-10-26T06:45:05.888454+0000","last_fullsized":"2023-10-26T06:45:05.888454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:15.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138153+0000","last_change":"2023-10-26T06:45:05.909101+0000","last_active":"2023-10-26T06:45:07.138153+0000","last_peered":"2023-10-26T06:45:07.138153+0000","last_clean":"2023-10-26T06:45:07.138153+0000","last_became_active":"2023-10-26T06:45:05.908800+0000","last_became_peered":"2023-10-26T06:45:05.908800+0000","last_unstale":"2023-10-26T06:45:07.138153+0000","last_undegraded":"2023-10-26T06:45:07.138153+0000","last_fullsized":"2023-10-26T06:45:07.138153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:37:18.605102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689147+0000","last_change":"2023-10-26T06:45:04.911714+0000","last_active":"2023-10-26T06:45:07.689147+0000","last_peered":"2023-10-26T06:45:07.689147+0000","last_clean":"2023-10-26T06:45:07.689147+0000","last_became_active":"2023-10-26T06:45:04.911343+0000","last_became_peered":"2023-10-26T06:45:04.911343+0000","last_unstale":"2023-10-26T06:45:07.689147+0000","last_undegraded":"2023-10-26T06:45:07.689147+0000","last_fullsized":"2023-10-26T06:45:07.689147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:30:35.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823521+0000","last_change":"2023-10-26T06:45:05.906804+0000","last_active":"2023-10-26T06:45:07.823521+0000","last_peered":"2023-10-26T06:45:07.823521+0000","last_clean":"2023-10-26T06:45:07.823521+0000","last_became_active":"2023-10-26T06:45:05.906522+0000","last_became_peered":"2023-10-26T06:45:05.906522+0000","last_unstale":"2023-10-26T06:45:07.823521+0000","last_undegraded":"2023-10-26T06:45:07.823521+0000","last_fullsized":"2023-10-26T06:45:07.823521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:32.707204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888946+0000","last_change":"2023-10-26T06:45:04.906345+0000","last_active":"2023-10-26T06:45:05.888946+0000","last_peered":"2023-10-26T06:45:05.888946+0000","last_clean":"2023-10-26T06:45:05.888946+0000","last_became_active":"2023-10-26T06:45:04.906034+0000","last_became_peered":"2023-10-26T06:45:04.906034+0000","last_unstale":"2023-10-26T06:45:05.888946+0000","last_undegraded":"2023-10-26T06:45:05.888946+0000","last_fullsized":"2023-10-26T06:45:05.888946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:53.690788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823815+0000","last_change":"2023-10-26T06:45:05.906368+0000","last_active":"2023-10-26T06:45:07.823815+0000","last_peered":"2023-10-26T06:45:07.823815+0000","last_clean":"2023-10-26T06:45:07.823815+0000","last_became_active":"2023-10-26T06:45:05.905471+0000","last_became_peered":"2023-10-26T06:45:05.905471+0000","last_unstale":"2023-10-26T06:45:07.823815+0000","last_undegraded":"2023-10-26T06:45:07.823815+0000","last_fullsized":"2023-10-26T06:45:07.823815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:45.493704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822626+0000","last_change":"2023-10-26T06:45:04.917530+0000","last_active":"2023-10-26T06:45:07.822626+0000","last_peered":"2023-10-26T06:45:07.822626+0000","last_clean":"2023-10-26T06:45:07.822626+0000","last_became_active":"2023-10-26T06:45:04.917193+0000","last_became_peered":"2023-10-26T06:45:04.917193+0000","last_unstale":"2023-10-26T06:45:07.822626+0000","last_undegraded":"2023-10-26T06:45:07.822626+0000","last_fullsized":"2023-10-26T06:45:07.822626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.965048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823728+0000","last_change":"2023-10-26T06:45:05.899429+0000","last_active":"2023-10-26T06:45:07.823728+0000","last_peered":"2023-10-26T06:45:07.823728+0000","last_clean":"2023-10-26T06:45:07.823728+0000","last_became_active":"2023-10-26T06:45:05.899081+0000","last_became_peered":"2023-10-26T06:45:05.899081+0000","last_unstale":"2023-10-26T06:45:07.823728+0000","last_undegraded":"2023-10-26T06:45:07.823728+0000","last_fullsized":"2023-10-26T06:45:07.823728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:41.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534374+0000","last_change":"2023-10-26T06:45:04.920295+0000","last_active":"2023-10-26T06:45:06.534374+0000","last_peered":"2023-10-26T06:45:06.534374+0000","last_clean":"2023-10-26T06:45:06.534374+0000","last_became_active":"2023-10-26T06:45:04.919920+0000","last_became_peered":"2023-10-26T06:45:04.919920+0000","last_unstale":"2023-10-26T06:45:06.534374+0000","last_undegraded":"2023-10-26T06:45:06.534374+0000","last_fullsized":"2023-10-26T06:45:06.534374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:59.260841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824231+0000","last_change":"2023-10-26T06:45:05.899610+0000","last_active":"2023-10-26T06:45:07.824231+0000","last_peered":"2023-10-26T06:45:07.824231+0000","last_clean":"2023-10-26T06:45:07.824231+0000","last_became_active":"2023-10-26T06:45:05.899283+0000","last_became_peered":"2023-10-26T06:45:05.899283+0000","last_unstale":"2023-10-26T06:45:07.824231+0000","last_undegraded":"2023-10-26T06:45:07.824231+0000","last_fullsized":"2023-10-26T06:45:07.824231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:48.730703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599798+0000","last_change":"2023-10-26T06:45:05.892012+0000","last_active":"2023-10-26T06:45:08.599798+0000","last_peered":"2023-10-26T06:45:08.599798+0000","last_clean":"2023-10-26T06:45:08.599798+0000","last_became_active":"2023-10-26T06:45:05.891744+0000","last_became_peered":"2023-10-26T06:45:05.891744+0000","last_unstale":"2023-10-26T06:45:08.599798+0000","last_undegraded":"2023-10-26T06:45:08.599798+0000","last_fullsized":"2023-10-26T06:45:08.599798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:29.599184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884873+0000","last_change":"2023-10-26T06:45:05.904348+0000","last_active":"2023-10-26T06:45:06.884873+0000","last_peered":"2023-10-26T06:45:06.884873+0000","last_clean":"2023-10-26T06:45:06.884873+0000","last_became_active":"2023-10-26T06:45:05.903949+0000","last_became_peered":"2023-10-26T06:45:05.903949+0000","last_unstale":"2023-10-26T06:45:06.884873+0000","last_undegraded":"2023-10-26T06:45:06.884873+0000","last_fullsized":"2023-10-26T06:45:06.884873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:29.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884744+0000","last_change":"2023-10-26T06:45:05.901313+0000","last_active":"2023-10-26T06:45:06.884744+0000","last_peered":"2023-10-26T06:45:06.884744+0000","last_clean":"2023-10-26T06:45:06.884744+0000","last_became_active":"2023-10-26T06:45:05.900993+0000","last_became_peered":"2023-10-26T06:45:05.900993+0000","last_unstale":"2023-10-26T06:45:06.884744+0000","last_undegraded":"2023-10-26T06:45:06.884744+0000","last_fullsized":"2023-10-26T06:45:06.884744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:00.093645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505407+0000","last_change":"2023-10-26T06:45:05.901225+0000","last_active":"2023-10-26T06:45:07.505407+0000","last_peered":"2023-10-26T06:45:07.505407+0000","last_clean":"2023-10-26T06:45:07.505407+0000","last_became_active":"2023-10-26T06:45:05.900743+0000","last_became_peered":"2023-10-26T06:45:05.900743+0000","last_unstale":"2023-10-26T06:45:07.505407+0000","last_undegraded":"2023-10-26T06:45:07.505407+0000","last_fullsized":"2023-10-26T06:45:07.505407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:02.359654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822222+0000","last_change":"2023-10-26T06:45:05.906805+0000","last_active":"2023-10-26T06:45:07.822222+0000","last_peered":"2023-10-26T06:45:07.822222+0000","last_clean":"2023-10-26T06:45:07.822222+0000","last_became_active":"2023-10-26T06:45:05.906464+0000","last_became_peered":"2023-10-26T06:45:05.906464+0000","last_unstale":"2023-10-26T06:45:07.822222+0000","last_undegraded":"2023-10-26T06:45:07.822222+0000","last_fullsized":"2023-10-26T06:45:07.822222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:24.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599992+0000","last_change":"2023-10-26T06:45:05.898568+0000","last_active":"2023-10-26T06:45:08.599992+0000","last_peered":"2023-10-26T06:45:08.599992+0000","last_clean":"2023-10-26T06:45:08.599992+0000","last_became_active":"2023-10-26T06:45:05.897886+0000","last_became_peered":"2023-10-26T06:45:05.897886+0000","last_unstale":"2023-10-26T06:45:08.599992+0000","last_undegraded":"2023-10-26T06:45:08.599992+0000","last_fullsized":"2023-10-26T06:45:08.599992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:14.998233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885177+0000","last_change":"2023-10-26T06:45:05.899326+0000","last_active":"2023-10-26T06:45:06.885177+0000","last_peered":"2023-10-26T06:45:06.885177+0000","last_clean":"2023-10-26T06:45:06.885177+0000","last_became_active":"2023-10-26T06:45:05.899040+0000","last_became_peered":"2023-10-26T06:45:05.899040+0000","last_unstale":"2023-10-26T06:45:06.885177+0000","last_undegraded":"2023-10-26T06:45:06.885177+0000","last_fullsized":"2023-10-26T06:45:06.885177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:55:35.666122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138250+0000","last_change":"2023-10-26T06:45:05.904155+0000","last_active":"2023-10-26T06:45:07.138250+0000","last_peered":"2023-10-26T06:45:07.138250+0000","last_clean":"2023-10-26T06:45:07.138250+0000","last_became_active":"2023-10-26T06:45:05.903734+0000","last_became_peered":"2023-10-26T06:45:05.903734+0000","last_unstale":"2023-10-26T06:45:07.138250+0000","last_undegraded":"2023-10-26T06:45:07.138250+0000","last_fullsized":"2023-10-26T06:45:07.138250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:43.252789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822424+0000","last_change":"2023-10-26T06:45:05.897174+0000","last_active":"2023-10-26T06:45:07.822424+0000","last_peered":"2023-10-26T06:45:07.822424+0000","last_clean":"2023-10-26T06:45:07.822424+0000","last_became_active":"2023-10-26T06:45:05.896893+0000","last_became_peered":"2023-10-26T06:45:05.896893+0000","last_unstale":"2023-10-26T06:45:07.822424+0000","last_undegraded":"2023-10-26T06:45:07.822424+0000","last_fullsized":"2023-10-26T06:45:07.822424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:14.291682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137904+0000","last_change":"2023-10-26T06:45:05.905987+0000","last_active":"2023-10-26T06:45:07.137904+0000","last_peered":"2023-10-26T06:45:07.137904+0000","last_clean":"2023-10-26T06:45:07.137904+0000","last_became_active":"2023-10-26T06:45:05.904933+0000","last_became_peered":"2023-10-26T06:45:05.904933+0000","last_unstale":"2023-10-26T06:45:07.137904+0000","last_undegraded":"2023-10-26T06:45:07.137904+0000","last_fullsized":"2023-10-26T06:45:07.137904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:19.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885189+0000","last_change":"2023-10-26T06:45:05.912781+0000","last_active":"2023-10-26T06:45:06.885189+0000","last_peered":"2023-10-26T06:45:06.885189+0000","last_clean":"2023-10-26T06:45:06.885189+0000","last_became_active":"2023-10-26T06:45:05.912415+0000","last_became_peered":"2023-10-26T06:45:05.912415+0000","last_unstale":"2023-10-26T06:45:06.885189+0000","last_undegraded":"2023-10-26T06:45:06.885189+0000","last_fullsized":"2023-10-26T06:45:06.885189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:25.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822121+0000","last_change":"2023-10-26T06:45:05.909106+0000","last_active":"2023-10-26T06:45:07.822121+0000","last_peered":"2023-10-26T06:45:07.822121+0000","last_clean":"2023-10-26T06:45:07.822121+0000","last_became_active":"2023-10-26T06:45:05.908751+0000","last_became_peered":"2023-10-26T06:45:05.908751+0000","last_unstale":"2023-10-26T06:45:07.822121+0000","last_undegraded":"2023-10-26T06:45:07.822121+0000","last_fullsized":"2023-10-26T06:45:07.822121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:17.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599847+0000","last_change":"2023-10-26T06:45:05.898262+0000","last_active":"2023-10-26T06:45:08.599847+0000","last_peered":"2023-10-26T06:45:08.599847+0000","last_clean":"2023-10-26T06:45:08.599847+0000","last_became_active":"2023-10-26T06:45:05.897947+0000","last_became_peered":"2023-10-26T06:45:05.897947+0000","last_unstale":"2023-10-26T06:45:08.599847+0000","last_undegraded":"2023-10-26T06:45:08.599847+0000","last_fullsized":"2023-10-26T06:45:08.599847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:39:45.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59373,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":111,"num_read_kb":114,"num_write":68,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":178119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416406,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":444588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.627}]},{"osd":2,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.753},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":9,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.85799999999999998}]},{"osd":10,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.873}]}]},{"osd":10,"up_from":72,"seq":309237645340,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1093848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44500000000000001,"15min":0.44400000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82499999999999996,"5min":1.004,"15min":1.004},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79800000000000004,"5min":0.874,"15min":0.874},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.94099999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.875}]},{"osd":7,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41699999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.752},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95599999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.91300000000000003}]},{"osd":9,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.046,"5min":1.133,"15min":1.133},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95699999999999996}]},{"osd":11,"last update":"Thu Oct 26 06:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.995}]}]},{"osd":9,"up_from":67,"seq":287762808863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":445170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56100000000000005,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.57699999999999996,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95699999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93799999999999994,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.002,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48899999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.872,"5min":1.03,"15min":1.03},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.83799999999999997,"15min":0.85399999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.84499999999999997,"15min":0.86499999999999999},"min":{"1min":0.55200000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93100000000000005,"5min":1.173,"15min":1.173},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.87},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97199999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0940000000000001,"5min":1.379,"15min":1.379},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.88800000000000001,"15min":0.91100000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.292,"5min":1.327,"15min":1.327},"last":0.622}]},{"osd":7,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.92700000000000005,"15min":0.94899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.143,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.94699999999999995,"15min":0.96899999999999997},"min":{"1min":0.59399999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.044,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.83199999999999996}]},{"osd":8,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.95699999999999996,"15min":0.97099999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.93700000000000006,"15min":0.94499999999999995},"min":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.92900000000000005}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.86699999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070500,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":444861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.749,"15min":0.78700000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.439,"15min":0.443},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47899999999999998,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67900000000000005,"5min":0.752,"15min":0.752},"last":0.498}]},{"osd":3,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.626,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.88,"15min":0.88},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85699999999999998,"5min":1.095,"15min":1.095},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.85599999999999998,"15min":0.89500000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.79100000000000004,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78100000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.81000000000000005,"5min":0.87,"15min":0.87},"last":0.745}]},{"osd":9,"last update":"Thu Oct 26 06:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.82899999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.79100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1035661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41699999999999998,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60299999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78500000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.748,"15min":0.748},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.51100000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.69699999999999995,"15min":0.68700000000000006},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Thu Oct 26 06:45:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.01,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.57799999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95199999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69199999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462957,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":444711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.379,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.54500000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.57299999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.629},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.63300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.748}]},{"osd":7,"last update":"Thu Oct 26 06:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.82399999999999995,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":5.532,"5min":5.532,"15min":5.532},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.1180000000000001,"5min":0.79900000000000004,"15min":0.73299999999999998},"min":{"1min":0.56499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":5.5810000000000004,"5min":5.5810000000000004,"15min":5.5810000000000004},"last":0.71399999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.53100000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89700000000000002,"5min":1.089,"15min":1.089},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.64200000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.80500000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Oct 26 06:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83899999999999997,"15min":0.83999999999999997},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84699999999999998,"15min":0.84599999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.76200000000000001}]},{"osd":10,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83399999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":503163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.53500000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.80900000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.33700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83999999999999997,"5min":1.256,"15min":1.256},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73499999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Oct 26 06:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.40200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58399999999999996,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.86799999999999999}]},{"osd":4,"last update":"Thu Oct 26 06:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.45500000000000002,"15min":0.442},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.42799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.82699999999999996}]},{"osd":5,"last update":"Thu Oct 26 06:45:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47099999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Oct 26 06:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81599999999999995,"5min":1.03,"15min":1.03},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.54800000000000004,"15min":0.53000000000000003},"min":{"1min":0.36199999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.80000000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86699999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":1.298,"15min":1.298},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.57299999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.84299999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.92300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":444678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 06:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.626,"15min":0.58999999999999997},"min":{"1min":0.47199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.601},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79700000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":1.135}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.434,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93400000000000005,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90800000000000003,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.5800000000000001}]},{"osd":3,"last update":"Thu Oct 26 06:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":2.198,"15min":2.198},"last":1.5229999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97599999999999998,"5min":1.48,"15min":1.48},"last":1.617}]},{"osd":4,"last update":"Thu Oct 26 06:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.651},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4450000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88400000000000001,"5min":1.002,"15min":1.002},"last":1.458},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.6339999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64600000000000002,"15min":0.61399999999999999},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69499999999999995,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.6919999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.089,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.486},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.66300000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.131,"5min":1.294,"15min":1.294},"last":1.673}]},{"osd":9,"last update":"Thu Oct 26 06:45:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.89500000000000002},"min":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.544}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":1.7250000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.214}]},{"osd":11,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.5669999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":1.7090000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":502528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81699999999999995,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Oct 26 06:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.39900000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.56999999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.46100000000000002},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:45:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Oct 26 06:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.78700000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.997,"15min":0.997},"last":1.3480000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.3919999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.78400000000000003,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.431,"5min":3.431,"15min":3.431},"last":0.84399999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.93500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.42999999999999999,"15min":0.40400000000000003},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.47599999999999998,"15min":0.44900000000000001},"min":{"1min":0.35399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.083,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.47699999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67600000000000005,"15min":0.66100000000000003},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 06:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1140000000000001,"5min":1.304,"15min":1.304},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.68400000000000005},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72999999999999998,"15min":0.69499999999999995},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.73599999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.84399999999999997,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84299999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":444610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.434}]},{"osd":1,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.35299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.57899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66400000000000003,"5min":1.222,"15min":1.222},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61799999999999999,"15min":0.59199999999999997},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Oct 26 06:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92000000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.66500000000000004},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84599999999999997,"5min":1.403,"15min":1.403},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70899999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69099999999999995,"15min":0.68600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.81999999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85299999999999998,"5min":1.026,"15min":1.026},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.175},"last":0.871}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1515520,"data_stored":1035261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.47499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.32500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.503},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65700000000000003,"15min":0.64100000000000001},"min":{"1min":0.57399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67300000000000004,"15min":0.65400000000000003},"min":{"1min":0.49299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84399999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.56200000000000006,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.85999999999999999}]},{"osd":8,"last update":"Thu Oct 26 06:45:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.38400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.875},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.66000000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.89300000000000002}]},{"osd":10,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.91700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":58769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-26T06:45:59.572 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph pg dump --format=json 2023-10-26T06:45:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:45:59 smithi192 ceph-mon[131661]: pgmap v280: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:00.313 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:00.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:00 smithi135 ceph-mon[131403]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:00.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:00 smithi089 ceph-mon[95031]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:00.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:00 smithi192 ceph-mon[131661]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:01.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:01 smithi135 ceph-mon[131403]: pgmap v281: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:01.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:01 smithi089 ceph-mon[95031]: pgmap v281: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:01 smithi192 ceph-mon[131661]: pgmap v281: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 -- 172.21.15.89:0/2436132038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c1057c0 msgr2=0x7fc01c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 --2- 172.21.15.89:0/2436132038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c1057c0 0x7fc01c105ba0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009990 tx=0x7fc00c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 -- 172.21.15.89:0/2436132038 shutdown_connections 2023-10-26T06:46:01.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 --2- 172.21.15.89:0/2436132038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c1057c0 0x7fc01c105ba0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 --2- 172.21.15.89:0/2436132038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc01c101260 0x7fc01c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 --2- 172.21.15.89:0/2436132038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c1008a0 0x7fc01c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 -- 172.21.15.89:0/2436132038 >> 172.21.15.89:0/2436132038 conn(0x7fc01c0779e0 msgr2=0x7fc01c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 -- 172.21.15.89:0/2436132038 shutdown_connections 2023-10-26T06:46:01.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.817+0000 7fc020c92700 1 -- 172.21.15.89:0/2436132038 wait complete. 2023-10-26T06:46:01.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.818+0000 7fc020c92700 1 Processor -- start 2023-10-26T06:46:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.818+0000 7fc020c92700 1 -- start start 2023-10-26T06:46:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc020c92700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c1008a0 0x7fc01c198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc020c92700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c101260 0x7fc01c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc020c92700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc01c1057c0 0x7fc01c19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:01.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc020c92700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c19de10 con 0x7fc01c101260 2023-10-26T06:46:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc020c92700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c19df80 con 0x7fc01c1057c0 2023-10-26T06:46:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc020c92700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c19e140 con 0x7fc01c1008a0 2023-10-26T06:46:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c101260 0x7fc01c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c101260 0x7fc01c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41842/0 (socket says 172.21.15.89:41842) 2023-10-26T06:46:01.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc019d9b700 1 -- 172.21.15.89:0/2514648349 learned_addr learned my addr 172.21.15.89:0/2514648349 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc01a59c700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c1008a0 0x7fc01c198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.819+0000 7fc01ad9d700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc01c1057c0 0x7fc01c19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc019d9b700 1 -- 172.21.15.89:0/2514648349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c1008a0 msgr2=0x7fc01c198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc019d9b700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c1008a0 0x7fc01c198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:01.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc019d9b700 1 -- 172.21.15.89:0/2514648349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc01c1057c0 msgr2=0x7fc01c19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:01.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc019d9b700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc01c1057c0 0x7fc01c19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:01.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc019d9b700 1 -- 172.21.15.89:0/2514648349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0095f0 con 0x7fc01c101260 2023-10-26T06:46:01.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc01ad9d700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc01c1057c0 0x7fc01c19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:01.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc019d9b700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c101260 0x7fc01c1994a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d950 tx=0x7fc01000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:01.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc00b7fe700 1 -- 172.21.15.89:0/2514648349 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010009940 con 0x7fc01c101260 2023-10-26T06:46:01.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc00b7fe700 1 -- 172.21.15.89:0/2514648349 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc010010460 con 0x7fc01c101260 2023-10-26T06:46:01.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c19e420 con 0x7fc01c101260 2023-10-26T06:46:01.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc00b7fe700 1 -- 172.21.15.89:0/2514648349 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000f620 con 0x7fc01c101260 2023-10-26T06:46:01.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.820+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c1a5c80 con 0x7fc01c101260 2023-10-26T06:46:01.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.821+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c101f30 con 0x7fc01c101260 2023-10-26T06:46:01.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.823+0000 7fc00b7fe700 1 -- 172.21.15.89:0/2514648349 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fc0100108d0 con 0x7fc01c101260 2023-10-26T06:46:01.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.824+0000 7fc00b7fe700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0040a15e0 0x7fc0040a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:01.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.825+0000 7fc00b7fe700 1 -- 172.21.15.89:0/2514648349 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8194+0+0 (secure 0 0 0) 0x7fc0100ccd00 con 0x7fc01c101260 2023-10-26T06:46:01.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.825+0000 7fc01a59c700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0040a15e0 0x7fc0040a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:01.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.825+0000 7fc01a59c700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0040a15e0 0x7fc0040a3aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc00c006010 tx=0x7fc00c00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:01.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:01.828+0000 7fc00b7fe700 1 -- 172.21.15.89:0/2514648349 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc0100961c0 con 0x7fc01c101260 2023-10-26T06:46:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.023+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc01c19a2c0 con 0x7fc0040a15e0 2023-10-26T06:46:02.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.033+0000 7fc00b7fe700 1 -- 172.21.15.89:0/2514648349 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340469 (secure 0 0 0) 0x7fc01c19a2c0 con 0x7fc0040a15e0 2023-10-26T06:46:02.041 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0040a15e0 msgr2=0x7fc0040a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0040a15e0 0x7fc0040a3aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc00c006010 tx=0x7fc00c00b540 comp rx=0 tx=0).stop 2023-10-26T06:46:02.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c101260 msgr2=0x7fc01c1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c101260 0x7fc01c1994a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d950 tx=0x7fc01000dc60 comp rx=0 tx=0).stop 2023-10-26T06:46:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 shutdown_connections 2023-10-26T06:46:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0040a15e0 0x7fc0040a3aa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:02.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc01c1057c0 0x7fc01c19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c1008a0 0x7fc01c198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.038+0000 7fc020c92700 1 --2- 172.21.15.89:0/2514648349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc01c101260 0x7fc01c1994a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.039+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 >> 172.21.15.89:0/2514648349 conn(0x7fc01c0779e0 msgr2=0x7fc01c10bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.039+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 shutdown_connections 2023-10-26T06:46:02.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:02.039+0000 7fc020c92700 1 -- 172.21.15.89:0/2514648349 wait complete. 2023-10-26T06:46:02.047 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-26T06:46:02.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-26T06:46:02.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-10-26T06:46:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-26T06:46:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-26T06:46:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-10-26T06:46:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-10-26T06:46:02.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2023-10-26T06:46:02.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-26T06:46:02.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:02 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-10-26T06:46:02.487 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-26T06:46:02.487 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-10-26T06:46:02.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-26T06:46:02.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-26T06:46:02.488 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-10-26T06:46:02.489 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-10-26T06:46:02.489 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2023-10-26T06:46:02.490 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-26T06:46:02.490 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:02 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-10-26T06:46:02.493 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2023-10-26T06:46:01.727358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649987,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":247,"num_read_kb":233,"num_write":251,"num_write_kb":3017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336832,"kb_used_data":12624,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604992,"statfs":{"total":1151940427776,"available":1151595511808,"internally_reserved":0,"allocated":12926976,"data_stored":7283686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003683"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600110+0000","last_change":"2023-10-26T06:45:05.903170+0000","last_active":"2023-10-26T06:45:08.600110+0000","last_peered":"2023-10-26T06:45:08.600110+0000","last_clean":"2023-10-26T06:45:08.600110+0000","last_became_active":"2023-10-26T06:45:05.902574+0000","last_became_peered":"2023-10-26T06:45:05.902574+0000","last_unstale":"2023-10-26T06:45:08.600110+0000","last_undegraded":"2023-10-26T06:45:08.600110+0000","last_fullsized":"2023-10-26T06:45:08.600110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:33.065828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505541+0000","last_change":"2023-10-26T06:45:05.911249+0000","last_active":"2023-10-26T06:45:07.505541+0000","last_peered":"2023-10-26T06:45:07.505541+0000","last_clean":"2023-10-26T06:45:07.505541+0000","last_became_active":"2023-10-26T06:45:05.910859+0000","last_became_peered":"2023-10-26T06:45:05.910859+0000","last_unstale":"2023-10-26T06:45:07.505541+0000","last_undegraded":"2023-10-26T06:45:07.505541+0000","last_fullsized":"2023-10-26T06:45:07.505541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:07:13.362460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505391+0000","last_change":"2023-10-26T06:45:05.909219+0000","last_active":"2023-10-26T06:45:07.505391+0000","last_peered":"2023-10-26T06:45:07.505391+0000","last_clean":"2023-10-26T06:45:07.505391+0000","last_became_active":"2023-10-26T06:45:05.906721+0000","last_became_peered":"2023-10-26T06:45:05.906721+0000","last_unstale":"2023-10-26T06:45:07.505391+0000","last_undegraded":"2023-10-26T06:45:07.505391+0000","last_fullsized":"2023-10-26T06:45:07.505391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:41.461357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823998+0000","last_change":"2023-10-26T06:45:05.897514+0000","last_active":"2023-10-26T06:45:07.823998+0000","last_peered":"2023-10-26T06:45:07.823998+0000","last_clean":"2023-10-26T06:45:07.823998+0000","last_became_active":"2023-10-26T06:45:05.897201+0000","last_became_peered":"2023-10-26T06:45:05.897201+0000","last_unstale":"2023-10-26T06:45:07.823998+0000","last_undegraded":"2023-10-26T06:45:07.823998+0000","last_fullsized":"2023-10-26T06:45:07.823998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:39.508654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689213+0000","last_change":"2023-10-26T06:45:05.900475+0000","last_active":"2023-10-26T06:45:07.689213+0000","last_peered":"2023-10-26T06:45:07.689213+0000","last_clean":"2023-10-26T06:45:07.689213+0000","last_became_active":"2023-10-26T06:45:05.900313+0000","last_became_peered":"2023-10-26T06:45:05.900313+0000","last_unstale":"2023-10-26T06:45:07.689213+0000","last_undegraded":"2023-10-26T06:45:07.689213+0000","last_fullsized":"2023-10-26T06:45:07.689213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:51.438741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823465+0000","last_change":"2023-10-26T06:45:05.909040+0000","last_active":"2023-10-26T06:45:07.823465+0000","last_peered":"2023-10-26T06:45:07.823465+0000","last_clean":"2023-10-26T06:45:07.823465+0000","last_became_active":"2023-10-26T06:45:05.908696+0000","last_became_peered":"2023-10-26T06:45:05.908696+0000","last_unstale":"2023-10-26T06:45:07.823465+0000","last_undegraded":"2023-10-26T06:45:07.823465+0000","last_fullsized":"2023-10-26T06:45:07.823465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:00.147712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823545+0000","last_change":"2023-10-26T06:45:05.898356+0000","last_active":"2023-10-26T06:45:07.823545+0000","last_peered":"2023-10-26T06:45:07.823545+0000","last_clean":"2023-10-26T06:45:07.823545+0000","last_became_active":"2023-10-26T06:45:05.898008+0000","last_became_peered":"2023-10-26T06:45:05.898008+0000","last_unstale":"2023-10-26T06:45:07.823545+0000","last_undegraded":"2023-10-26T06:45:07.823545+0000","last_fullsized":"2023-10-26T06:45:07.823545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:23.209987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137846+0000","last_change":"2023-10-26T06:45:05.911301+0000","last_active":"2023-10-26T06:45:07.137846+0000","last_peered":"2023-10-26T06:45:07.137846+0000","last_clean":"2023-10-26T06:45:07.137846+0000","last_became_active":"2023-10-26T06:45:05.910667+0000","last_became_peered":"2023-10-26T06:45:05.910667+0000","last_unstale":"2023-10-26T06:45:07.137846+0000","last_undegraded":"2023-10-26T06:45:07.137846+0000","last_fullsized":"2023-10-26T06:45:07.137846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:03:36.856248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885616+0000","last_change":"2023-10-26T06:45:05.904456+0000","last_active":"2023-10-26T06:45:06.885616+0000","last_peered":"2023-10-26T06:45:06.885616+0000","last_clean":"2023-10-26T06:45:06.885616+0000","last_became_active":"2023-10-26T06:45:05.904165+0000","last_became_peered":"2023-10-26T06:45:05.904165+0000","last_unstale":"2023-10-26T06:45:06.885616+0000","last_undegraded":"2023-10-26T06:45:06.885616+0000","last_fullsized":"2023-10-26T06:45:06.885616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:51:01.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824115+0000","last_change":"2023-10-26T06:45:05.909544+0000","last_active":"2023-10-26T06:45:07.824115+0000","last_peered":"2023-10-26T06:45:07.824115+0000","last_clean":"2023-10-26T06:45:07.824115+0000","last_became_active":"2023-10-26T06:45:05.909103+0000","last_became_peered":"2023-10-26T06:45:05.909103+0000","last_unstale":"2023-10-26T06:45:07.824115+0000","last_undegraded":"2023-10-26T06:45:07.824115+0000","last_fullsized":"2023-10-26T06:45:07.824115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:18.069186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823939+0000","last_change":"2023-10-26T06:45:05.906549+0000","last_active":"2023-10-26T06:45:07.823939+0000","last_peered":"2023-10-26T06:45:07.823939+0000","last_clean":"2023-10-26T06:45:07.823939+0000","last_became_active":"2023-10-26T06:45:05.905883+0000","last_became_peered":"2023-10-26T06:45:05.905883+0000","last_unstale":"2023-10-26T06:45:07.823939+0000","last_undegraded":"2023-10-26T06:45:07.823939+0000","last_fullsized":"2023-10-26T06:45:07.823939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:54.771031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823855+0000","last_change":"2023-10-26T06:45:05.909460+0000","last_active":"2023-10-26T06:45:07.823855+0000","last_peered":"2023-10-26T06:45:07.823855+0000","last_clean":"2023-10-26T06:45:07.823855+0000","last_became_active":"2023-10-26T06:45:05.909007+0000","last_became_peered":"2023-10-26T06:45:05.909007+0000","last_unstale":"2023-10-26T06:45:07.823855+0000","last_undegraded":"2023-10-26T06:45:07.823855+0000","last_fullsized":"2023-10-26T06:45:07.823855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:45:55.817606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822132+0000","last_change":"2023-10-26T06:45:05.908305+0000","last_active":"2023-10-26T06:45:07.822132+0000","last_peered":"2023-10-26T06:45:07.822132+0000","last_clean":"2023-10-26T06:45:07.822132+0000","last_became_active":"2023-10-26T06:45:05.907931+0000","last_became_peered":"2023-10-26T06:45:05.907931+0000","last_unstale":"2023-10-26T06:45:07.822132+0000","last_undegraded":"2023-10-26T06:45:07.822132+0000","last_fullsized":"2023-10-26T06:45:07.822132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:00:43.605324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505175+0000","last_change":"2023-10-26T06:45:05.909737+0000","last_active":"2023-10-26T06:45:07.505175+0000","last_peered":"2023-10-26T06:45:07.505175+0000","last_clean":"2023-10-26T06:45:07.505175+0000","last_became_active":"2023-10-26T06:45:05.909445+0000","last_became_peered":"2023-10-26T06:45:05.909445+0000","last_unstale":"2023-10-26T06:45:07.505175+0000","last_undegraded":"2023-10-26T06:45:07.505175+0000","last_fullsized":"2023-10-26T06:45:07.505175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:38.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.504905+0000","last_change":"2023-10-26T06:45:05.907722+0000","last_active":"2023-10-26T06:45:07.504905+0000","last_peered":"2023-10-26T06:45:07.504905+0000","last_clean":"2023-10-26T06:45:07.504905+0000","last_became_active":"2023-10-26T06:45:05.905622+0000","last_became_peered":"2023-10-26T06:45:05.905622+0000","last_unstale":"2023-10-26T06:45:07.504905+0000","last_undegraded":"2023-10-26T06:45:07.504905+0000","last_fullsized":"2023-10-26T06:45:07.504905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:30:01.048032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885013+0000","last_change":"2023-10-26T06:45:05.907801+0000","last_active":"2023-10-26T06:45:06.885013+0000","last_peered":"2023-10-26T06:45:06.885013+0000","last_clean":"2023-10-26T06:45:06.885013+0000","last_became_active":"2023-10-26T06:45:05.907485+0000","last_became_peered":"2023-10-26T06:45:05.907485+0000","last_unstale":"2023-10-26T06:45:06.885013+0000","last_undegraded":"2023-10-26T06:45:06.885013+0000","last_fullsized":"2023-10-26T06:45:06.885013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:16.850992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600141+0000","last_change":"2023-10-26T06:45:05.902859+0000","last_active":"2023-10-26T06:45:08.600141+0000","last_peered":"2023-10-26T06:45:08.600141+0000","last_clean":"2023-10-26T06:45:08.600141+0000","last_became_active":"2023-10-26T06:45:05.902218+0000","last_became_peered":"2023-10-26T06:45:05.902218+0000","last_unstale":"2023-10-26T06:45:08.600141+0000","last_undegraded":"2023-10-26T06:45:08.600141+0000","last_fullsized":"2023-10-26T06:45:08.600141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:47.717227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884920+0000","last_change":"2023-10-26T06:45:05.910445+0000","last_active":"2023-10-26T06:45:06.884920+0000","last_peered":"2023-10-26T06:45:06.884920+0000","last_clean":"2023-10-26T06:45:06.884920+0000","last_became_active":"2023-10-26T06:45:05.910043+0000","last_became_peered":"2023-10-26T06:45:05.910043+0000","last_unstale":"2023-10-26T06:45:06.884920+0000","last_undegraded":"2023-10-26T06:45:06.884920+0000","last_fullsized":"2023-10-26T06:45:06.884920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:33.751134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884907+0000","last_change":"2023-10-26T06:45:05.905340+0000","last_active":"2023-10-26T06:45:06.884907+0000","last_peered":"2023-10-26T06:45:06.884907+0000","last_clean":"2023-10-26T06:45:06.884907+0000","last_became_active":"2023-10-26T06:45:05.905030+0000","last_became_peered":"2023-10-26T06:45:05.905030+0000","last_unstale":"2023-10-26T06:45:06.884907+0000","last_undegraded":"2023-10-26T06:45:06.884907+0000","last_fullsized":"2023-10-26T06:45:06.884907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.303210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884956+0000","last_change":"2023-10-26T06:45:04.919128+0000","last_active":"2023-10-26T06:45:06.884956+0000","last_peered":"2023-10-26T06:45:06.884956+0000","last_clean":"2023-10-26T06:45:06.884956+0000","last_became_active":"2023-10-26T06:45:04.913034+0000","last_became_peered":"2023-10-26T06:45:04.913034+0000","last_unstale":"2023-10-26T06:45:06.884956+0000","last_undegraded":"2023-10-26T06:45:06.884956+0000","last_fullsized":"2023-10-26T06:45:06.884956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:50.835037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138323+0000","last_change":"2023-10-26T06:45:05.905315+0000","last_active":"2023-10-26T06:45:07.138323+0000","last_peered":"2023-10-26T06:45:07.138323+0000","last_clean":"2023-10-26T06:45:07.138323+0000","last_became_active":"2023-10-26T06:45:05.904260+0000","last_became_peered":"2023-10-26T06:45:05.904260+0000","last_unstale":"2023-10-26T06:45:07.138323+0000","last_undegraded":"2023-10-26T06:45:07.138323+0000","last_fullsized":"2023-10-26T06:45:07.138323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:02.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.080681+0000","last_change":"2023-10-26T06:45:04.920440+0000","last_active":"2023-10-26T06:45:07.080681+0000","last_peered":"2023-10-26T06:45:07.080681+0000","last_clean":"2023-10-26T06:45:07.080681+0000","last_became_active":"2023-10-26T06:45:04.920129+0000","last_became_peered":"2023-10-26T06:45:04.920129+0000","last_unstale":"2023-10-26T06:45:07.080681+0000","last_undegraded":"2023-10-26T06:45:07.080681+0000","last_fullsized":"2023-10-26T06:45:07.080681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:43.176830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505711+0000","last_change":"2023-10-26T06:45:05.902898+0000","last_active":"2023-10-26T06:45:07.505711+0000","last_peered":"2023-10-26T06:45:07.505711+0000","last_clean":"2023-10-26T06:45:07.505711+0000","last_became_active":"2023-10-26T06:45:05.902161+0000","last_became_peered":"2023-10-26T06:45:05.902161+0000","last_unstale":"2023-10-26T06:45:07.505711+0000","last_undegraded":"2023-10-26T06:45:07.505711+0000","last_fullsized":"2023-10-26T06:45:07.505711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:27.872225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063544+0000","last_change":"2023-10-26T06:45:04.911835+0000","last_active":"2023-10-26T06:45:06.063544+0000","last_peered":"2023-10-26T06:45:06.063544+0000","last_clean":"2023-10-26T06:45:06.063544+0000","last_became_active":"2023-10-26T06:45:04.911505+0000","last_became_peered":"2023-10-26T06:45:04.911505+0000","last_unstale":"2023-10-26T06:45:06.063544+0000","last_undegraded":"2023-10-26T06:45:06.063544+0000","last_fullsized":"2023-10-26T06:45:06.063544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:01:18.971381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885164+0000","last_change":"2023-10-26T06:45:05.909088+0000","last_active":"2023-10-26T06:45:06.885164+0000","last_peered":"2023-10-26T06:45:06.885164+0000","last_clean":"2023-10-26T06:45:06.885164+0000","last_became_active":"2023-10-26T06:45:05.908528+0000","last_became_peered":"2023-10-26T06:45:05.908528+0000","last_unstale":"2023-10-26T06:45:06.885164+0000","last_undegraded":"2023-10-26T06:45:06.885164+0000","last_fullsized":"2023-10-26T06:45:06.885164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:07.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063027+0000","last_change":"2023-10-26T06:45:04.913604+0000","last_active":"2023-10-26T06:45:06.063027+0000","last_peered":"2023-10-26T06:45:06.063027+0000","last_clean":"2023-10-26T06:45:06.063027+0000","last_became_active":"2023-10-26T06:45:04.913318+0000","last_became_peered":"2023-10-26T06:45:04.913318+0000","last_unstale":"2023-10-26T06:45:06.063027+0000","last_undegraded":"2023-10-26T06:45:06.063027+0000","last_fullsized":"2023-10-26T06:45:06.063027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:19:51.144044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822837+0000","last_change":"2023-10-26T06:45:05.898795+0000","last_active":"2023-10-26T06:45:07.822837+0000","last_peered":"2023-10-26T06:45:07.822837+0000","last_clean":"2023-10-26T06:45:07.822837+0000","last_became_active":"2023-10-26T06:45:05.898489+0000","last_became_peered":"2023-10-26T06:45:05.898489+0000","last_unstale":"2023-10-26T06:45:07.822837+0000","last_undegraded":"2023-10-26T06:45:07.822837+0000","last_fullsized":"2023-10-26T06:45:07.822837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:55.343736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062447+0000","last_change":"2023-10-26T06:45:04.908881+0000","last_active":"2023-10-26T06:45:06.062447+0000","last_peered":"2023-10-26T06:45:06.062447+0000","last_clean":"2023-10-26T06:45:06.062447+0000","last_became_active":"2023-10-26T06:45:04.908515+0000","last_became_peered":"2023-10-26T06:45:04.908515+0000","last_unstale":"2023-10-26T06:45:06.062447+0000","last_undegraded":"2023-10-26T06:45:06.062447+0000","last_fullsized":"2023-10-26T06:45:06.062447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:39.200294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689203+0000","last_change":"2023-10-26T06:45:05.901281+0000","last_active":"2023-10-26T06:45:07.689203+0000","last_peered":"2023-10-26T06:45:07.689203+0000","last_clean":"2023-10-26T06:45:07.689203+0000","last_became_active":"2023-10-26T06:45:05.900995+0000","last_became_peered":"2023-10-26T06:45:05.900995+0000","last_unstale":"2023-10-26T06:45:07.689203+0000","last_undegraded":"2023-10-26T06:45:07.689203+0000","last_fullsized":"2023-10-26T06:45:07.689203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:44:34.977571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.285852+0000","last_change":"2023-10-26T06:45:04.918480+0000","last_active":"2023-10-26T06:45:07.285852+0000","last_peered":"2023-10-26T06:45:07.285852+0000","last_clean":"2023-10-26T06:45:07.285852+0000","last_became_active":"2023-10-26T06:45:04.918185+0000","last_became_peered":"2023-10-26T06:45:04.918185+0000","last_unstale":"2023-10-26T06:45:07.285852+0000","last_undegraded":"2023-10-26T06:45:07.285852+0000","last_fullsized":"2023-10-26T06:45:07.285852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:07:58.263116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822711+0000","last_change":"2023-10-26T06:45:05.901825+0000","last_active":"2023-10-26T06:45:07.822711+0000","last_peered":"2023-10-26T06:45:07.822711+0000","last_clean":"2023-10-26T06:45:07.822711+0000","last_became_active":"2023-10-26T06:45:05.901509+0000","last_became_peered":"2023-10-26T06:45:05.901509+0000","last_unstale":"2023-10-26T06:45:07.822711+0000","last_undegraded":"2023-10-26T06:45:07.822711+0000","last_fullsized":"2023-10-26T06:45:07.822711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:16.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534260+0000","last_change":"2023-10-26T06:45:04.919813+0000","last_active":"2023-10-26T06:45:06.534260+0000","last_peered":"2023-10-26T06:45:06.534260+0000","last_clean":"2023-10-26T06:45:06.534260+0000","last_became_active":"2023-10-26T06:45:04.918016+0000","last_became_peered":"2023-10-26T06:45:04.918016+0000","last_unstale":"2023-10-26T06:45:06.534260+0000","last_undegraded":"2023-10-26T06:45:06.534260+0000","last_fullsized":"2023-10-26T06:45:06.534260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:32.336126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137929+0000","last_change":"2023-10-26T06:45:05.902182+0000","last_active":"2023-10-26T06:45:07.137929+0000","last_peered":"2023-10-26T06:45:07.137929+0000","last_clean":"2023-10-26T06:45:07.137929+0000","last_became_active":"2023-10-26T06:45:05.901428+0000","last_became_peered":"2023-10-26T06:45:05.901428+0000","last_unstale":"2023-10-26T06:45:07.137929+0000","last_undegraded":"2023-10-26T06:45:07.137929+0000","last_fullsized":"2023-10-26T06:45:07.137929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:12.722532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.890140+0000","last_change":"2023-10-26T06:45:04.909713+0000","last_active":"2023-10-26T06:45:05.890140+0000","last_peered":"2023-10-26T06:45:05.890140+0000","last_clean":"2023-10-26T06:45:05.890140+0000","last_became_active":"2023-10-26T06:45:04.909412+0000","last_became_peered":"2023-10-26T06:45:04.909412+0000","last_unstale":"2023-10-26T06:45:05.890140+0000","last_undegraded":"2023-10-26T06:45:05.890140+0000","last_fullsized":"2023-10-26T06:45:05.890140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:23.901745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884924+0000","last_change":"2023-10-26T06:45:05.908348+0000","last_active":"2023-10-26T06:45:06.884924+0000","last_peered":"2023-10-26T06:45:06.884924+0000","last_clean":"2023-10-26T06:45:06.884924+0000","last_became_active":"2023-10-26T06:45:05.908022+0000","last_became_peered":"2023-10-26T06:45:05.908022+0000","last_unstale":"2023-10-26T06:45:06.884924+0000","last_undegraded":"2023-10-26T06:45:06.884924+0000","last_fullsized":"2023-10-26T06:45:06.884924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:22:16.027732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.285938+0000","last_change":"2023-10-26T06:45:04.915491+0000","last_active":"2023-10-26T06:45:07.285938+0000","last_peered":"2023-10-26T06:45:07.285938+0000","last_clean":"2023-10-26T06:45:07.285938+0000","last_became_active":"2023-10-26T06:45:04.915191+0000","last_became_peered":"2023-10-26T06:45:04.915191+0000","last_unstale":"2023-10-26T06:45:07.285938+0000","last_undegraded":"2023-10-26T06:45:07.285938+0000","last_fullsized":"2023-10-26T06:45:07.285938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:57.758903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885545+0000","last_change":"2023-10-26T06:45:05.912676+0000","last_active":"2023-10-26T06:45:06.885545+0000","last_peered":"2023-10-26T06:45:06.885545+0000","last_clean":"2023-10-26T06:45:06.885545+0000","last_became_active":"2023-10-26T06:45:05.912372+0000","last_became_peered":"2023-10-26T06:45:05.912372+0000","last_unstale":"2023-10-26T06:45:06.885545+0000","last_undegraded":"2023-10-26T06:45:06.885545+0000","last_fullsized":"2023-10-26T06:45:06.885545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:33.299166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502215+0000","last_change":"2023-10-26T06:45:04.912123+0000","last_active":"2023-10-26T06:45:07.502215+0000","last_peered":"2023-10-26T06:45:07.502215+0000","last_clean":"2023-10-26T06:45:07.502215+0000","last_became_active":"2023-10-26T06:45:04.911893+0000","last_became_peered":"2023-10-26T06:45:04.911893+0000","last_unstale":"2023-10-26T06:45:07.502215+0000","last_undegraded":"2023-10-26T06:45:07.502215+0000","last_fullsized":"2023-10-26T06:45:07.502215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:32.907496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505559+0000","last_change":"2023-10-26T06:45:05.893434+0000","last_active":"2023-10-26T06:45:07.505559+0000","last_peered":"2023-10-26T06:45:07.505559+0000","last_clean":"2023-10-26T06:45:07.505559+0000","last_became_active":"2023-10-26T06:45:05.893100+0000","last_became_peered":"2023-10-26T06:45:05.893100+0000","last_unstale":"2023-10-26T06:45:07.505559+0000","last_undegraded":"2023-10-26T06:45:07.505559+0000","last_fullsized":"2023-10-26T06:45:07.505559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:22:48.317428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.062642+0000","last_change":"2023-10-26T06:45:04.908910+0000","last_active":"2023-10-26T06:45:06.062642+0000","last_peered":"2023-10-26T06:45:06.062642+0000","last_clean":"2023-10-26T06:45:06.062642+0000","last_became_active":"2023-10-26T06:45:04.908592+0000","last_became_peered":"2023-10-26T06:45:04.908592+0000","last_unstale":"2023-10-26T06:45:06.062642+0000","last_undegraded":"2023-10-26T06:45:06.062642+0000","last_fullsized":"2023-10-26T06:45:06.062642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:40.266013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885272+0000","last_change":"2023-10-26T06:45:05.908887+0000","last_active":"2023-10-26T06:45:06.885272+0000","last_peered":"2023-10-26T06:45:06.885272+0000","last_clean":"2023-10-26T06:45:06.885272+0000","last_became_active":"2023-10-26T06:45:05.908163+0000","last_became_peered":"2023-10-26T06:45:05.908163+0000","last_unstale":"2023-10-26T06:45:06.885272+0000","last_undegraded":"2023-10-26T06:45:06.885272+0000","last_fullsized":"2023-10-26T06:45:06.885272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:09.339374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'25","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:57.165190+0000","last_change":"2023-10-26T06:45:04.905691+0000","last_active":"2023-10-26T06:45:57.165190+0000","last_peered":"2023-10-26T06:45:57.165190+0000","last_clean":"2023-10-26T06:45:57.165190+0000","last_became_active":"2023-10-26T06:45:04.905378+0000","last_became_peered":"2023-10-26T06:45:04.905378+0000","last_unstale":"2023-10-26T06:45:57.165190+0000","last_undegraded":"2023-10-26T06:45:57.165190+0000","last_fullsized":"2023-10-26T06:45:57.165190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:24:21.498141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58575,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":108,"num_write":25,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689388+0000","last_change":"2023-10-26T06:45:05.897366+0000","last_active":"2023-10-26T06:45:07.689388+0000","last_peered":"2023-10-26T06:45:07.689388+0000","last_clean":"2023-10-26T06:45:07.689388+0000","last_became_active":"2023-10-26T06:45:05.897091+0000","last_became_peered":"2023-10-26T06:45:05.897091+0000","last_unstale":"2023-10-26T06:45:07.689388+0000","last_undegraded":"2023-10-26T06:45:07.689388+0000","last_fullsized":"2023-10-26T06:45:07.689388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:11.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884895+0000","last_change":"2023-10-26T06:45:04.914949+0000","last_active":"2023-10-26T06:45:06.884895+0000","last_peered":"2023-10-26T06:45:06.884895+0000","last_clean":"2023-10-26T06:45:06.884895+0000","last_became_active":"2023-10-26T06:45:04.914610+0000","last_became_peered":"2023-10-26T06:45:04.914610+0000","last_unstale":"2023-10-26T06:45:06.884895+0000","last_undegraded":"2023-10-26T06:45:06.884895+0000","last_fullsized":"2023-10-26T06:45:06.884895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:16.683352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822001+0000","last_change":"2023-10-26T06:45:05.911067+0000","last_active":"2023-10-26T06:45:07.822001+0000","last_peered":"2023-10-26T06:45:07.822001+0000","last_clean":"2023-10-26T06:45:07.822001+0000","last_became_active":"2023-10-26T06:45:05.908460+0000","last_became_peered":"2023-10-26T06:45:05.908460+0000","last_unstale":"2023-10-26T06:45:07.822001+0000","last_undegraded":"2023-10-26T06:45:07.822001+0000","last_fullsized":"2023-10-26T06:45:07.822001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:17.461781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.895349+0000","last_change":"2023-10-26T06:45:04.919947+0000","last_active":"2023-10-26T06:45:05.895349+0000","last_peered":"2023-10-26T06:45:05.895349+0000","last_clean":"2023-10-26T06:45:05.895349+0000","last_became_active":"2023-10-26T06:45:04.919535+0000","last_became_peered":"2023-10-26T06:45:04.919535+0000","last_unstale":"2023-10-26T06:45:05.895349+0000","last_undegraded":"2023-10-26T06:45:05.895349+0000","last_fullsized":"2023-10-26T06:45:05.895349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:46:09.201139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823839+0000","last_change":"2023-10-26T06:45:05.906381+0000","last_active":"2023-10-26T06:45:07.823839+0000","last_peered":"2023-10-26T06:45:07.823839+0000","last_clean":"2023-10-26T06:45:07.823839+0000","last_became_active":"2023-10-26T06:45:05.904487+0000","last_became_peered":"2023-10-26T06:45:05.904487+0000","last_unstale":"2023-10-26T06:45:07.823839+0000","last_undegraded":"2023-10-26T06:45:07.823839+0000","last_fullsized":"2023-10-26T06:45:07.823839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:00.164936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887797+0000","last_change":"2023-10-26T06:45:04.908779+0000","last_active":"2023-10-26T06:45:05.887797+0000","last_peered":"2023-10-26T06:45:05.887797+0000","last_clean":"2023-10-26T06:45:05.887797+0000","last_became_active":"2023-10-26T06:45:04.908490+0000","last_became_peered":"2023-10-26T06:45:04.908490+0000","last_unstale":"2023-10-26T06:45:05.887797+0000","last_undegraded":"2023-10-26T06:45:05.887797+0000","last_fullsized":"2023-10-26T06:45:05.887797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:12:13.199863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823130+0000","last_change":"2023-10-26T06:45:05.909408+0000","last_active":"2023-10-26T06:45:07.823130+0000","last_peered":"2023-10-26T06:45:07.823130+0000","last_clean":"2023-10-26T06:45:07.823130+0000","last_became_active":"2023-10-26T06:45:05.909107+0000","last_became_peered":"2023-10-26T06:45:05.909107+0000","last_unstale":"2023-10-26T06:45:07.823130+0000","last_undegraded":"2023-10-26T06:45:07.823130+0000","last_fullsized":"2023-10-26T06:45:07.823130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:05.887472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.063862+0000","last_change":"2023-10-26T06:45:04.918194+0000","last_active":"2023-10-26T06:45:06.063862+0000","last_peered":"2023-10-26T06:45:06.063862+0000","last_clean":"2023-10-26T06:45:06.063862+0000","last_became_active":"2023-10-26T06:45:04.917850+0000","last_became_peered":"2023-10-26T06:45:04.917850+0000","last_unstale":"2023-10-26T06:45:06.063862+0000","last_undegraded":"2023-10-26T06:45:06.063862+0000","last_fullsized":"2023-10-26T06:45:06.063862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:25.743939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822581+0000","last_change":"2023-10-26T06:45:05.902156+0000","last_active":"2023-10-26T06:45:07.822581+0000","last_peered":"2023-10-26T06:45:07.822581+0000","last_clean":"2023-10-26T06:45:07.822581+0000","last_became_active":"2023-10-26T06:45:05.901865+0000","last_became_peered":"2023-10-26T06:45:05.901865+0000","last_unstale":"2023-10-26T06:45:07.822581+0000","last_undegraded":"2023-10-26T06:45:07.822581+0000","last_fullsized":"2023-10-26T06:45:07.822581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:06:08.167280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689452+0000","last_change":"2023-10-26T06:45:04.913071+0000","last_active":"2023-10-26T06:45:07.689452+0000","last_peered":"2023-10-26T06:45:07.689452+0000","last_clean":"2023-10-26T06:45:07.689452+0000","last_became_active":"2023-10-26T06:45:04.912736+0000","last_became_peered":"2023-10-26T06:45:04.912736+0000","last_unstale":"2023-10-26T06:45:07.689452+0000","last_undegraded":"2023-10-26T06:45:07.689452+0000","last_fullsized":"2023-10-26T06:45:07.689452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:59.690355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.600012+0000","last_change":"2023-10-26T06:45:05.891949+0000","last_active":"2023-10-26T06:45:08.600012+0000","last_peered":"2023-10-26T06:45:08.600012+0000","last_clean":"2023-10-26T06:45:08.600012+0000","last_became_active":"2023-10-26T06:45:05.891583+0000","last_became_peered":"2023-10-26T06:45:05.891583+0000","last_unstale":"2023-10-26T06:45:08.600012+0000","last_undegraded":"2023-10-26T06:45:08.600012+0000","last_fullsized":"2023-10-26T06:45:08.600012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:54:18.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.883535+0000","last_change":"2023-10-26T06:45:04.913969+0000","last_active":"2023-10-26T06:45:05.883535+0000","last_peered":"2023-10-26T06:45:05.883535+0000","last_clean":"2023-10-26T06:45:05.883535+0000","last_became_active":"2023-10-26T06:45:04.913533+0000","last_became_peered":"2023-10-26T06:45:04.913533+0000","last_unstale":"2023-10-26T06:45:05.883535+0000","last_undegraded":"2023-10-26T06:45:05.883535+0000","last_fullsized":"2023-10-26T06:45:05.883535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:47:38.755102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824112+0000","last_change":"2023-10-26T06:45:05.904832+0000","last_active":"2023-10-26T06:45:07.824112+0000","last_peered":"2023-10-26T06:45:07.824112+0000","last_clean":"2023-10-26T06:45:07.824112+0000","last_became_active":"2023-10-26T06:45:05.904527+0000","last_became_peered":"2023-10-26T06:45:05.904527+0000","last_unstale":"2023-10-26T06:45:07.824112+0000","last_undegraded":"2023-10-26T06:45:07.824112+0000","last_fullsized":"2023-10-26T06:45:07.824112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:30.546198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502122+0000","last_change":"2023-10-26T06:45:04.922832+0000","last_active":"2023-10-26T06:45:07.502122+0000","last_peered":"2023-10-26T06:45:07.502122+0000","last_clean":"2023-10-26T06:45:07.502122+0000","last_became_active":"2023-10-26T06:45:04.922349+0000","last_became_peered":"2023-10-26T06:45:04.922349+0000","last_unstale":"2023-10-26T06:45:07.502122+0000","last_undegraded":"2023-10-26T06:45:07.502122+0000","last_fullsized":"2023-10-26T06:45:07.502122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:37.839093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823836+0000","last_change":"2023-10-26T06:45:05.909946+0000","last_active":"2023-10-26T06:45:07.823836+0000","last_peered":"2023-10-26T06:45:07.823836+0000","last_clean":"2023-10-26T06:45:07.823836+0000","last_became_active":"2023-10-26T06:45:05.909650+0000","last_became_peered":"2023-10-26T06:45:05.909650+0000","last_unstale":"2023-10-26T06:45:07.823836+0000","last_undegraded":"2023-10-26T06:45:07.823836+0000","last_fullsized":"2023-10-26T06:45:07.823836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:41.813928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822476+0000","last_change":"2023-10-26T06:45:05.903400+0000","last_active":"2023-10-26T06:45:07.822476+0000","last_peered":"2023-10-26T06:45:07.822476+0000","last_clean":"2023-10-26T06:45:07.822476+0000","last_became_active":"2023-10-26T06:45:05.903101+0000","last_became_peered":"2023-10-26T06:45:05.903101+0000","last_unstale":"2023-10-26T06:45:07.822476+0000","last_undegraded":"2023-10-26T06:45:07.822476+0000","last_fullsized":"2023-10-26T06:45:07.822476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534251+0000","last_change":"2023-10-26T06:45:04.923960+0000","last_active":"2023-10-26T06:45:06.534251+0000","last_peered":"2023-10-26T06:45:06.534251+0000","last_clean":"2023-10-26T06:45:06.534251+0000","last_became_active":"2023-10-26T06:45:04.923613+0000","last_became_peered":"2023-10-26T06:45:04.923613+0000","last_unstale":"2023-10-26T06:45:06.534251+0000","last_undegraded":"2023-10-26T06:45:06.534251+0000","last_fullsized":"2023-10-26T06:45:06.534251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:53.342463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138101+0000","last_change":"2023-10-26T06:45:05.903966+0000","last_active":"2023-10-26T06:45:07.138101+0000","last_peered":"2023-10-26T06:45:07.138101+0000","last_clean":"2023-10-26T06:45:07.138101+0000","last_became_active":"2023-10-26T06:45:05.903583+0000","last_became_peered":"2023-10-26T06:45:05.903583+0000","last_unstale":"2023-10-26T06:45:07.138101+0000","last_undegraded":"2023-10-26T06:45:07.138101+0000","last_fullsized":"2023-10-26T06:45:07.138101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:18.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.887855+0000","last_change":"2023-10-26T06:45:04.907838+0000","last_active":"2023-10-26T06:45:05.887855+0000","last_peered":"2023-10-26T06:45:05.887855+0000","last_clean":"2023-10-26T06:45:05.887855+0000","last_became_active":"2023-10-26T06:45:04.907508+0000","last_became_peered":"2023-10-26T06:45:04.907508+0000","last_unstale":"2023-10-26T06:45:05.887855+0000","last_undegraded":"2023-10-26T06:45:05.887855+0000","last_fullsized":"2023-10-26T06:45:05.887855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:34.852243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":319,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.892156+0000","last_change":"2023-10-26T06:43:59.765466+0000","last_active":"2023-10-26T06:45:05.892156+0000","last_peered":"2023-10-26T06:45:05.892156+0000","last_clean":"2023-10-26T06:45:05.892156+0000","last_became_active":"2023-10-26T06:43:59.765005+0000","last_became_peered":"2023-10-26T06:43:59.765005+0000","last_unstale":"2023-10-26T06:45:05.892156+0000","last_undegraded":"2023-10-26T06:45:05.892156+0000","last_fullsized":"2023-10-26T06:45:05.892156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_clean_scrub_stamp":"2023-10-26T06:40:50.556448+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:18.688941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.821877+0000","last_change":"2023-10-26T06:45:05.906909+0000","last_active":"2023-10-26T06:45:07.821877+0000","last_peered":"2023-10-26T06:45:07.821877+0000","last_clean":"2023-10-26T06:45:07.821877+0000","last_became_active":"2023-10-26T06:45:05.906531+0000","last_became_peered":"2023-10-26T06:45:05.906531+0000","last_unstale":"2023-10-26T06:45:07.821877+0000","last_undegraded":"2023-10-26T06:45:07.821877+0000","last_fullsized":"2023-10-26T06:45:07.821877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.502092+0000","last_change":"2023-10-26T06:45:04.911635+0000","last_active":"2023-10-26T06:45:07.502092+0000","last_peered":"2023-10-26T06:45:07.502092+0000","last_clean":"2023-10-26T06:45:07.502092+0000","last_became_active":"2023-10-26T06:45:04.911289+0000","last_became_peered":"2023-10-26T06:45:04.911289+0000","last_unstale":"2023-10-26T06:45:07.502092+0000","last_undegraded":"2023-10-26T06:45:07.502092+0000","last_fullsized":"2023-10-26T06:45:07.502092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.115211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888625+0000","last_change":"2023-10-26T06:45:04.919853+0000","last_active":"2023-10-26T06:45:05.888625+0000","last_peered":"2023-10-26T06:45:05.888625+0000","last_clean":"2023-10-26T06:45:05.888625+0000","last_became_active":"2023-10-26T06:45:04.919488+0000","last_became_peered":"2023-10-26T06:45:04.919488+0000","last_unstale":"2023-10-26T06:45:05.888625+0000","last_undegraded":"2023-10-26T06:45:05.888625+0000","last_fullsized":"2023-10-26T06:45:05.888625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:06.555851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822290+0000","last_change":"2023-10-26T06:45:05.903975+0000","last_active":"2023-10-26T06:45:07.822290+0000","last_peered":"2023-10-26T06:45:07.822290+0000","last_clean":"2023-10-26T06:45:07.822290+0000","last_became_active":"2023-10-26T06:45:05.903539+0000","last_became_peered":"2023-10-26T06:45:05.903539+0000","last_unstale":"2023-10-26T06:45:07.822290+0000","last_undegraded":"2023-10-26T06:45:07.822290+0000","last_fullsized":"2023-10-26T06:45:07.822290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:30.648947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'3","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:13.944339+0000","last_change":"2023-10-26T06:45:04.919815+0000","last_active":"2023-10-26T06:45:13.944339+0000","last_peered":"2023-10-26T06:45:13.944339+0000","last_clean":"2023-10-26T06:45:13.944339+0000","last_became_active":"2023-10-26T06:45:04.918141+0000","last_became_peered":"2023-10-26T06:45:04.918141+0000","last_unstale":"2023-10-26T06:45:13.944339+0000","last_undegraded":"2023-10-26T06:45:13.944339+0000","last_fullsized":"2023-10-26T06:45:13.944339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:53.024534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138397+0000","last_change":"2023-10-26T06:45:05.898180+0000","last_active":"2023-10-26T06:45:07.138397+0000","last_peered":"2023-10-26T06:45:07.138397+0000","last_clean":"2023-10-26T06:45:07.138397+0000","last_became_active":"2023-10-26T06:45:05.896937+0000","last_became_peered":"2023-10-26T06:45:05.896937+0000","last_unstale":"2023-10-26T06:45:07.138397+0000","last_undegraded":"2023-10-26T06:45:07.138397+0000","last_fullsized":"2023-10-26T06:45:07.138397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:48.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.061365+0000","last_change":"2023-10-26T06:45:04.922456+0000","last_active":"2023-10-26T06:45:06.061365+0000","last_peered":"2023-10-26T06:45:06.061365+0000","last_clean":"2023-10-26T06:45:06.061365+0000","last_became_active":"2023-10-26T06:45:04.922112+0000","last_became_peered":"2023-10-26T06:45:04.922112+0000","last_unstale":"2023-10-26T06:45:06.061365+0000","last_undegraded":"2023-10-26T06:45:06.061365+0000","last_fullsized":"2023-10-26T06:45:06.061365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:44.055569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505613+0000","last_change":"2023-10-26T06:45:05.895440+0000","last_active":"2023-10-26T06:45:07.505613+0000","last_peered":"2023-10-26T06:45:07.505613+0000","last_clean":"2023-10-26T06:45:07.505613+0000","last_became_active":"2023-10-26T06:45:05.892605+0000","last_became_peered":"2023-10-26T06:45:05.892605+0000","last_unstale":"2023-10-26T06:45:07.505613+0000","last_undegraded":"2023-10-26T06:45:07.505613+0000","last_fullsized":"2023-10-26T06:45:07.505613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:09:27.392772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.899985+0000","last_change":"2023-10-26T06:45:04.916042+0000","last_active":"2023-10-26T06:45:05.899985+0000","last_peered":"2023-10-26T06:45:05.899985+0000","last_clean":"2023-10-26T06:45:05.899985+0000","last_became_active":"2023-10-26T06:45:04.915649+0000","last_became_peered":"2023-10-26T06:45:04.915649+0000","last_unstale":"2023-10-26T06:45:05.899985+0000","last_undegraded":"2023-10-26T06:45:05.899985+0000","last_fullsized":"2023-10-26T06:45:05.899985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:30.060074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689096+0000","last_change":"2023-10-26T06:45:05.908179+0000","last_active":"2023-10-26T06:45:07.689096+0000","last_peered":"2023-10-26T06:45:07.689096+0000","last_clean":"2023-10-26T06:45:07.689096+0000","last_became_active":"2023-10-26T06:45:05.907857+0000","last_became_peered":"2023-10-26T06:45:05.907857+0000","last_unstale":"2023-10-26T06:45:07.689096+0000","last_undegraded":"2023-10-26T06:45:07.689096+0000","last_fullsized":"2023-10-26T06:45:07.689096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:20.017194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.080625+0000","last_change":"2023-10-26T06:45:04.921016+0000","last_active":"2023-10-26T06:45:07.080625+0000","last_peered":"2023-10-26T06:45:07.080625+0000","last_clean":"2023-10-26T06:45:07.080625+0000","last_became_active":"2023-10-26T06:45:04.920741+0000","last_became_peered":"2023-10-26T06:45:04.920741+0000","last_unstale":"2023-10-26T06:45:07.080625+0000","last_undegraded":"2023-10-26T06:45:07.080625+0000","last_fullsized":"2023-10-26T06:45:07.080625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:05:06.218205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599772+0000","last_change":"2023-10-26T06:45:05.893562+0000","last_active":"2023-10-26T06:45:08.599772+0000","last_peered":"2023-10-26T06:45:08.599772+0000","last_clean":"2023-10-26T06:45:08.599772+0000","last_became_active":"2023-10-26T06:45:05.893111+0000","last_became_peered":"2023-10-26T06:45:05.893111+0000","last_unstale":"2023-10-26T06:45:08.599772+0000","last_undegraded":"2023-10-26T06:45:08.599772+0000","last_fullsized":"2023-10-26T06:45:08.599772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.305664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888454+0000","last_change":"2023-10-26T06:45:04.914936+0000","last_active":"2023-10-26T06:45:05.888454+0000","last_peered":"2023-10-26T06:45:05.888454+0000","last_clean":"2023-10-26T06:45:05.888454+0000","last_became_active":"2023-10-26T06:45:04.914617+0000","last_became_peered":"2023-10-26T06:45:04.914617+0000","last_unstale":"2023-10-26T06:45:05.888454+0000","last_undegraded":"2023-10-26T06:45:05.888454+0000","last_fullsized":"2023-10-26T06:45:05.888454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:15.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138153+0000","last_change":"2023-10-26T06:45:05.909101+0000","last_active":"2023-10-26T06:45:07.138153+0000","last_peered":"2023-10-26T06:45:07.138153+0000","last_clean":"2023-10-26T06:45:07.138153+0000","last_became_active":"2023-10-26T06:45:05.908800+0000","last_became_peered":"2023-10-26T06:45:05.908800+0000","last_unstale":"2023-10-26T06:45:07.138153+0000","last_undegraded":"2023-10-26T06:45:07.138153+0000","last_fullsized":"2023-10-26T06:45:07.138153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:37:18.605102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.689147+0000","last_change":"2023-10-26T06:45:04.911714+0000","last_active":"2023-10-26T06:45:07.689147+0000","last_peered":"2023-10-26T06:45:07.689147+0000","last_clean":"2023-10-26T06:45:07.689147+0000","last_became_active":"2023-10-26T06:45:04.911343+0000","last_became_peered":"2023-10-26T06:45:04.911343+0000","last_unstale":"2023-10-26T06:45:07.689147+0000","last_undegraded":"2023-10-26T06:45:07.689147+0000","last_fullsized":"2023-10-26T06:45:07.689147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:30:35.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823521+0000","last_change":"2023-10-26T06:45:05.906804+0000","last_active":"2023-10-26T06:45:07.823521+0000","last_peered":"2023-10-26T06:45:07.823521+0000","last_clean":"2023-10-26T06:45:07.823521+0000","last_became_active":"2023-10-26T06:45:05.906522+0000","last_became_peered":"2023-10-26T06:45:05.906522+0000","last_unstale":"2023-10-26T06:45:07.823521+0000","last_undegraded":"2023-10-26T06:45:07.823521+0000","last_fullsized":"2023-10-26T06:45:07.823521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:32.707204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:05.888946+0000","last_change":"2023-10-26T06:45:04.906345+0000","last_active":"2023-10-26T06:45:05.888946+0000","last_peered":"2023-10-26T06:45:05.888946+0000","last_clean":"2023-10-26T06:45:05.888946+0000","last_became_active":"2023-10-26T06:45:04.906034+0000","last_became_peered":"2023-10-26T06:45:04.906034+0000","last_unstale":"2023-10-26T06:45:05.888946+0000","last_undegraded":"2023-10-26T06:45:05.888946+0000","last_fullsized":"2023-10-26T06:45:05.888946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:53.690788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823815+0000","last_change":"2023-10-26T06:45:05.906368+0000","last_active":"2023-10-26T06:45:07.823815+0000","last_peered":"2023-10-26T06:45:07.823815+0000","last_clean":"2023-10-26T06:45:07.823815+0000","last_became_active":"2023-10-26T06:45:05.905471+0000","last_became_peered":"2023-10-26T06:45:05.905471+0000","last_unstale":"2023-10-26T06:45:07.823815+0000","last_undegraded":"2023-10-26T06:45:07.823815+0000","last_fullsized":"2023-10-26T06:45:07.823815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:45.493704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822626+0000","last_change":"2023-10-26T06:45:04.917530+0000","last_active":"2023-10-26T06:45:07.822626+0000","last_peered":"2023-10-26T06:45:07.822626+0000","last_clean":"2023-10-26T06:45:07.822626+0000","last_became_active":"2023-10-26T06:45:04.917193+0000","last_became_peered":"2023-10-26T06:45:04.917193+0000","last_unstale":"2023-10-26T06:45:07.822626+0000","last_undegraded":"2023-10-26T06:45:07.822626+0000","last_fullsized":"2023-10-26T06:45:07.822626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.965048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.823728+0000","last_change":"2023-10-26T06:45:05.899429+0000","last_active":"2023-10-26T06:45:07.823728+0000","last_peered":"2023-10-26T06:45:07.823728+0000","last_clean":"2023-10-26T06:45:07.823728+0000","last_became_active":"2023-10-26T06:45:05.899081+0000","last_became_peered":"2023-10-26T06:45:05.899081+0000","last_unstale":"2023-10-26T06:45:07.823728+0000","last_undegraded":"2023-10-26T06:45:07.823728+0000","last_fullsized":"2023-10-26T06:45:07.823728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:41.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.534374+0000","last_change":"2023-10-26T06:45:04.920295+0000","last_active":"2023-10-26T06:45:06.534374+0000","last_peered":"2023-10-26T06:45:06.534374+0000","last_clean":"2023-10-26T06:45:06.534374+0000","last_became_active":"2023-10-26T06:45:04.919920+0000","last_became_peered":"2023-10-26T06:45:04.919920+0000","last_unstale":"2023-10-26T06:45:06.534374+0000","last_undegraded":"2023-10-26T06:45:06.534374+0000","last_fullsized":"2023-10-26T06:45:06.534374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:59.260841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.824231+0000","last_change":"2023-10-26T06:45:05.899610+0000","last_active":"2023-10-26T06:45:07.824231+0000","last_peered":"2023-10-26T06:45:07.824231+0000","last_clean":"2023-10-26T06:45:07.824231+0000","last_became_active":"2023-10-26T06:45:05.899283+0000","last_became_peered":"2023-10-26T06:45:05.899283+0000","last_unstale":"2023-10-26T06:45:07.824231+0000","last_undegraded":"2023-10-26T06:45:07.824231+0000","last_fullsized":"2023-10-26T06:45:07.824231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:48.730703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599798+0000","last_change":"2023-10-26T06:45:05.892012+0000","last_active":"2023-10-26T06:45:08.599798+0000","last_peered":"2023-10-26T06:45:08.599798+0000","last_clean":"2023-10-26T06:45:08.599798+0000","last_became_active":"2023-10-26T06:45:05.891744+0000","last_became_peered":"2023-10-26T06:45:05.891744+0000","last_unstale":"2023-10-26T06:45:08.599798+0000","last_undegraded":"2023-10-26T06:45:08.599798+0000","last_fullsized":"2023-10-26T06:45:08.599798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:29.599184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884873+0000","last_change":"2023-10-26T06:45:05.904348+0000","last_active":"2023-10-26T06:45:06.884873+0000","last_peered":"2023-10-26T06:45:06.884873+0000","last_clean":"2023-10-26T06:45:06.884873+0000","last_became_active":"2023-10-26T06:45:05.903949+0000","last_became_peered":"2023-10-26T06:45:05.903949+0000","last_unstale":"2023-10-26T06:45:06.884873+0000","last_undegraded":"2023-10-26T06:45:06.884873+0000","last_fullsized":"2023-10-26T06:45:06.884873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:29.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.884744+0000","last_change":"2023-10-26T06:45:05.901313+0000","last_active":"2023-10-26T06:45:06.884744+0000","last_peered":"2023-10-26T06:45:06.884744+0000","last_clean":"2023-10-26T06:45:06.884744+0000","last_became_active":"2023-10-26T06:45:05.900993+0000","last_became_peered":"2023-10-26T06:45:05.900993+0000","last_unstale":"2023-10-26T06:45:06.884744+0000","last_undegraded":"2023-10-26T06:45:06.884744+0000","last_fullsized":"2023-10-26T06:45:06.884744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:00.093645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.505407+0000","last_change":"2023-10-26T06:45:05.901225+0000","last_active":"2023-10-26T06:45:07.505407+0000","last_peered":"2023-10-26T06:45:07.505407+0000","last_clean":"2023-10-26T06:45:07.505407+0000","last_became_active":"2023-10-26T06:45:05.900743+0000","last_became_peered":"2023-10-26T06:45:05.900743+0000","last_unstale":"2023-10-26T06:45:07.505407+0000","last_undegraded":"2023-10-26T06:45:07.505407+0000","last_fullsized":"2023-10-26T06:45:07.505407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:02.359654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822222+0000","last_change":"2023-10-26T06:45:05.906805+0000","last_active":"2023-10-26T06:45:07.822222+0000","last_peered":"2023-10-26T06:45:07.822222+0000","last_clean":"2023-10-26T06:45:07.822222+0000","last_became_active":"2023-10-26T06:45:05.906464+0000","last_became_peered":"2023-10-26T06:45:05.906464+0000","last_unstale":"2023-10-26T06:45:07.822222+0000","last_undegraded":"2023-10-26T06:45:07.822222+0000","last_fullsized":"2023-10-26T06:45:07.822222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:24.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599992+0000","last_change":"2023-10-26T06:45:05.898568+0000","last_active":"2023-10-26T06:45:08.599992+0000","last_peered":"2023-10-26T06:45:08.599992+0000","last_clean":"2023-10-26T06:45:08.599992+0000","last_became_active":"2023-10-26T06:45:05.897886+0000","last_became_peered":"2023-10-26T06:45:05.897886+0000","last_unstale":"2023-10-26T06:45:08.599992+0000","last_undegraded":"2023-10-26T06:45:08.599992+0000","last_fullsized":"2023-10-26T06:45:08.599992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:14.998233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885177+0000","last_change":"2023-10-26T06:45:05.899326+0000","last_active":"2023-10-26T06:45:06.885177+0000","last_peered":"2023-10-26T06:45:06.885177+0000","last_clean":"2023-10-26T06:45:06.885177+0000","last_became_active":"2023-10-26T06:45:05.899040+0000","last_became_peered":"2023-10-26T06:45:05.899040+0000","last_unstale":"2023-10-26T06:45:06.885177+0000","last_undegraded":"2023-10-26T06:45:06.885177+0000","last_fullsized":"2023-10-26T06:45:06.885177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:55:35.666122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.138250+0000","last_change":"2023-10-26T06:45:05.904155+0000","last_active":"2023-10-26T06:45:07.138250+0000","last_peered":"2023-10-26T06:45:07.138250+0000","last_clean":"2023-10-26T06:45:07.138250+0000","last_became_active":"2023-10-26T06:45:05.903734+0000","last_became_peered":"2023-10-26T06:45:05.903734+0000","last_unstale":"2023-10-26T06:45:07.138250+0000","last_undegraded":"2023-10-26T06:45:07.138250+0000","last_fullsized":"2023-10-26T06:45:07.138250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:43.252789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822424+0000","last_change":"2023-10-26T06:45:05.897174+0000","last_active":"2023-10-26T06:45:07.822424+0000","last_peered":"2023-10-26T06:45:07.822424+0000","last_clean":"2023-10-26T06:45:07.822424+0000","last_became_active":"2023-10-26T06:45:05.896893+0000","last_became_peered":"2023-10-26T06:45:05.896893+0000","last_unstale":"2023-10-26T06:45:07.822424+0000","last_undegraded":"2023-10-26T06:45:07.822424+0000","last_fullsized":"2023-10-26T06:45:07.822424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:14.291682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.137904+0000","last_change":"2023-10-26T06:45:05.905987+0000","last_active":"2023-10-26T06:45:07.137904+0000","last_peered":"2023-10-26T06:45:07.137904+0000","last_clean":"2023-10-26T06:45:07.137904+0000","last_became_active":"2023-10-26T06:45:05.904933+0000","last_became_peered":"2023-10-26T06:45:05.904933+0000","last_unstale":"2023-10-26T06:45:07.137904+0000","last_undegraded":"2023-10-26T06:45:07.137904+0000","last_fullsized":"2023-10-26T06:45:07.137904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:19.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:06.885189+0000","last_change":"2023-10-26T06:45:05.912781+0000","last_active":"2023-10-26T06:45:06.885189+0000","last_peered":"2023-10-26T06:45:06.885189+0000","last_clean":"2023-10-26T06:45:06.885189+0000","last_became_active":"2023-10-26T06:45:05.912415+0000","last_became_peered":"2023-10-26T06:45:05.912415+0000","last_unstale":"2023-10-26T06:45:06.885189+0000","last_undegraded":"2023-10-26T06:45:06.885189+0000","last_fullsized":"2023-10-26T06:45:06.885189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:25.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:07.822121+0000","last_change":"2023-10-26T06:45:05.909106+0000","last_active":"2023-10-26T06:45:07.822121+0000","last_peered":"2023-10-26T06:45:07.822121+0000","last_clean":"2023-10-26T06:45:07.822121+0000","last_became_active":"2023-10-26T06:45:05.908751+0000","last_became_peered":"2023-10-26T06:45:05.908751+0000","last_unstale":"2023-10-26T06:45:07.822121+0000","last_undegraded":"2023-10-26T06:45:07.822121+0000","last_fullsized":"2023-10-26T06:45:07.822121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:17.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T06:45:08.599847+0000","last_change":"2023-10-26T06:45:05.898262+0000","last_active":"2023-10-26T06:45:08.599847+0000","last_peered":"2023-10-26T06:45:08.599847+0000","last_clean":"2023-10-26T06:45:08.599847+0000","last_became_active":"2023-10-26T06:45:05.897947+0000","last_became_peered":"2023-10-26T06:45:05.897947+0000","last_unstale":"2023-10-26T06:45:08.599847+0000","last_undegraded":"2023-10-26T06:45:08.599847+0000","last_fullsized":"2023-10-26T06:45:08.599847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:39:45.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59373,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":111,"num_read_kb":114,"num_write":68,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":178119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416407,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":444588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.751}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Oct 26 06:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.86699999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645340,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1093848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44500000000000001,"15min":0.44400000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82499999999999996,"5min":1.004,"15min":1.004},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79800000000000004,"5min":0.874,"15min":0.874},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.94099999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.875}]},{"osd":7,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41699999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.752},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95599999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.91300000000000003}]},{"osd":9,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.046,"5min":1.133,"15min":1.133},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95699999999999996}]},{"osd":11,"last update":"Thu Oct 26 06:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.995}]}]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":445170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56100000000000005,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.57699999999999996,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72499999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":1.236}]},{"osd":1,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95699999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93799999999999994,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.012}]},{"osd":2,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.002,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.48899999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.872,"5min":1.03,"15min":1.03},"last":0.84199999999999997}]},{"osd":3,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.369},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.83799999999999997,"15min":0.85399999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.2729999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.84499999999999997,"15min":0.86499999999999999},"min":{"1min":0.55200000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93100000000000005,"5min":1.173,"15min":1.173},"last":1.397},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.87},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97199999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0940000000000001,"5min":1.379,"15min":1.379},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.88800000000000001,"15min":0.91100000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.292,"5min":1.327,"15min":1.327},"last":1.0980000000000001}]},{"osd":7,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.92700000000000005,"15min":0.94899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.143,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.304},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.94699999999999995,"15min":0.96899999999999997},"min":{"1min":0.59399999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.044,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.751}]},{"osd":8,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.95699999999999996,"15min":0.97099999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.93700000000000006,"15min":0.94499999999999995},"min":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.4330000000000001}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.337}]}]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":444861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.749,"15min":0.78700000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.439,"15min":0.443},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47899999999999998,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67900000000000005,"5min":0.752,"15min":0.752},"last":0.439}]},{"osd":3,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88500000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.626,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.88,"15min":0.88},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85699999999999998,"5min":1.095,"15min":1.095},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.85599999999999998,"15min":0.89500000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.79100000000000004,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78100000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.81000000000000005,"5min":0.87,"15min":0.87},"last":0.76600000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.745},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72199999999999998}]},{"osd":10,"last update":"Thu Oct 26 06:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1035661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41699999999999998,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60299999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78500000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.748,"15min":0.748},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.51100000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.69699999999999995,"15min":0.68700000000000006},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Thu Oct 26 06:45:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.01,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.57799999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95199999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69199999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462958,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":444711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.379,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.54500000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.624}]},{"osd":3,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.57299999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.55500000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.63300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Oct 26 06:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.82399999999999995,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":5.532,"5min":5.532,"15min":5.532},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":0.79900000000000004,"15min":0.73299999999999998},"min":{"1min":0.56499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":5.5810000000000004,"5min":5.5810000000000004,"15min":5.5810000000000004},"last":0.70799999999999996}]},{"osd":8,"last update":"Thu Oct 26 06:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.53100000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89700000000000002,"5min":1.089,"15min":1.089},"last":0.745},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.64200000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.80500000000000005,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68999999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83899999999999997,"15min":0.83999999999999997},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84699999999999998,"15min":0.84599999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.66200000000000003}]},{"osd":10,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73699999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":503163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.53500000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.80900000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.33700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83999999999999997,"5min":1.256,"15min":1.256},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73499999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.40200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.58399999999999996,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Oct 26 06:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.45500000000000002,"15min":0.442},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.42799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Oct 26 06:45:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47099999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.496},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Oct 26 06:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81599999999999995,"5min":1.03,"15min":1.03},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Oct 26 06:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79400000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86699999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73499999999999999}]},{"osd":9,"last update":"Thu Oct 26 06:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":1.298,"15min":1.298},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.57299999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.877}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":444678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 06:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.626,"15min":0.58999999999999997},"min":{"1min":0.47199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.601},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79700000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":1.135}]},{"osd":2,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.434,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93400000000000005,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90800000000000003,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.5800000000000001}]},{"osd":3,"last update":"Thu Oct 26 06:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":2.198,"15min":2.198},"last":1.5229999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.97599999999999998,"5min":1.48,"15min":1.48},"last":1.617}]},{"osd":4,"last update":"Thu Oct 26 06:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.651},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4450000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88400000000000001,"5min":1.002,"15min":1.002},"last":1.458},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.6339999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64600000000000002,"15min":0.61399999999999999},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69499999999999995,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.6919999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.84199999999999997,"15min":0.85499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.089,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.486},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91100000000000003,"15min":0.92000000000000004},"min":{"1min":0.66300000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.131,"5min":1.294,"15min":1.294},"last":1.673}]},{"osd":9,"last update":"Thu Oct 26 06:45:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.89500000000000002},"min":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.544}]},{"osd":10,"last update":"Thu Oct 26 06:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":1.7250000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.214}]},{"osd":11,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.5669999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":1.7090000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378689,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":502528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58099999999999996,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":0.75},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.56499999999999995,"15min":0.52300000000000002},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Oct 26 06:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.56999999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.46100000000000002},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.84699999999999998}]},{"osd":6,"last update":"Thu Oct 26 06:45:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Oct 26 06:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.78700000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.997,"15min":0.997},"last":1.3480000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.3919999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Oct 26 06:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.78400000000000003,"15min":0.77100000000000002},"min":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.431,"5min":3.431,"15min":3.431},"last":0.86199999999999999}]},{"osd":10,"last update":"Thu Oct 26 06:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.90300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.42999999999999999,"15min":0.40400000000000003},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.47599999999999998,"15min":0.44900000000000001},"min":{"1min":0.35399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.63700000000000001},"min":{"1min":0.47399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.083,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Oct 26 06:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.47699999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67600000000000005,"15min":0.66100000000000003},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 06:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.64600000000000002},"min":{"1min":0.46700000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1140000000000001,"5min":1.304,"15min":1.304},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.68400000000000005},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72999999999999998,"15min":0.69499999999999995},"min":{"1min":0.58999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.73599999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.84399999999999997,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84299999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":444610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.434}]},{"osd":1,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.35299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.57899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40899999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.442},{"interface":"front","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66400000000000003,"5min":1.222,"15min":1.222},"last":0.623},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Oct 26 06:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61799999999999999,"15min":0.59199999999999997},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Oct 26 06:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.47799999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92000000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.66500000000000004},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84599999999999997,"5min":1.403,"15min":1.403},"last":0.745}]},{"osd":8,"last update":"Thu Oct 26 06:45:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70899999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69099999999999995,"15min":0.68600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85299999999999998,"5min":1.026,"15min":1.026},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.95699999999999996,"5min":1.175,"15min":1.175},"last":0.79400000000000004}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94099999999999995,"5min":1.141,"15min":1.141},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91700000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724596,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1515520,"data_stored":1035261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.48199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.44,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.47499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.32500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.498},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.438,"15min":0.43099999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77300000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.59199999999999997,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Oct 26 06:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65700000000000003,"15min":0.64100000000000001},"min":{"1min":0.57399999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67300000000000004,"15min":0.65400000000000003},"min":{"1min":0.49299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84399999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.56200000000000006,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.86899999999999999}]},{"osd":8,"last update":"Thu Oct 26 06:45:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.38400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.754}]},{"osd":9,"last update":"Thu Oct 26 06:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.66000000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.92000000000000004}]},{"osd":10,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.66500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":58769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-26T06:46:02.501 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-26T06:46:02.501 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-26T06:46:02.501 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-26T06:46:02.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph health --format=json 2023-10-26T06:46:02.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-10-26T06:46:02.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-10-26T06:46:02.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-26T06:46:02.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-26T06:46:02.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-10-26T06:46:02.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-10-26T06:46:02.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2023-10-26T06:46:02.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-26T06:46:02.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:02 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-10-26T06:46:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:03.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: pgmap v282: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:03.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:03.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-26T06:46:03.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-10-26T06:46:03.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-26T06:46:03.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-26T06:46:03.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-10-26T06:46:03.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-10-26T06:46:03.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2023-10-26T06:46:03.535 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-26T06:46:03.535 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-10-26T06:46:03.535 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:03 smithi089 ceph-mon[95031]: osdmap e85: 12 total, 12 up, 12 in 2023-10-26T06:46:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: pgmap v282: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:03.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:03.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-26T06:46:03.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-10-26T06:46:03.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-26T06:46:03.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-26T06:46:03.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-10-26T06:46:03.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-10-26T06:46:03.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2023-10-26T06:46:03.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-26T06:46:03.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-10-26T06:46:03.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:03 smithi192 ceph-mon[131661]: osdmap e85: 12 total, 12 up, 12 in 2023-10-26T06:46:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: pgmap v282: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:03.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-10-26T06:46:03.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-10-26T06:46:03.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-26T06:46:03.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-26T06:46:03.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-10-26T06:46:03.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-10-26T06:46:03.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2023-10-26T06:46:03.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-26T06:46:03.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-10-26T06:46:03.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:03 smithi135 ceph-mon[131403]: osdmap e85: 12 total, 12 up, 12 in 2023-10-26T06:46:04.521 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:04 smithi089 ceph-mon[95031]: osdmap e86: 12 total, 12 up, 12 in 2023-10-26T06:46:04.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:04 smithi192 ceph-mon[131661]: osdmap e86: 12 total, 12 up, 12 in 2023-10-26T06:46:04.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:04 smithi135 ceph-mon[131403]: osdmap e86: 12 total, 12 up, 12 in 2023-10-26T06:46:04.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 -- 172.21.15.89:0/1004698350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 msgr2=0x7f22d8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 --2- 172.21.15.89:0/1004698350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8100d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009990 tx=0x7f22cc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 -- 172.21.15.89:0/1004698350 shutdown_connections 2023-10-26T06:46:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 --2- 172.21.15.89:0/1004698350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d8101260 0x7f22d810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 --2- 172.21.15.89:0/1004698350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d81057c0 0x7f22d8105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 --2- 172.21.15.89:0/1004698350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8100d20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 -- 172.21.15.89:0/1004698350 >> 172.21.15.89:0/1004698350 conn(0x7f22d80779e0 msgr2=0x7f22d8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 -- 172.21.15.89:0/1004698350 shutdown_connections 2023-10-26T06:46:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.721+0000 7f22dd05e700 1 -- 172.21.15.89:0/1004698350 wait complete. 2023-10-26T06:46:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.722+0000 7f22dd05e700 1 Processor -- start 2023-10-26T06:46:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.723+0000 7f22dd05e700 1 -- start start 2023-10-26T06:46:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.723+0000 7f22dd05e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.723+0000 7f22dd05e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8101260 0x7f22d8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.723+0000 7f22dd05e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d81057c0 0x7f22d8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.723+0000 7f22dd05e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81a4f90 con 0x7f22d81008a0 2023-10-26T06:46:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.723+0000 7f22dd05e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81a5100 con 0x7f22d8101260 2023-10-26T06:46:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.723+0000 7f22dd05e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81a52c0 con 0x7f22d81057c0 2023-10-26T06:46:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d659c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8101260 0x7f22d8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d659c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8101260 0x7f22d8073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:48024/0 (socket says 172.21.15.89:48024) 2023-10-26T06:46:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d659c700 1 -- 172.21.15.89:0/4089914919 learned_addr learned my addr 172.21.15.89:0/4089914919 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41864/0 (socket says 172.21.15.89:41864) 2023-10-26T06:46:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d759e700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d81057c0 0x7f22d8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 -- 172.21.15.89:0/4089914919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d81057c0 msgr2=0x7f22d8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d81057c0 0x7f22d8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 -- 172.21.15.89:0/4089914919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8101260 msgr2=0x7f22d8073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8101260 0x7f22d8073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 -- 172.21.15.89:0/4089914919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc0095f0 con 0x7f22d81008a0 2023-10-26T06:46:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d659c700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8101260 0x7f22d8073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22d6d9d700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8072af0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9c0 tx=0x7f22c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22bffff700 1 -- 172.21.15.89:0/4089914919 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8004220 con 0x7f22d81008a0 2023-10-26T06:46:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22bffff700 1 -- 172.21.15.89:0/4089914919 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c8004380 con 0x7f22d81008a0 2023-10-26T06:46:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d81a55a0 con 0x7f22d81008a0 2023-10-26T06:46:04.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.724+0000 7f22bffff700 1 -- 172.21.15.89:0/4089914919 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8003be0 con 0x7f22d81008a0 2023-10-26T06:46:04.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.725+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d81a5af0 con 0x7f22d81008a0 2023-10-26T06:46:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.726+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d8108da0 con 0x7f22d81008a0 2023-10-26T06:46:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.728+0000 7f22bffff700 1 -- 172.21.15.89:0/4089914919 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f22c8003d40 con 0x7f22d81008a0 2023-10-26T06:46:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.729+0000 7f22bffff700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f22c00a15e0 0x7f22c00a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.729+0000 7f22bffff700 1 -- 172.21.15.89:0/4089914919 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f22c8014070 con 0x7f22d81008a0 2023-10-26T06:46:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.729+0000 7f22d659c700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f22c00a15e0 0x7f22c00a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.730+0000 7f22d659c700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f22c00a15e0 0x7f22c00a3aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22d80741d0 tx=0x7f22cc00b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.731+0000 7f22bffff700 1 -- 172.21.15.89:0/4089914919 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f22c8096720 con 0x7f22d81008a0 2023-10-26T06:46:04.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.973+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f22d8073f80 con 0x7f22d81008a0 2023-10-26T06:46:04.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.973+0000 7f22bffff700 1 -- 172.21.15.89:0/4089914919 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f22c8099d40 con 0x7f22d81008a0 2023-10-26T06:46:04.976 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:04.976 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-26T06:46:04.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f22c00a15e0 msgr2=0x7f22c00a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:04.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f22c00a15e0 0x7f22c00a3aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22d80741d0 tx=0x7f22cc00b260 comp rx=0 tx=0).stop 2023-10-26T06:46:04.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 msgr2=0x7f22d8072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:04.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8072af0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9c0 tx=0x7f22c800ecd0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 shutdown_connections 2023-10-26T06:46:04.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f22c00a15e0 0x7f22c00a3aa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22d81008a0 0x7f22d8072af0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d81057c0 0x7f22d8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 --2- 172.21.15.89:0/4089914919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8101260 0x7f22d8073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:04.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 >> 172.21.15.89:0/4089914919 conn(0x7f22d80779e0 msgr2=0x7f22d80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:04.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 shutdown_connections 2023-10-26T06:46:04.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:04.976+0000 7f22dd05e700 1 -- 172.21.15.89:0/4089914919 wait complete. 2023-10-26T06:46:05.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:05 smithi089 ceph-mon[95031]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:05.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4089914919' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-26T06:46:05.426 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-26T06:46:05.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:05.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 -- 172.21.15.89:0/2753259825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c1066b0 msgr2=0x7fbf3c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 --2- 172.21.15.89:0/2753259825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c1066b0 0x7fbf3c106a90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009990 tx=0x7fbf30009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 -- 172.21.15.89:0/2753259825 shutdown_connections 2023-10-26T06:46:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 --2- 172.21.15.89:0/2753259825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c1066b0 0x7fbf3c106a90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 --2- 172.21.15.89:0/2753259825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c101060 0x7fbf3c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 --2- 172.21.15.89:0/2753259825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3c1006a0 0x7fbf3c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 -- 172.21.15.89:0/2753259825 >> 172.21.15.89:0/2753259825 conn(0x7fbf3c0fc170 msgr2=0x7fbf3c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 -- 172.21.15.89:0/2753259825 shutdown_connections 2023-10-26T06:46:05.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.627+0000 7fbf42bff700 1 -- 172.21.15.89:0/2753259825 wait complete. 2023-10-26T06:46:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.628+0000 7fbf42bff700 1 Processor -- start 2023-10-26T06:46:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf42bff700 1 -- start start 2023-10-26T06:46:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf42bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3c1006a0 0x7fbf3c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf42bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c101060 0x7fbf3c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf42bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c1066b0 0x7fbf3c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf42bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c196fe0 con 0x7fbf3c101060 2023-10-26T06:46:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf42bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c197150 con 0x7fbf3c1066b0 2023-10-26T06:46:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf42bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c197310 con 0x7fbf3c1006a0 2023-10-26T06:46:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf3bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c101060 0x7fbf3c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf3bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c101060 0x7fbf3c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41890/0 (socket says 172.21.15.89:41890) 2023-10-26T06:46:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf3bfff700 1 -- 172.21.15.89:0/1669943441 learned_addr learned my addr 172.21.15.89:0/1669943441 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:05.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf4099b700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3c1006a0 0x7fbf3c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.629+0000 7fbf4119c700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c1066b0 0x7fbf3c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf3bfff700 1 -- 172.21.15.89:0/1669943441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3c1006a0 msgr2=0x7fbf3c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:05.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf3bfff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3c1006a0 0x7fbf3c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf3bfff700 1 -- 172.21.15.89:0/1669943441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c1066b0 msgr2=0x7fbf3c196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf3bfff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c1066b0 0x7fbf3c196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf3bfff700 1 -- 172.21.15.89:0/1669943441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300095f0 con 0x7fbf3c101060 2023-10-26T06:46:05.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf3bfff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c101060 0x7fbf3c19cd40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800d980 tx=0x7fbf2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf39ffb700 1 -- 172.21.15.89:0/1669943441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28004220 con 0x7fbf3c101060 2023-10-26T06:46:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf39ffb700 1 -- 172.21.15.89:0/1669943441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf28007640 con 0x7fbf3c101060 2023-10-26T06:46:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c1975f0 con 0x7fbf3c101060 2023-10-26T06:46:05.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf39ffb700 1 -- 172.21.15.89:0/1669943441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28005690 con 0x7fbf3c101060 2023-10-26T06:46:05.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.630+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c1a37f0 con 0x7fbf3c101060 2023-10-26T06:46:05.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.631+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c108ca0 con 0x7fbf3c101060 2023-10-26T06:46:05.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.633+0000 7fbf39ffb700 1 -- 172.21.15.89:0/1669943441 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fbf28004750 con 0x7fbf3c101060 2023-10-26T06:46:05.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.634+0000 7fbf39ffb700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf2c0a15e0 0x7fbf2c0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:05.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.635+0000 7fbf39ffb700 1 -- 172.21.15.89:0/1669943441 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbf280ccd60 con 0x7fbf3c101060 2023-10-26T06:46:05.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.635+0000 7fbf4099b700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf2c0a15e0 0x7fbf2c0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:05.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.635+0000 7fbf4099b700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf2c0a15e0 0x7fbf2c0a3aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf30006010 tx=0x7fbf3000b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:05.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.639+0000 7fbf39ffb700 1 -- 172.21.15.89:0/1669943441 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf28096120 con 0x7fbf3c101060 2023-10-26T06:46:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:05 smithi192 ceph-mon[131661]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4089914919' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-26T06:46:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:05 smithi135 ceph-mon[131403]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4089914919' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-26T06:46:05.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.844+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf3c101de0 con 0x7fbf3c101060 2023-10-26T06:46:05.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.846+0000 7fbf39ffb700 1 -- 172.21.15.89:0/1669943441 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7fbf28016020 con 0x7fbf3c101060 2023-10-26T06:46:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:05.849 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:05.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf2c0a15e0 msgr2=0x7fbf2c0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf2c0a15e0 0x7fbf2c0a3aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf30006010 tx=0x7fbf3000b260 comp rx=0 tx=0).stop 2023-10-26T06:46:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c101060 msgr2=0x7fbf3c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c101060 0x7fbf3c19cd40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800d980 tx=0x7fbf2800dc90 comp rx=0 tx=0).stop 2023-10-26T06:46:05.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 shutdown_connections 2023-10-26T06:46:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf2c0a15e0 0x7fbf2c0a3aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c1066b0 0x7fbf3c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf3c1006a0 0x7fbf3c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 --2- 172.21.15.89:0/1669943441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf3c101060 0x7fbf3c19cd40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.848+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 >> 172.21.15.89:0/1669943441 conn(0x7fbf3c0fc170 msgr2=0x7fbf3c0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:05.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.849+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 shutdown_connections 2023-10-26T06:46:05.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:05.849+0000 7fbf42bff700 1 -- 172.21.15.89:0/1669943441 wait complete. 2023-10-26T06:46:05.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:46:05.871 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:46:06.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 -- 172.21.15.89:0/835663738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d78106720 msgr2=0x7f3d78106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/835663738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d78106720 0x7f3d78106b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009990 tx=0x7f3d68009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 -- 172.21.15.89:0/835663738 shutdown_connections 2023-10-26T06:46:06.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/835663738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d78106720 0x7f3d78106b00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/835663738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d781010d0 0x7f3d7810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/835663738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d78100710 0x7f3d78100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 -- 172.21.15.89:0/835663738 >> 172.21.15.89:0/835663738 conn(0x7f3d780fc200 msgr2=0x7f3d780fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:06.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 -- 172.21.15.89:0/835663738 shutdown_connections 2023-10-26T06:46:06.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.069+0000 7f3d7cdff700 1 -- 172.21.15.89:0/835663738 wait complete. 2023-10-26T06:46:06.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.070+0000 7f3d7cdff700 1 Processor -- start 2023-10-26T06:46:06.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d7cdff700 1 -- start start 2023-10-26T06:46:06.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d7cdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d78100710 0x7f3d7819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d7cdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d781010d0 0x7f3d7819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d7cdff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d78106720 0x7f3d781969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d7cdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78197020 con 0x7f3d781010d0 2023-10-26T06:46:06.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d7cdff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78197190 con 0x7f3d78106720 2023-10-26T06:46:06.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d7cdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78197350 con 0x7f3d78100710 2023-10-26T06:46:06.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.071+0000 7f3d75d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d781010d0 0x7f3d7819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d781010d0 0x7f3d7819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41916/0 (socket says 172.21.15.89:41916) 2023-10-26T06:46:06.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 -- 172.21.15.89:0/949494026 learned_addr learned my addr 172.21.15.89:0/949494026 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:06.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d7659c700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d78100710 0x7f3d7819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d76d9d700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d78106720 0x7f3d781969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 -- 172.21.15.89:0/949494026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d78100710 msgr2=0x7f3d7819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d78100710 0x7f3d7819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 -- 172.21.15.89:0/949494026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d78106720 msgr2=0x7f3d781969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d78106720 0x7f3d781969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 -- 172.21.15.89:0/949494026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d680095f0 con 0x7f3d781010d0 2023-10-26T06:46:06.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d75d9b700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d781010d0 0x7f3d7819cdd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00e9a0 tx=0x7f3d6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:06.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d677fe700 1 -- 172.21.15.89:0/949494026 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c009790 con 0x7f3d781010d0 2023-10-26T06:46:06.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d78197630 con 0x7f3d781010d0 2023-10-26T06:46:06.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d677fe700 1 -- 172.21.15.89:0/949494026 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d6c010430 con 0x7f3d781010d0 2023-10-26T06:46:06.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.072+0000 7f3d677fe700 1 -- 172.21.15.89:0/949494026 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c018620 con 0x7f3d781010d0 2023-10-26T06:46:06.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.073+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d781a3880 con 0x7f3d781010d0 2023-10-26T06:46:06.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.074+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d78108d10 con 0x7f3d781010d0 2023-10-26T06:46:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.075+0000 7f3d677fe700 1 -- 172.21.15.89:0/949494026 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f3d6c0105a0 con 0x7f3d781010d0 2023-10-26T06:46:06.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.077+0000 7f3d677fe700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3d600a1560 0x7f3d600a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.077+0000 7f3d677fe700 1 -- 172.21.15.89:0/949494026 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3d6c014070 con 0x7f3d781010d0 2023-10-26T06:46:06.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.077+0000 7f3d7659c700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3d600a1560 0x7f3d600a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.078+0000 7f3d7659c700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3d600a1560 0x7f3d600a3a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800b2e0 tx=0x7f3d68005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:06.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.080+0000 7f3d677fe700 1 -- 172.21.15.89:0/949494026 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3d6c052100 con 0x7f3d781010d0 2023-10-26T06:46:06.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.282+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d78197e80 con 0x7f3d781010d0 2023-10-26T06:46:06.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.282+0000 7f3d677fe700 1 -- 172.21.15.89:0/949494026 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d6c096540 con 0x7f3d781010d0 2023-10-26T06:46:06.285 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:46:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.284+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3d600a1560 msgr2=0x7f3d600a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.284+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3d600a1560 0x7f3d600a3a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800b2e0 tx=0x7f3d68005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.284+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d781010d0 msgr2=0x7f3d7819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.284+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d781010d0 0x7f3d7819cdd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00e9a0 tx=0x7f3d6c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 shutdown_connections 2023-10-26T06:46:06.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3d600a1560 0x7f3d600a3a20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d78106720 0x7f3d781969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d78100710 0x7f3d7819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 --2- 172.21.15.89:0/949494026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d781010d0 0x7f3d7819cdd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 >> 172.21.15.89:0/949494026 conn(0x7f3d780fc200 msgr2=0x7f3d780fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:06.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 shutdown_connections 2023-10-26T06:46:06.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.285+0000 7f3d7cdff700 1 -- 172.21.15.89:0/949494026 wait complete. 2023-10-26T06:46:06.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:46:06.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 -- 172.21.15.89:0/2117492582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 msgr2=0x7fbcc410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 --2- 172.21.15.89:0/2117492582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc410d5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00b0a0 tx=0x7fbcbc00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 -- 172.21.15.89:0/2117492582 shutdown_connections 2023-10-26T06:46:06.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 --2- 172.21.15.89:0/2117492582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc410d5f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 --2- 172.21.15.89:0/2117492582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4106730 0x7fbcc4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 --2- 172.21.15.89:0/2117492582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4100720 0x7fbcc4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 -- 172.21.15.89:0/2117492582 >> 172.21.15.89:0/2117492582 conn(0x7fbcc40fc1b0 msgr2=0x7fbcc40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:06.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 -- 172.21.15.89:0/2117492582 shutdown_connections 2023-10-26T06:46:06.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.505+0000 7fbccabff700 1 -- 172.21.15.89:0/2117492582 wait complete. 2023-10-26T06:46:06.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.506+0000 7fbccabff700 1 Processor -- start 2023-10-26T06:46:06.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbccabff700 1 -- start start 2023-10-26T06:46:06.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbccabff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4100720 0x7fbcc4198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbccabff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbccabff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4106730 0x7fbcc419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbccabff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc419dc90 con 0x7fbcc41010e0 2023-10-26T06:46:06.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbccabff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc419de00 con 0x7fbcc4106730 2023-10-26T06:46:06.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbccabff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc419dfc0 con 0x7fbcc4100720 2023-10-26T06:46:06.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.507+0000 7fbcc3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41928/0 (socket says 172.21.15.89:41928) 2023-10-26T06:46:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc3fff700 1 -- 172.21.15.89:0/1490696988 learned_addr learned my addr 172.21.15.89:0/1490696988 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc899b700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4100720 0x7fbcc4198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc919c700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4106730 0x7fbcc419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc3fff700 1 -- 172.21.15.89:0/1490696988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4100720 msgr2=0x7fbcc4198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc3fff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4100720 0x7fbcc4198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc3fff700 1 -- 172.21.15.89:0/1490696988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4106730 msgr2=0x7fbcc419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc3fff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4106730 0x7fbcc419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc3fff700 1 -- 172.21.15.89:0/1490696988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc009d00 con 0x7fbcc41010e0 2023-10-26T06:46:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc3fff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc41992b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d980 tx=0x7fbcb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc1ffb700 1 -- 172.21.15.89:0/1490696988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0004220 con 0x7fbcc41010e0 2023-10-26T06:46:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc1ffb700 1 -- 172.21.15.89:0/1490696988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb0007640 con 0x7fbcc41010e0 2023-10-26T06:46:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbcc1ffb700 1 -- 172.21.15.89:0/1490696988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0005690 con 0x7fbcc41010e0 2023-10-26T06:46:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.508+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc419e2a0 con 0x7fbcc41010e0 2023-10-26T06:46:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.509+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc41a5ae0 con 0x7fbcc41010e0 2023-10-26T06:46:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.509+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc4101e60 con 0x7fbcc41010e0 2023-10-26T06:46:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.511+0000 7fbcc1ffb700 1 -- 172.21.15.89:0/1490696988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fbcb0004750 con 0x7fbcc41010e0 2023-10-26T06:46:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.513+0000 7fbcc1ffb700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbcb40a15e0 0x7fbcb40a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.513+0000 7fbcc1ffb700 1 -- 172.21.15.89:0/1490696988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbcb00ccd60 con 0x7fbcc41010e0 2023-10-26T06:46:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.513+0000 7fbcc899b700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbcb40a15e0 0x7fbcb40a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.514+0000 7fbcc899b700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbcb40a15e0 0x7fbcb40a3aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009be0 tx=0x7fbcb8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.517+0000 7fbcc1ffb700 1 -- 172.21.15.89:0/1490696988 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbcb0096120 con 0x7fbcc41010e0 2023-10-26T06:46:06.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:06 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1669943441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:06.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:06 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1669943441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:06.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:06 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1669943441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:06.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.716+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbcc41106f0 con 0x7fbcc41010e0 2023-10-26T06:46:06.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.717+0000 7fbcc1ffb700 1 -- 172.21.15.89:0/1490696988 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbcb00160a0 con 0x7fbcc41010e0 2023-10-26T06:46:06.719 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:46:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbcb40a15e0 msgr2=0x7fbcb40a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbcb40a15e0 0x7fbcb40a3aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009be0 tx=0x7fbcb8009260 comp rx=0 tx=0).stop 2023-10-26T06:46:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 msgr2=0x7fbcc41992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc41992b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d980 tx=0x7fbcb000dc90 comp rx=0 tx=0).stop 2023-10-26T06:46:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 shutdown_connections 2023-10-26T06:46:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbcb40a15e0 0x7fbcb40a3aa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4106730 0x7fbcc419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcc4100720 0x7fbcc4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 --2- 172.21.15.89:0/1490696988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcc41010e0 0x7fbcc41992b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 >> 172.21.15.89:0/1490696988 conn(0x7fbcc40fc1b0 msgr2=0x7fbcc410be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.719+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 shutdown_connections 2023-10-26T06:46:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.720+0000 7fbccabff700 1 -- 172.21.15.89:0/1490696988 wait complete. 2023-10-26T06:46:06.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:46:06.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 -- 172.21.15.89:0/2002007912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c100f90 msgr2=0x7fd40c10d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 --2- 172.21.15.89:0/2002007912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c100f90 0x7fd40c10d450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd40400b0d0 tx=0x7fd40400b3e0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 -- 172.21.15.89:0/2002007912 shutdown_connections 2023-10-26T06:46:06.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 --2- 172.21.15.89:0/2002007912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c106690 0x7fd40c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 --2- 172.21.15.89:0/2002007912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c100f90 0x7fd40c10d450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 --2- 172.21.15.89:0/2002007912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c1005d0 0x7fd40c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 -- 172.21.15.89:0/2002007912 >> 172.21.15.89:0/2002007912 conn(0x7fd40c0fc0c0 msgr2=0x7fd40c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 -- 172.21.15.89:0/2002007912 shutdown_connections 2023-10-26T06:46:06.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.937+0000 7fd40bfff700 1 -- 172.21.15.89:0/2002007912 wait complete. 2023-10-26T06:46:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.938+0000 7fd40bfff700 1 Processor -- start 2023-10-26T06:46:06.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.938+0000 7fd40bfff700 1 -- start start 2023-10-26T06:46:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40bfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c1005d0 0x7fd40c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c100f90 0x7fd40c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c106690 0x7fd40c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40bfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c19dcb0 con 0x7fd40c106690 2023-10-26T06:46:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40bfff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c19de20 con 0x7fd40c1005d0 2023-10-26T06:46:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40bfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c19dfe0 con 0x7fd40c100f90 2023-10-26T06:46:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c100f90 0x7fd40c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40affd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c1005d0 0x7fd40c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c100f90 0x7fd40c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44200/0 (socket says 172.21.15.89:44200) 2023-10-26T06:46:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c106690 0x7fd40c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 -- 172.21.15.89:0/106699657 learned_addr learned my addr 172.21.15.89:0/106699657 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 -- 172.21.15.89:0/106699657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c1005d0 msgr2=0x7fd40c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c1005d0 0x7fd40c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 -- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c106690 msgr2=0x7fd40c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c106690 0x7fd40c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:06.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40a7fc700 1 -- 172.21.15.89:0/106699657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009d00 con 0x7fd40c100f90 2023-10-26T06:46:06.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40affd700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c1005d0 0x7fd40c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:06.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.939+0000 7fd40b7fe700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c106690 0x7fd40c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:06.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.940+0000 7fd40a7fc700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c100f90 0x7fd40c1992d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd40000e9a0 tx=0x7fd40000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:06.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.940+0000 7fd3f3fff700 1 -- 172.21.15.89:0/106699657 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400009790 con 0x7fd40c100f90 2023-10-26T06:46:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.940+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c19e2c0 con 0x7fd40c100f90 2023-10-26T06:46:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.940+0000 7fd3f3fff700 1 -- 172.21.15.89:0/106699657 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd400010430 con 0x7fd40c100f90 2023-10-26T06:46:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.940+0000 7fd3f3fff700 1 -- 172.21.15.89:0/106699657 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400018680 con 0x7fd40c100f90 2023-10-26T06:46:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.940+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c1a5b00 con 0x7fd40c100f90 2023-10-26T06:46:06.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.941+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c101cf0 con 0x7fd40c100f90 2023-10-26T06:46:06.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.943+0000 7fd3f3fff700 1 -- 172.21.15.89:0/106699657 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd400010810 con 0x7fd40c100f90 2023-10-26T06:46:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.944+0000 7fd3f3fff700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd3f40a1560 0x7fd3f40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.944+0000 7fd3f3fff700 1 -- 172.21.15.89:0/106699657 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd400014070 con 0x7fd40c100f90 2023-10-26T06:46:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.945+0000 7fd40affd700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd3f40a1560 0x7fd3f40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.945+0000 7fd40affd700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd3f40a1560 0x7fd3f40a3a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009fd0 tx=0x7fd3fc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:06.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:06.946+0000 7fd3f3fff700 1 -- 172.21.15.89:0/106699657 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd400052100 con 0x7fd40c100f90 2023-10-26T06:46:07.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.149+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd40c19a090 con 0x7fd40c100f90 2023-10-26T06:46:07.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.150+0000 7fd3f3fff700 1 -- 172.21.15.89:0/106699657 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd40009a0e0 con 0x7fd40c100f90 2023-10-26T06:46:07.153 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:46:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd3f40a1560 msgr2=0x7fd3f40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd3f40a1560 0x7fd3f40a3a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009fd0 tx=0x7fd3fc009260 comp rx=0 tx=0).stop 2023-10-26T06:46:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c100f90 msgr2=0x7fd40c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c100f90 0x7fd40c1992d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd40000e9a0 tx=0x7fd40000ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 shutdown_connections 2023-10-26T06:46:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd3f40a1560 0x7fd3f40a3a20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd40c106690 0x7fd40c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c1005d0 0x7fd40c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 --2- 172.21.15.89:0/106699657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd40c100f90 0x7fd40c1992d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.152+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 >> 172.21.15.89:0/106699657 conn(0x7fd40c0fc0c0 msgr2=0x7fd40c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.153+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 shutdown_connections 2023-10-26T06:46:07.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.153+0000 7fd40bfff700 1 -- 172.21.15.89:0/106699657 wait complete. 2023-10-26T06:46:07.173 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:46:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 -- 172.21.15.89:0/2765186052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 msgr2=0x7f2cb810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/2765186052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb810c9a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009990 tx=0x7f2ca8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 -- 172.21.15.89:0/2765186052 shutdown_connections 2023-10-26T06:46:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/2765186052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb810cee0 0x7f2cb810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/2765186052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cb8072c20 0x7f2cb8073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/2765186052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb810c9a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 -- 172.21.15.89:0/2765186052 >> 172.21.15.89:0/2765186052 conn(0x7f2cb80fc170 msgr2=0x7f2cb80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 -- 172.21.15.89:0/2765186052 shutdown_connections 2023-10-26T06:46:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.369+0000 7f2cc07ff700 1 -- 172.21.15.89:0/2765186052 wait complete. 2023-10-26T06:46:07.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.370+0000 7f2cc07ff700 1 Processor -- start 2023-10-26T06:46:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.370+0000 7f2cc07ff700 1 -- start start 2023-10-26T06:46:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cc07ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8072c20 0x7f2cb8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cc07ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cc07ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cb810cee0 0x7f2cb819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cc07ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819dca0 con 0x7f2cb8073540 2023-10-26T06:46:07.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cc07ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819de10 con 0x7f2cb8072c20 2023-10-26T06:46:07.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cc07ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819dfd0 con 0x7f2cb810cee0 2023-10-26T06:46:07.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb8199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41964/0 (socket says 172.21.15.89:41964) 2023-10-26T06:46:07.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 -- 172.21.15.89:0/514239139 learned_addr learned my addr 172.21.15.89:0/514239139 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbed9c700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cb810cee0 0x7f2cb819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbe59b700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8072c20 0x7f2cb8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 -- 172.21.15.89:0/514239139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cb810cee0 msgr2=0x7f2cb819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cb810cee0 0x7f2cb819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 -- 172.21.15.89:0/514239139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8072c20 msgr2=0x7f2cb8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8072c20 0x7f2cb8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.371+0000 7f2cbdd9a700 1 -- 172.21.15.89:0/514239139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80095f0 con 0x7f2cb8073540 2023-10-26T06:46:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.372+0000 7f2cbdd9a700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb8199310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8005910 tx=0x7f2ca8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.372+0000 7f2caf7fe700 1 -- 172.21.15.89:0/514239139 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca801d070 con 0x7f2cb8073540 2023-10-26T06:46:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.372+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb819e250 con 0x7f2cb8073540 2023-10-26T06:46:07.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.372+0000 7f2caf7fe700 1 -- 172.21.15.89:0/514239139 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca8022470 con 0x7f2cb8073540 2023-10-26T06:46:07.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.372+0000 7f2caf7fe700 1 -- 172.21.15.89:0/514239139 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8017650 con 0x7f2cb8073540 2023-10-26T06:46:07.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.372+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb81a5af0 con 0x7f2cb8073540 2023-10-26T06:46:07.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.373+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb81080f0 con 0x7f2cb8073540 2023-10-26T06:46:07.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.375+0000 7f2caf7fe700 1 -- 172.21.15.89:0/514239139 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f2ca80225e0 con 0x7f2cb8073540 2023-10-26T06:46:07.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.376+0000 7f2caf7fe700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2ca40a1560 0x7f2ca40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.376+0000 7f2caf7fe700 1 -- 172.21.15.89:0/514239139 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2ca80ce1e0 con 0x7f2cb8073540 2023-10-26T06:46:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.376+0000 7f2cbe59b700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2ca40a1560 0x7f2ca40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.377+0000 7f2cbe59b700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2ca40a1560 0x7f2ca40a3a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009c60 tx=0x7f2cb4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:07.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.380+0000 7f2caf7fe700 1 -- 172.21.15.89:0/514239139 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ca80977b0 con 0x7f2cb8073540 2023-10-26T06:46:07.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:07 smithi089 ceph-mon[95031]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:07.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:07 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/949494026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:07 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1490696988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:07 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/106699657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.626+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cb819a080 con 0x7f2cb8073540 2023-10-26T06:46:07.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.627+0000 7f2caf7fe700 1 -- 172.21.15.89:0/514239139 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f2ca8027090 con 0x7f2cb8073540 2023-10-26T06:46:07.629 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:07.629 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:46:07.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2ca40a1560 msgr2=0x7f2ca40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2ca40a1560 0x7f2ca40a3a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009c60 tx=0x7f2cb4009260 comp rx=0 tx=0).stop 2023-10-26T06:46:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 msgr2=0x7f2cb8199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb8199310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8005910 tx=0x7f2ca8004730 comp rx=0 tx=0).stop 2023-10-26T06:46:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 shutdown_connections 2023-10-26T06:46:07.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2ca40a1560 0x7f2ca40a3a20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cb810cee0 0x7f2cb819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8072c20 0x7f2cb8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 --2- 172.21.15.89:0/514239139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb8073540 0x7f2cb8199310 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 >> 172.21.15.89:0/514239139 conn(0x7f2cb80fc170 msgr2=0x7f2cb80fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:07.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.630+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 shutdown_connections 2023-10-26T06:46:07.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.631+0000 7f2cc07ff700 1 -- 172.21.15.89:0/514239139 wait complete. 2023-10-26T06:46:07.634 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:07 smithi192 ceph-mon[131661]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:07 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/949494026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:07 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1490696988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:07 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/106699657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:07 smithi135 ceph-mon[131403]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:07 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/949494026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:07 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1490696988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:07 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/106699657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:07.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 -- 172.21.15.89:0/2253579075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 msgr2=0x7f5a98067870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2253579075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 0x7f5a98067870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a840098f0 tx=0x7f5a84009c00 comp rx=0 tx=0).stop 2023-10-26T06:46:07.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 -- 172.21.15.89:0/2253579075 shutdown_connections 2023-10-26T06:46:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2253579075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a98067fa0 0x7f5a9810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2253579075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a981066c0 0x7f5a98106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2253579075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 0x7f5a98067870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 -- 172.21.15.89:0/2253579075 >> 172.21.15.89:0/2253579075 conn(0x7f5a98076400 msgr2=0x7f5a98076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:07.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 -- 172.21.15.89:0/2253579075 shutdown_connections 2023-10-26T06:46:07.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.851+0000 7f5a9edff700 1 -- 172.21.15.89:0/2253579075 wait complete. 2023-10-26T06:46:07.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 Processor -- start 2023-10-26T06:46:07.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 -- start start 2023-10-26T06:46:07.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a98067fa0 0x7f5a98198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a981066c0 0x7f5a98199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 0x7f5a9819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9819dd20 con 0x7f5a981066c0 2023-10-26T06:46:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9819de90 con 0x7f5a98106fe0 2023-10-26T06:46:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.852+0000 7f5a9edff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9819e050 con 0x7f5a98067fa0 2023-10-26T06:46:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a981066c0 0x7f5a98199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a981066c0 0x7f5a98199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41992/0 (socket says 172.21.15.89:41992) 2023-10-26T06:46:07.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 -- 172.21.15.89:0/2895347080 learned_addr learned my addr 172.21.15.89:0/2895347080 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a9d39c700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 0x7f5a9819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a9cb9b700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a98067fa0 0x7f5a98198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 -- 172.21.15.89:0/2895347080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a98067fa0 msgr2=0x7f5a98198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a98067fa0 0x7f5a98198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 -- 172.21.15.89:0/2895347080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 msgr2=0x7f5a9819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 0x7f5a9819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 -- 172.21.15.89:0/2895347080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84009520 con 0x7f5a981066c0 2023-10-26T06:46:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.853+0000 7f5a97fff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a981066c0 0x7f5a98199390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009230 tx=0x7f5a8400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.854+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2895347080 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8401c070 con 0x7f5a981066c0 2023-10-26T06:46:07.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.854+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9819e2d0 con 0x7f5a981066c0 2023-10-26T06:46:07.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.854+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2895347080 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a8400bb20 con 0x7f5a981066c0 2023-10-26T06:46:07.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.854+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2895347080 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a840038e0 con 0x7f5a981066c0 2023-10-26T06:46:07.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.854+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a981a5b70 con 0x7f5a981066c0 2023-10-26T06:46:07.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.855+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a98101de0 con 0x7f5a981066c0 2023-10-26T06:46:07.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.856+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2895347080 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f5a84020070 con 0x7f5a981066c0 2023-10-26T06:46:07.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.858+0000 7f5a95ffb700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5a880a1560 0x7f5a880a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:07.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.858+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2895347080 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f5a840cd970 con 0x7f5a981066c0 2023-10-26T06:46:07.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.858+0000 7f5a9cb9b700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5a880a1560 0x7f5a880a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:07.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.859+0000 7f5a9cb9b700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5a880a1560 0x7f5a880a3a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005fd0 tx=0x7f5a8c009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:07.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:07.862+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2895347080 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5a84096dc0 con 0x7f5a981066c0 2023-10-26T06:46:08.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.079+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a9819a160 con 0x7f5a981066c0 2023-10-26T06:46:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.080+0000 7f5a95ffb700 1 -- 172.21.15.89:0/2895347080 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7f5a8409a3e0 con 0x7f5a981066c0 2023-10-26T06:46:08.082 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:08.082 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5a880a1560 msgr2=0x7f5a880a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5a880a1560 0x7f5a880a3a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005fd0 tx=0x7f5a8c009310 comp rx=0 tx=0).stop 2023-10-26T06:46:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a981066c0 msgr2=0x7f5a98199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a981066c0 0x7f5a98199390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009230 tx=0x7f5a8400fa00 comp rx=0 tx=0).stop 2023-10-26T06:46:08.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 shutdown_connections 2023-10-26T06:46:08.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5a880a1560 0x7f5a880a3a20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a98106fe0 0x7f5a9819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a98067fa0 0x7f5a98198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 --2- 172.21.15.89:0/2895347080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a981066c0 0x7f5a98199390 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 >> 172.21.15.89:0/2895347080 conn(0x7f5a98076400 msgr2=0x7f5a9810b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:08.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 shutdown_connections 2023-10-26T06:46:08.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.083+0000 7f5a9edff700 1 -- 172.21.15.89:0/2895347080 wait complete. 2023-10-26T06:46:08.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:46:08.107 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:46:08.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.315+0000 7f6a06bff700 1 -- 172.21.15.89:0/810178352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 msgr2=0x7f6a00100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.315+0000 7f6a06bff700 1 --2- 172.21.15.89:0/810178352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 0x7f6a00100b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009990 tx=0x7f69f0009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.315+0000 7f6a06bff700 1 -- 172.21.15.89:0/810178352 shutdown_connections 2023-10-26T06:46:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.315+0000 7f6a06bff700 1 --2- 172.21.15.89:0/810178352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a001010d0 0x7f6a0010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.315+0000 7f6a06bff700 1 --2- 172.21.15.89:0/810178352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a00106720 0x7f6a00106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.315+0000 7f6a06bff700 1 --2- 172.21.15.89:0/810178352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 0x7f6a00100b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.315+0000 7f6a06bff700 1 -- 172.21.15.89:0/810178352 >> 172.21.15.89:0/810178352 conn(0x7f6a000fc200 msgr2=0x7f6a000fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.316+0000 7f6a06bff700 1 -- 172.21.15.89:0/810178352 shutdown_connections 2023-10-26T06:46:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.316+0000 7f6a06bff700 1 -- 172.21.15.89:0/810178352 wait complete. 2023-10-26T06:46:08.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 Processor -- start 2023-10-26T06:46:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 -- start start 2023-10-26T06:46:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 0x7f6a00072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a001010d0 0x7f6a00073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a00106720 0x7f6a00073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001a4fd0 con 0x7f6a00100710 2023-10-26T06:46:08.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001a5140 con 0x7f6a001010d0 2023-10-26T06:46:08.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.317+0000 7f6a06bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001a5300 con 0x7f6a00106720 2023-10-26T06:46:08.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 0x7f6a00072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f69f7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a001010d0 0x7f6a00073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f69f7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a001010d0 0x7f6a00073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:48158/0 (socket says 172.21.15.89:48158) 2023-10-26T06:46:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 0x7f6a00072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42006/0 (socket says 172.21.15.89:42006) 2023-10-26T06:46:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0519c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a00106720 0x7f6a00073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f69f7fff700 1 -- 172.21.15.89:0/305689628 learned_addr learned my addr 172.21.15.89:0/305689628 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:08.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0519c700 1 -- 172.21.15.89:0/305689628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a001010d0 msgr2=0x7f6a00073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0519c700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a001010d0 0x7f6a00073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0519c700 1 -- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 msgr2=0x7f6a00072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0519c700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 0x7f6a00072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0519c700 1 -- 172.21.15.89:0/305689628 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00095f0 con 0x7f6a00106720 2023-10-26T06:46:08.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f6a0519c700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a00106720 0x7f6a00073570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f69f800efc0 tx=0x7f69f800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:08.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.318+0000 7f69f5ffb700 1 -- 172.21.15.89:0/305689628 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8018070 con 0x7f6a00106720 2023-10-26T06:46:08.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.319+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a001a5580 con 0x7f6a00106720 2023-10-26T06:46:08.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.319+0000 7f69f5ffb700 1 -- 172.21.15.89:0/305689628 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69f800f040 con 0x7f6a00106720 2023-10-26T06:46:08.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.319+0000 7f69f5ffb700 1 -- 172.21.15.89:0/305689628 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800cd40 con 0x7f6a00106720 2023-10-26T06:46:08.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.319+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a001a5a40 con 0x7f6a00106720 2023-10-26T06:46:08.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.320+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a00108d30 con 0x7f6a00106720 2023-10-26T06:46:08.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.322+0000 7f69f5ffb700 1 -- 172.21.15.89:0/305689628 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f69f8003a00 con 0x7f6a00106720 2023-10-26T06:46:08.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.323+0000 7f69f5ffb700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69e80a1560 0x7f69e80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.323+0000 7f69f5ffb700 1 -- 172.21.15.89:0/305689628 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f69f80ccc10 con 0x7f6a00106720 2023-10-26T06:46:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.323+0000 7f6a0499b700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69e80a1560 0x7f69e80a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.324+0000 7f6a0499b700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69e80a1560 0x7f69e80a3a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009be0 tx=0x7f69fc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:08.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.327+0000 7f69f5ffb700 1 -- 172.21.15.89:0/305689628 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f69f8096060 con 0x7f6a00106720 2023-10-26T06:46:08.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.534+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a00073f80 con 0x7f6a00106720 2023-10-26T06:46:08.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.535+0000 7f69f5ffb700 1 -- 172.21.15.89:0/305689628 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69f8099680 con 0x7f6a00106720 2023-10-26T06:46:08.536 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:46:08.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69e80a1560 msgr2=0x7f69e80a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69e80a1560 0x7f69e80a3a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009be0 tx=0x7f69fc009260 comp rx=0 tx=0).stop 2023-10-26T06:46:08.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a00106720 msgr2=0x7f6a00073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a00106720 0x7f6a00073570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f69f800efc0 tx=0x7f69f800c3a0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 shutdown_connections 2023-10-26T06:46:08.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f69e80a1560 0x7f69e80a3a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a00100710 0x7f6a00072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a00106720 0x7f6a00073570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 --2- 172.21.15.89:0/305689628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a001010d0 0x7f6a00073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 >> 172.21.15.89:0/305689628 conn(0x7f6a000fc200 msgr2=0x7f6a000fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:08.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 shutdown_connections 2023-10-26T06:46:08.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:08 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/514239139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:08 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:08 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2895347080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.538+0000 7f6a06bff700 1 -- 172.21.15.89:0/305689628 wait complete. 2023-10-26T06:46:08.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:46:08.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:08 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/514239139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:08 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:08 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2895347080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:08 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/514239139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:08 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:08 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2895347080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:08.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 -- 172.21.15.89:0/1989029899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8106720 msgr2=0x7f36d8106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 --2- 172.21.15.89:0/1989029899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8106720 0x7f36d8106b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009990 tx=0x7f36d4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 -- 172.21.15.89:0/1989029899 shutdown_connections 2023-10-26T06:46:08.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 --2- 172.21.15.89:0/1989029899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8106720 0x7f36d8106b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 --2- 172.21.15.89:0/1989029899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d81010d0 0x7f36d810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 --2- 172.21.15.89:0/1989029899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d8100710 0x7f36d8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 -- 172.21.15.89:0/1989029899 >> 172.21.15.89:0/1989029899 conn(0x7f36d80fc200 msgr2=0x7f36d80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:08.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 -- 172.21.15.89:0/1989029899 shutdown_connections 2023-10-26T06:46:08.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.775+0000 7f36df3ff700 1 -- 172.21.15.89:0/1989029899 wait complete. 2023-10-26T06:46:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.776+0000 7f36df3ff700 1 Processor -- start 2023-10-26T06:46:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.776+0000 7f36df3ff700 1 -- start start 2023-10-26T06:46:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36df3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8100710 0x7f36d819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36df3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d81010d0 0x7f36d819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36df3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d8106720 0x7f36d81969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36df3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8197020 con 0x7f36d8100710 2023-10-26T06:46:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36df3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8197190 con 0x7f36d8106720 2023-10-26T06:46:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36df3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8197350 con 0x7f36d81010d0 2023-10-26T06:46:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8100710 0x7f36d819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8100710 0x7f36d819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42034/0 (socket says 172.21.15.89:42034) 2023-10-26T06:46:08.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 -- 172.21.15.89:0/3390808981 learned_addr learned my addr 172.21.15.89:0/3390808981 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:08.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd99c700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d8106720 0x7f36d81969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dc99a700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d81010d0 0x7f36d819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 -- 172.21.15.89:0/3390808981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d81010d0 msgr2=0x7f36d819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d81010d0 0x7f36d819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 -- 172.21.15.89:0/3390808981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d8106720 msgr2=0x7f36d81969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:08.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d8106720 0x7f36d81969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:08.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.777+0000 7f36dd19b700 1 -- 172.21.15.89:0/3390808981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40095f0 con 0x7f36d8100710 2023-10-26T06:46:08.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.778+0000 7f36dd19b700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8100710 0x7f36d819c890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005910 tx=0x7f36d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:08.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.778+0000 7f36ce7fc700 1 -- 172.21.15.89:0/3390808981 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d401d070 con 0x7f36d8100710 2023-10-26T06:46:08.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.778+0000 7f36ce7fc700 1 -- 172.21.15.89:0/3390808981 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d4022470 con 0x7f36d8100710 2023-10-26T06:46:08.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.778+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d81975d0 con 0x7f36d8100710 2023-10-26T06:46:08.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.778+0000 7f36ce7fc700 1 -- 172.21.15.89:0/3390808981 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4017650 con 0x7f36d8100710 2023-10-26T06:46:08.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.778+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d81a3880 con 0x7f36d8100710 2023-10-26T06:46:08.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.779+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d8108d10 con 0x7f36d8100710 2023-10-26T06:46:08.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.781+0000 7f36ce7fc700 1 -- 172.21.15.89:0/3390808981 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f36d40225e0 con 0x7f36d8100710 2023-10-26T06:46:08.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.782+0000 7f36ce7fc700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36c40a15e0 0x7f36c40a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:08.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.782+0000 7f36ce7fc700 1 -- 172.21.15.89:0/3390808981 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f36d40ce190 con 0x7f36d8100710 2023-10-26T06:46:08.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.782+0000 7f36dc99a700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36c40a15e0 0x7f36c40a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:08.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.783+0000 7f36dc99a700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36c40a15e0 0x7f36c40a3aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36d8197b80 tx=0x7f36c8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:08.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:08.788+0000 7f36ce7fc700 1 -- 172.21.15.89:0/3390808981 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f36d4097760 con 0x7f36d8100710 2023-10-26T06:46:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.015+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36d8197e80 con 0x7f36d8100710 2023-10-26T06:46:09.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.016+0000 7f36ce7fc700 1 -- 172.21.15.89:0/3390808981 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36d4027020 con 0x7f36d8100710 2023-10-26T06:46:09.018 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:46:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36c40a15e0 msgr2=0x7f36c40a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36c40a15e0 0x7f36c40a3aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36d8197b80 tx=0x7f36c8009260 comp rx=0 tx=0).stop 2023-10-26T06:46:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8100710 msgr2=0x7f36d819c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8100710 0x7f36d819c890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005910 tx=0x7f36d4004730 comp rx=0 tx=0).stop 2023-10-26T06:46:09.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 shutdown_connections 2023-10-26T06:46:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36c40a15e0 0x7f36c40a3aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36d8100710 0x7f36d819c890 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36d8106720 0x7f36d81969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 --2- 172.21.15.89:0/3390808981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d81010d0 0x7f36d819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.019+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 >> 172.21.15.89:0/3390808981 conn(0x7f36d80fc200 msgr2=0x7f36d80fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.020+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 shutdown_connections 2023-10-26T06:46:09.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.020+0000 7f36df3ff700 1 -- 172.21.15.89:0/3390808981 wait complete. 2023-10-26T06:46:09.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:46:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.251+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2626512566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc74073540 msgr2=0x7efc7410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.251+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2626512566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc74073540 0x7efc7410ca10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc68009990 tx=0x7efc68009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.251+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2626512566 shutdown_connections 2023-10-26T06:46:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.251+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2626512566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc7410cf50 0x7efc7410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.251+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2626512566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc74072c20 0x7efc74073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.251+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2626512566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc74073540 0x7efc7410ca10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.251+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2626512566 >> 172.21.15.89:0/2626512566 conn(0x7efc740fc200 msgr2=0x7efc740fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.252+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2626512566 shutdown_connections 2023-10-26T06:46:09.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.252+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2626512566 wait complete. 2023-10-26T06:46:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.253+0000 7efc7a5ff700 1 Processor -- start 2023-10-26T06:46:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.253+0000 7efc7a5ff700 1 -- start start 2023-10-26T06:46:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc7a5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74072c20 0x7efc74198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc7a5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc74073540 0x7efc74199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc7a5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc7410cf50 0x7efc7419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc7a5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7419dd50 con 0x7efc7410cf50 2023-10-26T06:46:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc7a5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7419dec0 con 0x7efc74072c20 2023-10-26T06:46:09.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc7a5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7419e080 con 0x7efc74073540 2023-10-26T06:46:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc78b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc7410cf50 0x7efc7419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc78b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc7410cf50 0x7efc7419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42056/0 (socket says 172.21.15.89:42056) 2023-10-26T06:46:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc78b9c700 1 -- 172.21.15.89:0/2942213521 learned_addr learned my addr 172.21.15.89:0/2942213521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc73fff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74072c20 0x7efc74198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.254+0000 7efc737fe700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc74073540 0x7efc74199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc78b9c700 1 -- 172.21.15.89:0/2942213521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc74073540 msgr2=0x7efc74199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc78b9c700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc74073540 0x7efc74199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc78b9c700 1 -- 172.21.15.89:0/2942213521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74072c20 msgr2=0x7efc74198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc78b9c700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74072c20 0x7efc74198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc78b9c700 1 -- 172.21.15.89:0/2942213521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680095f0 con 0x7efc7410cf50 2023-10-26T06:46:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc73fff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74072c20 0x7efc74198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc78b9c700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc7410cf50 0x7efc7419d710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00ef30 tx=0x7efc6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc717fa700 1 -- 172.21.15.89:0/2942213521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c018070 con 0x7efc7410cf50 2023-10-26T06:46:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc717fa700 1 -- 172.21.15.89:0/2942213521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6c00f040 con 0x7efc7410cf50 2023-10-26T06:46:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7419e300 con 0x7efc7410cf50 2023-10-26T06:46:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc717fa700 1 -- 172.21.15.89:0/2942213521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00cd00 con 0x7efc7410cf50 2023-10-26T06:46:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.255+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc741a5ba0 con 0x7efc7410cf50 2023-10-26T06:46:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.256+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc74108160 con 0x7efc7410cf50 2023-10-26T06:46:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.260+0000 7efc717fa700 1 -- 172.21.15.89:0/2942213521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7efc6c003a00 con 0x7efc7410cf50 2023-10-26T06:46:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.262+0000 7efc717fa700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efc5c0a1560 0x7efc5c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.262+0000 7efc717fa700 1 -- 172.21.15.89:0/2942213521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7efc6c0cd470 con 0x7efc7410cf50 2023-10-26T06:46:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.262+0000 7efc73fff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efc5c0a1560 0x7efc5c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.263+0000 7efc717fa700 1 -- 172.21.15.89:0/2942213521 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efc6c096920 con 0x7efc7410cf50 2023-10-26T06:46:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.263+0000 7efc73fff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efc5c0a1560 0x7efc5c0a3a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc64011490 tx=0x7efc64009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:09.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:09 smithi089 ceph-mon[95031]: pgmap v287: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:09.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:09 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/305689628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:09.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:09 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3390808981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:09.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.476+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc7419a130 con 0x7efc7410cf50 2023-10-26T06:46:09.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.477+0000 7efc717fa700 1 -- 172.21.15.89:0/2942213521 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc6c099f40 con 0x7efc7410cf50 2023-10-26T06:46:09.478 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:46:09.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efc5c0a1560 msgr2=0x7efc5c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efc5c0a1560 0x7efc5c0a3a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc64011490 tx=0x7efc64009260 comp rx=0 tx=0).stop 2023-10-26T06:46:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc7410cf50 msgr2=0x7efc7419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc7410cf50 0x7efc7419d710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00ef30 tx=0x7efc6c00c3a0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 shutdown_connections 2023-10-26T06:46:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efc5c0a1560 0x7efc5c0a3a20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc7410cf50 0x7efc7419d710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74072c20 0x7efc74198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 --2- 172.21.15.89:0/2942213521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc74073540 0x7efc74199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 >> 172.21.15.89:0/2942213521 conn(0x7efc740fc200 msgr2=0x7efc740fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 shutdown_connections 2023-10-26T06:46:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.479+0000 7efc7a5ff700 1 -- 172.21.15.89:0/2942213521 wait complete. 2023-10-26T06:46:09.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:46:09.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:09 smithi192 ceph-mon[131661]: pgmap v287: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:09 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/305689628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:09 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3390808981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:09 smithi135 ceph-mon[131403]: pgmap v287: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:09 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/305689628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:09 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3390808981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:09.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.719+0000 7f96e03ff700 1 -- 172.21.15.89:0/2814662486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81006a0 msgr2=0x7f96d8100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.719+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2814662486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81006a0 0x7f96d8100b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009990 tx=0x7f96c8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.719+0000 7f96e03ff700 1 -- 172.21.15.89:0/2814662486 shutdown_connections 2023-10-26T06:46:09.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.719+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2814662486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96d8101060 0x7f96d810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.719+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2814662486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d81066b0 0x7f96d8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.719+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2814662486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81006a0 0x7f96d8100b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.719+0000 7f96e03ff700 1 -- 172.21.15.89:0/2814662486 >> 172.21.15.89:0/2814662486 conn(0x7f96d80fc170 msgr2=0x7f96d80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:09.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.720+0000 7f96e03ff700 1 -- 172.21.15.89:0/2814662486 shutdown_connections 2023-10-26T06:46:09.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.720+0000 7f96e03ff700 1 -- 172.21.15.89:0/2814662486 wait complete. 2023-10-26T06:46:09.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.721+0000 7f96e03ff700 1 Processor -- start 2023-10-26T06:46:09.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.721+0000 7f96e03ff700 1 -- start start 2023-10-26T06:46:09.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96e03ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96d81006a0 0x7f96d8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96e03ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d8101060 0x7f96d8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96e03ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81066b0 0x7f96d8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96e03ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d81a4fa0 con 0x7f96d81006a0 2023-10-26T06:46:09.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96e03ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d81a5110 con 0x7f96d81066b0 2023-10-26T06:46:09.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96e03ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d81a52d0 con 0x7f96d8101060 2023-10-26T06:46:09.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96de19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96d81006a0 0x7f96d8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d8101060 0x7f96d8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d8101060 0x7f96d8073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44326/0 (socket says 172.21.15.89:44326) 2023-10-26T06:46:09.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 -- 172.21.15.89:0/2872231970 learned_addr learned my addr 172.21.15.89:0/2872231970 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:09.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96de99c700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81066b0 0x7f96d8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 -- 172.21.15.89:0/2872231970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81066b0 msgr2=0x7f96d8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81066b0 0x7f96d8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 -- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96d81006a0 msgr2=0x7f96d8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:09.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96d81006a0 0x7f96d8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:09.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.722+0000 7f96dd99a700 1 -- 172.21.15.89:0/2872231970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c80095f0 con 0x7f96d8101060 2023-10-26T06:46:09.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.723+0000 7f96de19b700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96d81006a0 0x7f96d8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:09.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.723+0000 7f96dd99a700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d8101060 0x7f96d8073030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96c8000c00 tx=0x7f96c8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:09.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.723+0000 7f96cf7fe700 1 -- 172.21.15.89:0/2872231970 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c801d070 con 0x7f96d8101060 2023-10-26T06:46:09.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.723+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d81a5550 con 0x7f96d8101060 2023-10-26T06:46:09.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.723+0000 7f96cf7fe700 1 -- 172.21.15.89:0/2872231970 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96c8022470 con 0x7f96d8101060 2023-10-26T06:46:09.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.723+0000 7f96cf7fe700 1 -- 172.21.15.89:0/2872231970 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8017650 con 0x7f96d8101060 2023-10-26T06:46:09.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.723+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d81a5a30 con 0x7f96d8101060 2023-10-26T06:46:09.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.724+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8108cc0 con 0x7f96d8101060 2023-10-26T06:46:09.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.727+0000 7f96cf7fe700 1 -- 172.21.15.89:0/2872231970 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f96c80225e0 con 0x7f96d8101060 2023-10-26T06:46:09.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.729+0000 7f96cf7fe700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96c40a1560 0x7f96c40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:09.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.729+0000 7f96cf7fe700 1 -- 172.21.15.89:0/2872231970 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f96c80ce300 con 0x7f96d8101060 2023-10-26T06:46:09.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.729+0000 7f96de19b700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96c40a1560 0x7f96c40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:09.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.730+0000 7f96de19b700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96c40a1560 0x7f96c40a3a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009fd0 tx=0x7f96d4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:09.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.732+0000 7f96cf7fe700 1 -- 172.21.15.89:0/2872231970 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f96c8097750 con 0x7f96d8101060 2023-10-26T06:46:09.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.994+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96d8073fd0 con 0x7f96d8101060 2023-10-26T06:46:09.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.996+0000 7f96cf7fe700 1 -- 172.21.15.89:0/2872231970 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f96c809ad70 con 0x7f96d8101060 2023-10-26T06:46:09.997 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:09.997 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:46:10.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.998+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96c40a1560 msgr2=0x7f96c40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.998+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96c40a1560 0x7f96c40a3a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009fd0 tx=0x7f96d4009260 comp rx=0 tx=0).stop 2023-10-26T06:46:10.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.998+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d8101060 msgr2=0x7f96d8073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.998+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d8101060 0x7f96d8073030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96c8000c00 tx=0x7f96c8005740 comp rx=0 tx=0).stop 2023-10-26T06:46:10.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 shutdown_connections 2023-10-26T06:46:10.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96c40a1560 0x7f96c40a3a20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96d81006a0 0x7f96d8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96d81066b0 0x7f96d8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 --2- 172.21.15.89:0/2872231970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d8101060 0x7f96d8073030 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 >> 172.21.15.89:0/2872231970 conn(0x7f96d80fc170 msgr2=0x7f96d80fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:10.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 shutdown_connections 2023-10-26T06:46:10.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:09.999+0000 7f96e03ff700 1 -- 172.21.15.89:0/2872231970 wait complete. 2023-10-26T06:46:10.002 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:10.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:10.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 -- 172.21.15.89:0/2772445142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 msgr2=0x7fd5d0106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/2772445142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0106a90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000b0a0 tx=0x7fd5c000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 -- 172.21.15.89:0/2772445142 shutdown_connections 2023-10-26T06:46:10.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/2772445142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d0101060 0x7fd5d010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/2772445142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0106a90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/2772445142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5d01006a0 0x7fd5d0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 -- 172.21.15.89:0/2772445142 >> 172.21.15.89:0/2772445142 conn(0x7fd5d00fc170 msgr2=0x7fd5d00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:10.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 -- 172.21.15.89:0/2772445142 shutdown_connections 2023-10-26T06:46:10.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.234+0000 7fd5d4dff700 1 -- 172.21.15.89:0/2772445142 wait complete. 2023-10-26T06:46:10.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.235+0000 7fd5d4dff700 1 Processor -- start 2023-10-26T06:46:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.235+0000 7fd5d4dff700 1 -- start start 2023-10-26T06:46:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5d4dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d01006a0 0x7fd5d0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5d4dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5d0101060 0x7fd5d0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5d4dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5d4dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d01a4ea0 con 0x7fd5d01006a0 2023-10-26T06:46:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5d4dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d01a5010 con 0x7fd5d01066b0 2023-10-26T06:46:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5d4dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d01a51d0 con 0x7fd5d0101060 2023-10-26T06:46:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ce59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d01006a0 0x7fd5d0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:56260/0 (socket says 172.21.15.89:56260) 2023-10-26T06:46:10.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ce59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d01006a0 0x7fd5d0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41002/0 (socket says 172.21.15.89:41002) 2023-10-26T06:46:10.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 -- 172.21.15.89:0/1073156412 learned_addr learned my addr 172.21.15.89:0/1073156412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:10.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5c7fff700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5d0101060 0x7fd5d0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 -- 172.21.15.89:0/1073156412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5d0101060 msgr2=0x7fd5d0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5d0101060 0x7fd5d0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 -- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d01006a0 msgr2=0x7fd5d0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d01006a0 0x7fd5d0072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ced9d700 1 -- 172.21.15.89:0/1073156412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0009d00 con 0x7fd5d01066b0 2023-10-26T06:46:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.236+0000 7fd5ce59c700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d01006a0 0x7fd5d0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:10.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.237+0000 7fd5ced9d700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0073570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e940 tx=0x7fd5c800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.237+0000 7fd5c77fe700 1 -- 172.21.15.89:0/1073156412 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80097b0 con 0x7fd5d01066b0 2023-10-26T06:46:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.237+0000 7fd5c77fe700 1 -- 172.21.15.89:0/1073156412 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5c8005020 con 0x7fd5d01066b0 2023-10-26T06:46:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.237+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d01a54b0 con 0x7fd5d01066b0 2023-10-26T06:46:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.237+0000 7fd5c77fe700 1 -- 172.21.15.89:0/1073156412 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800cc40 con 0x7fd5d01066b0 2023-10-26T06:46:10.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.237+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d01a5970 con 0x7fd5d01066b0 2023-10-26T06:46:10.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.238+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d0108ca0 con 0x7fd5d01066b0 2023-10-26T06:46:10.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.240+0000 7fd5c77fe700 1 -- 172.21.15.89:0/1073156412 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fd5c8005400 con 0x7fd5d01066b0 2023-10-26T06:46:10.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.241+0000 7fd5c77fe700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a1560 0x7fd5bc0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.241+0000 7fd5c77fe700 1 -- 172.21.15.89:0/1073156412 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd5c8014070 con 0x7fd5d01066b0 2023-10-26T06:46:10.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.241+0000 7fd5ce59c700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a1560 0x7fd5bc0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.242+0000 7fd5ce59c700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a1560 0x7fd5bc0a3a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000b780 tx=0x7fd5c0006210 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:10.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.243+0000 7fd5c77fe700 1 -- 172.21.15.89:0/1073156412 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5c80967b0 con 0x7fd5d01066b0 2023-10-26T06:46:10.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.459+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5d0101de0 con 0x7fd5d01066b0 2023-10-26T06:46:10.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.461+0000 7fd5c77fe700 1 -- 172.21.15.89:0/1073156412 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7fd5c8005190 con 0x7fd5d01066b0 2023-10-26T06:46:10.462 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:10.463 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:10.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a1560 msgr2=0x7fd5bc0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a1560 0x7fd5bc0a3a20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000b780 tx=0x7fd5c0006210 comp rx=0 tx=0).stop 2023-10-26T06:46:10.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 msgr2=0x7fd5d0073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0073570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e940 tx=0x7fd5c800ec50 comp rx=0 tx=0).stop 2023-10-26T06:46:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 shutdown_connections 2023-10-26T06:46:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a1560 0x7fd5bc0a3a20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5d01006a0 0x7fd5d0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5d01066b0 0x7fd5d0073570 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 --2- 172.21.15.89:0/1073156412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5d0101060 0x7fd5d0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 >> 172.21.15.89:0/1073156412 conn(0x7fd5d00fc170 msgr2=0x7fd5d00fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.463+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 shutdown_connections 2023-10-26T06:46:10.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.464+0000 7fd5d4dff700 1 -- 172.21.15.89:0/1073156412 wait complete. 2023-10-26T06:46:10.484 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:46:10.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:10 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2942213521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:10.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:10 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2872231970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:10.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:10 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2942213521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:10.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:10 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2872231970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 -- 172.21.15.89:0/3508151343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6781066b0 msgr2=0x7fb678106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3508151343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6781066b0 0x7fb678106a90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb6680098f0 tx=0x7fb668009c00 comp rx=0 tx=0).stop 2023-10-26T06:46:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 -- 172.21.15.89:0/3508151343 shutdown_connections 2023-10-26T06:46:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3508151343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678101060 0x7fb67810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3508151343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6781066b0 0x7fb678106a90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3508151343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 0x7fb678100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 -- 172.21.15.89:0/3508151343 >> 172.21.15.89:0/3508151343 conn(0x7fb6780fc1d0 msgr2=0x7fb6780fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:10.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 -- 172.21.15.89:0/3508151343 shutdown_connections 2023-10-26T06:46:10.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.694+0000 7fb67dbff700 1 -- 172.21.15.89:0/3508151343 wait complete. 2023-10-26T06:46:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.695+0000 7fb67dbff700 1 Processor -- start 2023-10-26T06:46:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.695+0000 7fb67dbff700 1 -- start start 2023-10-26T06:46:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb67dbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 0x7fb678198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb67dbff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb678101060 0x7fb6781992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb67dbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6781066b0 0x7fb67819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb67dbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67819dcd0 con 0x7fb6781006a0 2023-10-26T06:46:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb67dbff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67819de40 con 0x7fb678101060 2023-10-26T06:46:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb67dbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67819e000 con 0x7fb6781066b0 2023-10-26T06:46:10.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb6777fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 0x7fb678198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb6777fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 0x7fb678198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41016/0 (socket says 172.21.15.89:41016) 2023-10-26T06:46:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb6777fe700 1 -- 172.21.15.89:0/3097514816 learned_addr learned my addr 172.21.15.89:0/3097514816 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb677fff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6781066b0 0x7fb67819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.696+0000 7fb66ffff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb678101060 0x7fb6781992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6777fe700 1 -- 172.21.15.89:0/3097514816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6781066b0 msgr2=0x7fb67819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6777fe700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6781066b0 0x7fb67819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6777fe700 1 -- 172.21.15.89:0/3097514816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb678101060 msgr2=0x7fb6781992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6777fe700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb678101060 0x7fb6781992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6777fe700 1 -- 172.21.15.89:0/3097514816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb668009520 con 0x7fb6781006a0 2023-10-26T06:46:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb677fff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6781066b0 0x7fb67819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb66ffff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb678101060 0x7fb6781992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:10.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6777fe700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 0x7fb678198db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb668005950 tx=0x7fb66800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:10.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6757fa700 1 -- 172.21.15.89:0/3097514816 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66801c070 con 0x7fb6781006a0 2023-10-26T06:46:10.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67819e280 con 0x7fb6781006a0 2023-10-26T06:46:10.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6757fa700 1 -- 172.21.15.89:0/3097514816 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb66800bb20 con 0x7fb6781006a0 2023-10-26T06:46:10.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb6757fa700 1 -- 172.21.15.89:0/3097514816 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680038e0 con 0x7fb6781006a0 2023-10-26T06:46:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.697+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6781a5b20 con 0x7fb6781006a0 2023-10-26T06:46:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.699+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb678101de0 con 0x7fb6781006a0 2023-10-26T06:46:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.700+0000 7fb6757fa700 1 -- 172.21.15.89:0/3097514816 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fb668020030 con 0x7fb6781006a0 2023-10-26T06:46:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.702+0000 7fb6757fa700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb6640a1560 0x7fb6640a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.702+0000 7fb6757fa700 1 -- 172.21.15.89:0/3097514816 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb6680cdda0 con 0x7fb6781006a0 2023-10-26T06:46:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.702+0000 7fb66ffff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb6640a1560 0x7fb6640a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.703+0000 7fb66ffff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb6640a1560 0x7fb6640a3a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb67819a300 tx=0x7fb660009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:10.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.706+0000 7fb6757fa700 1 -- 172.21.15.89:0/3097514816 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb6680971f0 con 0x7fb6781006a0 2023-10-26T06:46:10.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:10 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2942213521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:10.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:10 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2872231970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:10.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.971+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb67819a0b0 con 0x7fb6781006a0 2023-10-26T06:46:10.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.972+0000 7fb6757fa700 1 -- 172.21.15.89:0/3097514816 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fb66809a810 con 0x7fb6781006a0 2023-10-26T06:46:10.973 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:10.974 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:46:10.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb6640a1560 msgr2=0x7fb6640a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb6640a1560 0x7fb6640a3a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb67819a300 tx=0x7fb660009310 comp rx=0 tx=0).stop 2023-10-26T06:46:10.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 msgr2=0x7fb678198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:10.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 0x7fb678198db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb668005950 tx=0x7fb66800fa00 comp rx=0 tx=0).stop 2023-10-26T06:46:10.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 shutdown_connections 2023-10-26T06:46:10.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb6640a1560 0x7fb6640a3a20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6781006a0 0x7fb678198db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6781066b0 0x7fb67819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 --2- 172.21.15.89:0/3097514816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb678101060 0x7fb6781992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:10.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 >> 172.21.15.89:0/3097514816 conn(0x7fb6780fc1d0 msgr2=0x7fb6780fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:10.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 shutdown_connections 2023-10-26T06:46:10.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:10.975+0000 7fb67dbff700 1 -- 172.21.15.89:0/3097514816 wait complete. 2023-10-26T06:46:10.979 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:10.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-26T06:45:05.445103+0000', 'modified': '2023-10-26T06:45:09.558391+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24703': {'gid': 24703, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6839/2442523019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 2442523019}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 2442523019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6839/609246385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 609246385}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 609246385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T06:46:10.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T06:46:10.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-26T06:46:11.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/1681731526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f180734b0 msgr2=0x7f9f18111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/1681731526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f180734b0 0x7f9f18111170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009990 tx=0x7f9f0c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/1681731526 shutdown_connections 2023-10-26T06:46:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/1681731526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f18072b90 0x7f9f18072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/1681731526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f181116b0 0x7f9f18113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/1681731526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f180734b0 0x7f9f18111170 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/1681731526 >> 172.21.15.89:0/1681731526 conn(0x7f9f180fc190 msgr2=0x7f9f180fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/1681731526 shutdown_connections 2023-10-26T06:46:11.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.224+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/1681731526 wait complete. 2023-10-26T06:46:11.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.225+0000 7f9f1e9ff700 1 Processor -- start 2023-10-26T06:46:11.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.225+0000 7f9f1e9ff700 1 -- start start 2023-10-26T06:46:11.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f1e9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18072b90 0x7f9f1819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f1e9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f180734b0 0x7f9f1819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f1e9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f181116b0 0x7f9f181a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f1e9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f181a2070 con 0x7f9f180734b0 2023-10-26T06:46:11.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f1e9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f181a21e0 con 0x7f9f181116b0 2023-10-26T06:46:11.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f1e9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f181a23a0 con 0x7f9f18072b90 2023-10-26T06:46:11.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f180734b0 0x7f9f1819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f180734b0 0x7f9f1819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41032/0 (socket says 172.21.15.89:41032) 2023-10-26T06:46:11.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 -- 172.21.15.89:0/557352859 learned_addr learned my addr 172.21.15.89:0/557352859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:11.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f17fff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18072b90 0x7f9f1819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f1cf9c700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f181116b0 0x7f9f181a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 -- 172.21.15.89:0/557352859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18072b90 msgr2=0x7f9f1819d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18072b90 0x7f9f1819d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 -- 172.21.15.89:0/557352859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f181116b0 msgr2=0x7f9f181a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f181116b0 0x7f9f181a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.226+0000 7f9f177fe700 1 -- 172.21.15.89:0/557352859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0095f0 con 0x7f9f180734b0 2023-10-26T06:46:11.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.227+0000 7f9f177fe700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f180734b0 0x7f9f1819d6e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c000c00 tx=0x7f9f0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.227+0000 7f9f157fa700 1 -- 172.21.15.89:0/557352859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c01d070 con 0x7f9f180734b0 2023-10-26T06:46:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.227+0000 7f9f157fa700 1 -- 172.21.15.89:0/557352859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f0c022470 con 0x7f9f180734b0 2023-10-26T06:46:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.227+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f181a2620 con 0x7f9f180734b0 2023-10-26T06:46:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.227+0000 7f9f157fa700 1 -- 172.21.15.89:0/557352859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c017650 con 0x7f9f180734b0 2023-10-26T06:46:11.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.227+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f181a9ec0 con 0x7f9f180734b0 2023-10-26T06:46:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.230+0000 7f9f157fa700 1 -- 172.21.15.89:0/557352859 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f9f0c0225e0 con 0x7f9f180734b0 2023-10-26T06:46:11.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.231+0000 7f9f3231eb80 1 Processor -- start 2023-10-26T06:46:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.231+0000 7f9f157fa700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f000a15e0 0x7f9f000a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f157fa700 1 -- 172.21.15.89:0/557352859 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f9f0c0ce160 con 0x7f9f180734b0 2023-10-26T06:46:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f3231eb80 1 -- start start 2023-10-26T06:46:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734cff8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734cffc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734cffd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cf5a690 con 0x55734cffccb0 2023-10-26T06:46:11.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cf40c10 con 0x55734cff8240 2023-10-26T06:46:11.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.232+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cf3e530 con 0x55734cff8b60 2023-10-26T06:46:11.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.233+0000 7f9f17fff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f000a15e0 0x7f9f000a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734cffc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734cffc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60766/0 (socket says 172.21.15.89:60766) 2023-10-26T06:46:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f1cf9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734cffd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 -- 172.21.15.89:0/3892174667 learned_addr learned my addr 172.21.15.89:0/3892174667 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:11.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 -- 172.21.15.89:0/3892174667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 msgr2=0x55734cff8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 --2- 172.21.15.89:0/3892174667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734cff8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 -- 172.21.15.89:0/3892174667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 msgr2=0x55734cffd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 --2- 172.21.15.89:0/3892174667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734cffd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 -- 172.21.15.89:0/3892174667 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55734cef73a0 con 0x55734cff8b60 2023-10-26T06:46:11.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f177fe700 1 --2- 172.21.15.89:0/3892174667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734cffc770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c097d40 tx=0x7f9f0c09acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce3096b8775280b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9f17fff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f000a15e0 0x7f9f000a3aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009c70 tx=0x7f9f08011720 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9efe7fc700 1 -- 172.21.15.89:0/3892174667 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c09b2d0 con 0x55734cff8b60 2023-10-26T06:46:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9efe7fc700 1 -- 172.21.15.89:0/3892174667 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f0c09f420 con 0x55734cff8b60 2023-10-26T06:46:11.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.234+0000 7f9efe7fc700 1 -- 172.21.15.89:0/3892174667 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c09b790 con 0x55734cff8b60 2023-10-26T06:46:11.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3892174667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 msgr2=0x55734cffc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3892174667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734cffc770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c097d40 tx=0x7f9f0c09acb0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3892174667 shutdown_connections 2023-10-26T06:46:11.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3892174667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734cffd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3892174667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734cff8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3892174667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734cffc770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3892174667 >> 172.21.15.89:0/3892174667 conn(0x55734cf5b6d0 msgr2=0x55734cfe1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:11.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3892174667 shutdown_connections 2023-10-26T06:46:11.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.235+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3892174667 wait complete. 2023-10-26T06:46:11.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.236+0000 7f9f3231eb80 1 -- start start 2023-10-26T06:46:11.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.236+0000 7f9f3231eb80 1 Processor -- start 2023-10-26T06:46:11.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d004380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d001710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d001cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cf39e20 con 0x55734cffccb0 2023-10-26T06:46:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734ced66c0 con 0x55734cff8240 2023-10-26T06:46:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734ced6360 con 0x55734cff8b60 2023-10-26T06:46:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f3231eb80 10 client.0 _ll_register_callbacks cb 0x55734ce20a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f17fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d004380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f1cf9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d001cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f17fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d004380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:56320/0 (socket says 172.21.15.89:56320) 2023-10-26T06:46:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f17fff700 1 -- 172.21.15.89:0/66712004 learned_addr learned my addr 172.21.15.89:0/66712004 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.237+0000 7f9f177fe700 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d001710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f17fff700 1 -- 172.21.15.89:0/66712004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 msgr2=0x55734d001710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f17fff700 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d001710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f17fff700 1 -- 172.21.15.89:0/66712004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 msgr2=0x55734d001cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f17fff700 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d001cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f17fff700 1 -- 172.21.15.89:0/66712004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c0a1040 con 0x55734cff8240 2023-10-26T06:46:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f177fe700 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d001710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f17fff700 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d004380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f080092a0 tx=0x7f9f08004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a58f6dfe40dc27e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:11.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9ef77fe700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:46:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9ef77fe700 1 -- 172.21.15.89:0/66712004 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004a30 con 0x55734cff8240 2023-10-26T06:46:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55734cf3eea0 con 0x55734cff8240 2023-10-26T06:46:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9ef77fe700 1 -- 172.21.15.89:0/66712004 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f9f08004b90 con 0x55734cff8240 2023-10-26T06:46:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9ef77fe700 1 -- 172.21.15.89:0/66712004 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f08004d50 con 0x55734cff8240 2023-10-26T06:46:11.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.238+0000 7f9f3231eb80 10 client.25133 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55734cd64d90 con 0x55734cff8240 2023-10-26T06:46:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9ef77fe700 1 -- 172.21.15.89:0/66712004 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55734cf3eea0 con 0x55734cff8240 2023-10-26T06:46:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9ef77fe700 1 -- 172.21.15.89:0/66712004 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f08026780 con 0x55734cff8240 2023-10-26T06:46:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f3231eb80 10 client.25133 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:11.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f3231eb80 10 client.25133 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T06:46:11.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f3231eb80 20 client.25133 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:11.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d011ae0 0x55734d010600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f3231eb80 4 client.25133 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:46:11.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55734cfb6860 con 0x55734d011ae0 2023-10-26T06:46:11.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.239+0000 7f9f177fe700 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d011ae0 0x55734d010600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.240+0000 7f9ef77fe700 10 client.25133 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.240+0000 7f9f177fe700 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d011ae0 0x55734d010600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665bf591a54021cc server_cookie=36c4942c6ca655d7 in_seq=0 out_seq=0 2023-10-26T06:46:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.240+0000 7f9ef77fe700 1 -- 172.21.15.89:0/66712004 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55734cfb6860 con 0x55734d011ae0 2023-10-26T06:46:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.240+0000 7f9ef77fe700 10 client.25133 handle_command_reply: tid=0 2023-10-26T06:46:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.240+0000 7f9f3231eb80 1 client.25133 shutdown 2023-10-26T06:46:11.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 msgr2=0x55734d004380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d004380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f080092a0 tx=0x7f9f08004750 comp rx=0 tx=0).stop 2023-10-26T06:46:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 shutdown_connections 2023-10-26T06:46:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d001cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d011ae0 0x55734d010600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d004380 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/66712004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d001710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 >> 172.21.15.89:0/66712004 conn(0x55734cf5b6d0 msgr2=0x55734cfe6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9ef77fe700 0 client.25133 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 shutdown_connections 2023-10-26T06:46:11.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 1 -- 172.21.15.89:0/66712004 wait complete. 2023-10-26T06:46:11.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.241+0000 7f9f3231eb80 20 client.25133 trim_cache size 0 max 16384 2023-10-26T06:46:11.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 Processor -- start 2023-10-26T06:46:11.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 -- start start 2023-10-26T06:46:11.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734cfe17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cff8b60 0x55734cffad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cffccb0 0x55734cffb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cdba8d0 con 0x55734cff8b60 2023-10-26T06:46:11.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cfb6890 con 0x55734cff8240 2023-10-26T06:46:11.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.259+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cf3b0b0 con 0x55734cffccb0 2023-10-26T06:46:11.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cff8b60 0x55734cffad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cff8b60 0x55734cffad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41068/0 (socket says 172.21.15.89:41068) 2023-10-26T06:46:11.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 -- 172.21.15.89:0/3170219399 learned_addr learned my addr 172.21.15.89:0/3170219399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f1cf9c700 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cffccb0 0x55734cffb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f17fff700 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734cfe17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 -- 172.21.15.89:0/3170219399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cffccb0 msgr2=0x55734cffb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cffccb0 0x55734cffb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 -- 172.21.15.89:0/3170219399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 msgr2=0x55734cfe17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734cfe17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 -- 172.21.15.89:0/3170219399 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55734cd4dc10 con 0x55734cff8b60 2023-10-26T06:46:11.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9f177fe700 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cff8b60 0x55734cffad90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0cbab0 tx=0x7f9f0c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c0ce5333542cdb server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:11.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9efeffd700 1 -- 172.21.15.89:0/3170219399 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c097610 con 0x55734cff8b60 2023-10-26T06:46:11.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.260+0000 7f9efeffd700 1 -- 172.21.15.89:0/3170219399 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f0c0a35d0 con 0x55734cff8b60 2023-10-26T06:46:11.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9efeffd700 1 -- 172.21.15.89:0/3170219399 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c09b790 con 0x55734cff8b60 2023-10-26T06:46:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3170219399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cff8b60 msgr2=0x55734cffad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cff8b60 0x55734cffad90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0cbab0 tx=0x7f9f0c01a040 comp rx=0 tx=0).stop 2023-10-26T06:46:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3170219399 shutdown_connections 2023-10-26T06:46:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cffccb0 0x55734cffb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734cfe17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/3170219399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cff8b60 0x55734cffad90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3170219399 >> 172.21.15.89:0/3170219399 conn(0x55734cf5b6d0 msgr2=0x55734cfe6840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:11.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3170219399 shutdown_connections 2023-10-26T06:46:11.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.261+0000 7f9f3231eb80 1 -- 172.21.15.89:0/3170219399 wait complete. 2023-10-26T06:46:11.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 -- start start 2023-10-26T06:46:11.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 Processor -- start 2023-10-26T06:46:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d000c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d0011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d0016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cff9660 con 0x55734cffccb0 2023-10-26T06:46:11.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cff97d0 con 0x55734cff8240 2023-10-26T06:46:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734cff9990 con 0x55734cff8b60 2023-10-26T06:46:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.262+0000 7f9f3231eb80 10 client.0 _ll_register_callbacks cb 0x55734ce20a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d0016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d0016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41078/0 (socket says 172.21.15.89:41078) 2023-10-26T06:46:11.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f17fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d000c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 -- 172.21.15.89:0/2455698302 learned_addr learned my addr 172.21.15.89:0/2455698302 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f177fe700 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d0011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 -- 172.21.15.89:0/2455698302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 msgr2=0x55734d0011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d0011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 -- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 msgr2=0x55734d000c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:11.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d000c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 -- 172.21.15.89:0/2455698302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55734cef73a0 con 0x55734cffccb0 2023-10-26T06:46:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f17fff700 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d000c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f1cf9c700 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d0016f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f10017bb0 tx=0x7f9f10017be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c7fba0d9942e8f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9ef77fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:46:11.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9ef77fe700 1 -- 172.21.15.89:0/2455698302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1001c070 con 0x55734cffccb0 2023-10-26T06:46:11.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f0c09f420 con 0x55734cffccb0 2023-10-26T06:46:11.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.263+0000 7f9ef77fe700 1 -- 172.21.15.89:0/2455698302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f9f10007770 con 0x55734cffccb0 2023-10-26T06:46:11.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9ef77fe700 1 -- 172.21.15.89:0/2455698302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f10007930 con 0x55734cffccb0 2023-10-26T06:46:11.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 10 client.15123 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:11.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55734cd8b200 con 0x55734cffccb0 2023-10-26T06:46:11.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9ef77fe700 1 -- 172.21.15.89:0/2455698302 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f0c09f420 con 0x55734cffccb0 2023-10-26T06:46:11.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9ef77fe700 1 -- 172.21.15.89:0/2455698302 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f10021ca0 con 0x55734cffccb0 2023-10-26T06:46:11.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 10 client.15123 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:11.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 10 client.15123 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T06:46:11.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 20 client.15123 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:11.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d06a3a0 0x55734d019e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:11.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 4 client.15123 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T06:46:11.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55734cf32e00 con 0x55734d06a3a0 2023-10-26T06:46:11.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.264+0000 7f9f17fff700 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d06a3a0 0x55734d019e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:11.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f17fff700 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d06a3a0 0x55734d019e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db7eab8b446e49d server_cookie=6618b2b1a6962cc7 in_seq=0 out_seq=0 2023-10-26T06:46:11.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9ef77fe700 10 client.15123 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:11.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9ef77fe700 1 -- 172.21.15.89:0/2455698302 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55734cf32e00 con 0x55734d06a3a0 2023-10-26T06:46:11.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9ef77fe700 10 client.15123 handle_command_reply: tid=0 2023-10-26T06:46:11.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 client.15123 shutdown 2023-10-26T06:46:11.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 msgr2=0x55734d0016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d0016f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f10017bb0 tx=0x7f9f10017be0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 shutdown_connections 2023-10-26T06:46:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55734cffccb0 0x55734d0016f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734d06a3a0 0x55734d019e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55734cff8240 0x55734d000c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 --2- 172.21.15.89:0/2455698302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55734cff8b60 0x55734d0011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9ef77fe700 0 client.15123 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 >> 172.21.15.89:0/2455698302 conn(0x55734cf5b6d0 msgr2=0x55734cfe17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 shutdown_connections 2023-10-26T06:46:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.265+0000 7f9f3231eb80 1 -- 172.21.15.89:0/2455698302 wait complete. 2023-10-26T06:46:11.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f3231eb80 20 client.15123 trim_cache size 0 max 16384 2023-10-26T06:46:11.281 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:46:11.281 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "e22b2d1a-73c9-11ee-8db9-212e2dc638e7", 2023-10-26T06:46:11.281 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-26T06:46:11.281 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24743, 2023-10-26T06:46:11.282 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-26T06:46:11.282 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-26T06:46:11.282 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-26T06:46:11.282 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 65.212864706999994, 2023-10-26T06:46:11.282 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 20, 2023-10-26T06:46:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 86, 2023-10-26T06:46:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 84, 2023-10-26T06:46:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 72.325433797000002 2023-10-26T06:46:11.283 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:46:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f000a15e0 msgr2=0x7f9f000a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f000a15e0 0x7f9f000a3aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009c70 tx=0x7f9f08011720 comp rx=0 tx=0).stop 2023-10-26T06:46:11.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f180734b0 msgr2=0x7f9f1819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f180734b0 0x7f9f1819d6e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c000c00 tx=0x7f9f0c004730 comp rx=0 tx=0).stop 2023-10-26T06:46:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 shutdown_connections 2023-10-26T06:46:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9f000a15e0 0x7f9f000a3aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f181116b0 0x7f9f181a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f18072b90 0x7f9f1819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 --2- 172.21.15.89:0/557352859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f180734b0 0x7f9f1819d6e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:11.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 >> 172.21.15.89:0/557352859 conn(0x7f9f180fc190 msgr2=0x7f9f180fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:11.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 shutdown_connections 2023-10-26T06:46:11.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:11.266+0000 7f9f1e9ff700 1 -- 172.21.15.89:0/557352859 wait complete. 2023-10-26T06:46:11.287 INFO:tasks.cephadm:Setup complete, yielding 2023-10-26T06:46:11.288 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-26T06:46:11.296 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T06:46:11.297 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-26T06:46:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:11 smithi089 ceph-mon[95031]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:11 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1073156412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:11 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:11 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3097514816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:11 smithi192 ceph-mon[131661]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:11 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1073156412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:11.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:11 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:11 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3097514816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:11 smithi135 ceph-mon[131403]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:11.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:11 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1073156412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:11.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:11 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:11.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:11 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3097514816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:12.033 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:12.878 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.878 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.878 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:12 smithi089 ceph-mon[95031]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:12.879 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:46:12.879 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:46:12.879 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:12 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:12 smithi192 ceph-mon[131661]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:12.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:46:12.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:46:12.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:12 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:12.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:12 smithi135 ceph-mon[131403]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:12.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:46:12.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:46:12.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:12 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:46:13.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 -- 172.21.15.89:0/2802164965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 msgr2=0x7fe9a0106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:13.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/2802164965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 0x7fe9a0106a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe990009990 tx=0x7fe990009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 -- 172.21.15.89:0/2802164965 shutdown_connections 2023-10-26T06:46:13.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/2802164965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 0x7fe9a0106a60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/2802164965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a0100fb0 0x7fe9a010d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/2802164965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a01005f0 0x7fe9a0100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 -- 172.21.15.89:0/2802164965 >> 172.21.15.89:0/2802164965 conn(0x7fe9a00fc120 msgr2=0x7fe9a00fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:13.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 -- 172.21.15.89:0/2802164965 shutdown_connections 2023-10-26T06:46:13.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.515+0000 7fe9a4b90700 1 -- 172.21.15.89:0/2802164965 wait complete. 2023-10-26T06:46:13.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.516+0000 7fe9a4b90700 1 Processor -- start 2023-10-26T06:46:13.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.516+0000 7fe9a4b90700 1 -- start start 2023-10-26T06:46:13.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe9a4b90700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a01005f0 0x7fe9a019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:13.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe9a4b90700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a0100fb0 0x7fe9a019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:13.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe9a4b90700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 0x7fe9a0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:13.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe9a4b90700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a01990a0 con 0x7fe9a0106680 2023-10-26T06:46:13.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe9a4b90700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a0199210 con 0x7fe9a0100fb0 2023-10-26T06:46:13.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe9a4b90700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a01993d0 con 0x7fe9a01005f0 2023-10-26T06:46:13.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe99f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a01005f0 0x7fe9a019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:13.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe99ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 0x7fe9a0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:13.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe99f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a01005f0 0x7fe9a019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60808/0 (socket says 172.21.15.89:60808) 2023-10-26T06:46:13.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe99f7fe700 1 -- 172.21.15.89:0/3530429827 learned_addr learned my addr 172.21.15.89:0/3530429827 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:13.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.517+0000 7fe99effd700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a0100fb0 0x7fe9a019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:13.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99ffff700 1 -- 172.21.15.89:0/3530429827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a01005f0 msgr2=0x7fe9a019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:13.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99ffff700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a01005f0 0x7fe9a019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99ffff700 1 -- 172.21.15.89:0/3530429827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a0100fb0 msgr2=0x7fe9a019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:13.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99ffff700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a0100fb0 0x7fe9a019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99ffff700 1 -- 172.21.15.89:0/3530429827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900095f0 con 0x7fe9a0106680 2023-10-26T06:46:13.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99effd700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a0100fb0 0x7fe9a019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:13.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99f7fe700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a01005f0 0x7fe9a019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:13.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99ffff700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 0x7fe9a0198a40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe998007f10 tx=0x7fe99800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:13.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99cff9700 1 -- 172.21.15.89:0/3530429827 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998009ea0 con 0x7fe9a0106680 2023-10-26T06:46:13.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99cff9700 1 -- 172.21.15.89:0/3530429827 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe99800f040 con 0x7fe9a0106680 2023-10-26T06:46:13.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe99cff9700 1 -- 172.21.15.89:0/3530429827 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998004400 con 0x7fe9a0106680 2023-10-26T06:46:13.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a01996b0 con 0x7fe9a0106680 2023-10-26T06:46:13.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.518+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a01a58f0 con 0x7fe9a0106680 2023-10-26T06:46:13.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.519+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a0101d30 con 0x7fe9a0106680 2023-10-26T06:46:13.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.521+0000 7fe99cff9700 1 -- 172.21.15.89:0/3530429827 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fe998004ad0 con 0x7fe9a0106680 2023-10-26T06:46:13.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.523+0000 7fe99cff9700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe9880a15e0 0x7fe9880a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:13.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.523+0000 7fe99cff9700 1 -- 172.21.15.89:0/3530429827 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fe9980cd2a0 con 0x7fe9a0106680 2023-10-26T06:46:13.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.523+0000 7fe99f7fe700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe9880a15e0 0x7fe9880a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:13.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.523+0000 7fe99f7fe700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe9880a15e0 0x7fe9880a3aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe990006010 tx=0x7fe99000b280 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:13.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.525+0000 7fe99cff9700 1 -- 172.21.15.89:0/3530429827 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe998096660 con 0x7fe9a0106680 2023-10-26T06:46:13.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.742+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9a0110580 con 0x7fe9880a15e0 2023-10-26T06:46:13.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.743+0000 7fe99cff9700 1 -- 172.21.15.89:0/3530429827 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe9a0110580 con 0x7fe9880a15e0 2023-10-26T06:46:13.744 INFO:teuthology.orchestra.run.smithi089.stdout:Backend: cephadm 2023-10-26T06:46:13.745 INFO:teuthology.orchestra.run.smithi089.stdout:Available: Yes 2023-10-26T06:46:13.745 INFO:teuthology.orchestra.run.smithi089.stdout:Paused: No 2023-10-26T06:46:13.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe9880a15e0 msgr2=0x7fe9880a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:13.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe9880a15e0 0x7fe9880a3aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe990006010 tx=0x7fe99000b280 comp rx=0 tx=0).stop 2023-10-26T06:46:13.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 msgr2=0x7fe9a0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:13.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 0x7fe9a0198a40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe998007f10 tx=0x7fe99800d3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 shutdown_connections 2023-10-26T06:46:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe9880a15e0 0x7fe9880a3aa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9a0106680 0x7fe9a0198a40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9a01005f0 0x7fe9a019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 --2- 172.21.15.89:0/3530429827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9a0100fb0 0x7fe9a019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 >> 172.21.15.89:0/3530429827 conn(0x7fe9a00fc120 msgr2=0x7fe9a00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 shutdown_connections 2023-10-26T06:46:13.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:13.745+0000 7fe9a4b90700 1 -- 172.21.15.89:0/3530429827 wait complete. 2023-10-26T06:46:14.311 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-26T06:46:15.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:14 smithi089 ceph-mon[95031]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 974 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:15.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:14 smithi089 ceph-mon[95031]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:15.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:14 smithi192 ceph-mon[131661]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 974 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:15.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:14 smithi192 ceph-mon[131661]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:15.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:14 smithi135 ceph-mon[131403]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 974 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:15.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:14 smithi135 ceph-mon[131403]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:16.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 -- 172.21.15.89:0/4114181624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb541057c0 msgr2=0x7fbb54105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:16.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/4114181624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb541057c0 0x7fbb54105ba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009990 tx=0x7fbb50009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 -- 172.21.15.89:0/4114181624 shutdown_connections 2023-10-26T06:46:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/4114181624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb541057c0 0x7fbb54105ba0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/4114181624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb54101260 0x7fbb5410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/4114181624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb541008a0 0x7fbb54100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 -- 172.21.15.89:0/4114181624 >> 172.21.15.89:0/4114181624 conn(0x7fbb540779e0 msgr2=0x7fbb54077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 -- 172.21.15.89:0/4114181624 shutdown_connections 2023-10-26T06:46:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.550+0000 7fbb5b37e700 1 -- 172.21.15.89:0/4114181624 wait complete. 2023-10-26T06:46:16.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 Processor -- start 2023-10-26T06:46:16.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 -- start start 2023-10-26T06:46:16.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb541008a0 0x7fbb5419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:16.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb54101260 0x7fbb5419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb541057c0 0x7fbb54196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54197160 con 0x7fbb54101260 2023-10-26T06:46:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541972d0 con 0x7fbb541057c0 2023-10-26T06:46:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.551+0000 7fbb5b37e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54197490 con 0x7fbb541008a0 2023-10-26T06:46:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb58919700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb54101260 0x7fbb5419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb58919700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb54101260 0x7fbb5419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41106/0 (socket says 172.21.15.89:41106) 2023-10-26T06:46:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb58919700 1 -- 172.21.15.89:0/2657135679 learned_addr learned my addr 172.21.15.89:0/2657135679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5911a700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb541008a0 0x7fbb5419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5991b700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb541057c0 0x7fbb54196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5911a700 1 -- 172.21.15.89:0/2657135679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb541057c0 msgr2=0x7fbb54196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5911a700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb541057c0 0x7fbb54196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5911a700 1 -- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb54101260 msgr2=0x7fbb5419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5911a700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb54101260 0x7fbb5419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5911a700 1 -- 172.21.15.89:0/2657135679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500095f0 con 0x7fbb541008a0 2023-10-26T06:46:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb58919700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb54101260 0x7fbb5419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.552+0000 7fbb5911a700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb541008a0 0x7fbb5419c970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000c00 tx=0x7fbb50009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.553+0000 7fbb4a7fc700 1 -- 172.21.15.89:0/2657135679 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5001d070 con 0x7fbb541008a0 2023-10-26T06:46:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.553+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb54197710 con 0x7fbb541008a0 2023-10-26T06:46:16.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.553+0000 7fbb4a7fc700 1 -- 172.21.15.89:0/2657135679 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb50021410 con 0x7fbb541008a0 2023-10-26T06:46:16.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.553+0000 7fbb4a7fc700 1 -- 172.21.15.89:0/2657135679 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50017650 con 0x7fbb541008a0 2023-10-26T06:46:16.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.553+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb541a3960 con 0x7fbb541008a0 2023-10-26T06:46:16.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.554+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb54108d80 con 0x7fbb541008a0 2023-10-26T06:46:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.556+0000 7fbb4a7fc700 1 -- 172.21.15.89:0/2657135679 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7fbb50021580 con 0x7fbb541008a0 2023-10-26T06:46:16.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.557+0000 7fbb4a7fc700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb400a1560 0x7fbb400a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.557+0000 7fbb4a7fc700 1 -- 172.21.15.89:0/2657135679 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbb500ce9b0 con 0x7fbb541008a0 2023-10-26T06:46:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.558+0000 7fbb58919700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb400a1560 0x7fbb400a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.558+0000 7fbb58919700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb400a1560 0x7fbb400a3a20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb54197c70 tx=0x7fbb44009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:16.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.559+0000 7fbb4a7fc700 1 -- 172.21.15.89:0/2657135679 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb50097e00 con 0x7fbb541008a0 2023-10-26T06:46:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.771+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb5404e4c0 con 0x7fbb400a1560 2023-10-26T06:46:16.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.785+0000 7fbb4a7fc700 1 -- 172.21.15.89:0/2657135679 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fbb5404e4c0 con 0x7fbb400a1560 2023-10-26T06:46:16.787 INFO:teuthology.orchestra.run.smithi089.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-26T06:46:16.787 INFO:teuthology.orchestra.run.smithi089.stdout:mds.a smithi089 running (80s) 5s ago 80s 13.7M - 18.0.0-6863-g38499368 3c558842a6a5 8408a6e86e9b 2023-10-26T06:46:16.787 INFO:teuthology.orchestra.run.smithi089.stdout:mds.b smithi135 running (78s) 6s ago 77s 31.5M - 18.0.0-6863-g38499368 3c558842a6a5 4088f71f66d3 2023-10-26T06:46:16.787 INFO:teuthology.orchestra.run.smithi089.stdout:mds.c smithi192 running (91s) 6s ago 90s 15.1M - 18.0.0-6863-g38499368 3c558842a6a5 9fa49fb886ad 2023-10-26T06:46:16.788 INFO:teuthology.orchestra.run.smithi089.stdout:mds.d smithi089 running (97s) 5s ago 97s 13.8M - 18.0.0-6863-g38499368 3c558842a6a5 a1775dd227e0 2023-10-26T06:46:16.788 INFO:teuthology.orchestra.run.smithi089.stdout:mds.e smithi135 running (95s) 6s ago 95s 15.1M - 18.0.0-6863-g38499368 3c558842a6a5 c9cc36d145f6 2023-10-26T06:46:16.788 INFO:teuthology.orchestra.run.smithi089.stdout:mds.f smithi192 running (93s) 6s ago 93s 15.1M - 18.0.0-6863-g38499368 3c558842a6a5 3f195a45af56 2023-10-26T06:46:16.788 INFO:teuthology.orchestra.run.smithi089.stdout:mds.g smithi089 running (83s) 5s ago 83s 13.7M - 18.0.0-6863-g38499368 3c558842a6a5 b68d08e7bce8 2023-10-26T06:46:16.788 INFO:teuthology.orchestra.run.smithi089.stdout:mds.h smithi135 running (88s) 6s ago 88s 17.5M - 18.0.0-6863-g38499368 3c558842a6a5 c62a16d96d25 2023-10-26T06:46:16.789 INFO:teuthology.orchestra.run.smithi089.stdout:mds.i smithi192 running (86s) 6s ago 86s 19.5M - 18.0.0-6863-g38499368 3c558842a6a5 135c63b907ff 2023-10-26T06:46:16.789 INFO:teuthology.orchestra.run.smithi089.stdout:mds.j smithi089 running (100s) 5s ago 100s 13.8M - 18.0.0-6863-g38499368 3c558842a6a5 e746f99c230b 2023-10-26T06:46:16.789 INFO:teuthology.orchestra.run.smithi089.stdout:mds.k smithi135 running (103s) 6s ago 103s 16.6M - 18.0.0-6863-g38499368 3c558842a6a5 5fa4686d8535 2023-10-26T06:46:16.789 INFO:teuthology.orchestra.run.smithi089.stdout:mds.l smithi192 running (106s) 6s ago 105s 15.2M - 18.0.0-6863-g38499368 3c558842a6a5 4b1b2fc318c5 2023-10-26T06:46:16.789 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.x smithi089 *:9283,8765 running (9m) 5s ago 9m 520M - 18.0.0-6863-g38499368 3c558842a6a5 1fe63502cfde 2023-10-26T06:46:16.790 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.y smithi135 *:8443,8765 running (7m) 6s ago 7m 450M - 18.0.0-6863-g38499368 3c558842a6a5 a80388b3afbe 2023-10-26T06:46:16.790 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.z smithi192 *:8443,8765 running (7m) 6s ago 7m 450M - 18.0.0-6863-g38499368 3c558842a6a5 bb5a7ff15e5b 2023-10-26T06:46:16.790 INFO:teuthology.orchestra.run.smithi089.stdout:mon.a smithi089 running (9m) 5s ago 9m 98.7M 2048M 18.0.0-6863-g38499368 3c558842a6a5 72ab460de1dd 2023-10-26T06:46:16.790 INFO:teuthology.orchestra.run.smithi089.stdout:mon.b smithi135 running (7m) 6s ago 7m 68.4M 2048M 18.0.0-6863-g38499368 3c558842a6a5 185c8937ba0c 2023-10-26T06:46:16.790 INFO:teuthology.orchestra.run.smithi089.stdout:mon.c smithi192 running (7m) 6s ago 7m 68.9M 2048M 18.0.0-6863-g38499368 3c558842a6a5 b12c0617d732 2023-10-26T06:46:16.791 INFO:teuthology.orchestra.run.smithi089.stdout:osd.0 smithi089 running (6m) 5s ago 6m 61.4M 4317M 18.0.0-6863-g38499368 3c558842a6a5 fb583aea4455 2023-10-26T06:46:16.791 INFO:teuthology.orchestra.run.smithi089.stdout:osd.1 smithi135 running (5m) 6s ago 5m 68.6M 4292M 18.0.0-6863-g38499368 3c558842a6a5 c3549e6c4fd0 2023-10-26T06:46:16.791 INFO:teuthology.orchestra.run.smithi089.stdout:osd.2 smithi192 running (5m) 6s ago 5m 63.8M 4292M 18.0.0-6863-g38499368 3c558842a6a5 ce094d7e5e41 2023-10-26T06:46:16.791 INFO:teuthology.orchestra.run.smithi089.stdout:osd.3 smithi089 running (5m) 5s ago 5m 58.1M 4317M 18.0.0-6863-g38499368 3c558842a6a5 595e5206c5fe 2023-10-26T06:46:16.791 INFO:teuthology.orchestra.run.smithi089.stdout:osd.4 smithi135 running (4m) 6s ago 4m 57.6M 4292M 18.0.0-6863-g38499368 3c558842a6a5 878c3f90d180 2023-10-26T06:46:16.792 INFO:teuthology.orchestra.run.smithi089.stdout:osd.5 smithi192 running (4m) 6s ago 4m 59.7M 4292M 18.0.0-6863-g38499368 3c558842a6a5 ba0797af66c2 2023-10-26T06:46:16.792 INFO:teuthology.orchestra.run.smithi089.stdout:osd.6 smithi089 running (3m) 5s ago 3m 55.1M 4317M 18.0.0-6863-g38499368 3c558842a6a5 fefa43884db5 2023-10-26T06:46:16.792 INFO:teuthology.orchestra.run.smithi089.stdout:osd.7 smithi135 running (3m) 6s ago 3m 59.9M 4292M 18.0.0-6863-g38499368 3c558842a6a5 f346fefe31d0 2023-10-26T06:46:16.792 INFO:teuthology.orchestra.run.smithi089.stdout:osd.8 smithi192 running (3m) 6s ago 3m 62.2M 4292M 18.0.0-6863-g38499368 3c558842a6a5 319f33dfa468 2023-10-26T06:46:16.792 INFO:teuthology.orchestra.run.smithi089.stdout:osd.9 smithi089 running (2m) 5s ago 2m 53.5M 4317M 18.0.0-6863-g38499368 3c558842a6a5 b2da2728498f 2023-10-26T06:46:16.792 INFO:teuthology.orchestra.run.smithi089.stdout:osd.10 smithi135 running (2m) 6s ago 2m 59.2M 4292M 18.0.0-6863-g38499368 3c558842a6a5 0c84b7c9204c 2023-10-26T06:46:16.793 INFO:teuthology.orchestra.run.smithi089.stdout:osd.11 smithi192 running (2m) 6s ago 2m 53.8M 4292M 18.0.0-6863-g38499368 3c558842a6a5 67df1a5f2a9f 2023-10-26T06:46:16.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.787+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb400a1560 msgr2=0x7fbb400a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.787+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb400a1560 0x7fbb400a3a20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb54197c70 tx=0x7fbb44009260 comp rx=0 tx=0).stop 2023-10-26T06:46:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.787+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb541008a0 msgr2=0x7fbb5419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.787+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb541008a0 0x7fbb5419c970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000c00 tx=0x7fbb50009580 comp rx=0 tx=0).stop 2023-10-26T06:46:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 shutdown_connections 2023-10-26T06:46:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb400a1560 0x7fbb400a3a20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb541057c0 0x7fbb54196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb541008a0 0x7fbb5419c970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 --2- 172.21.15.89:0/2657135679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb54101260 0x7fbb5419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 >> 172.21.15.89:0/2657135679 conn(0x7fbb540779e0 msgr2=0x7fbb540fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 shutdown_connections 2023-10-26T06:46:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:16.788+0000 7fbb5b37e700 1 -- 172.21.15.89:0/2657135679 wait complete. 2023-10-26T06:46:17.225 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-26T06:46:17.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:17 smithi192 ceph-mon[131661]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:17.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:17 smithi135 ceph-mon[131403]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:17.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:17 smithi089 ceph-mon[95031]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2023-10-26T06:46:17.956 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:18.376 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:18 smithi089 ceph-mon[95031]: from='client.25159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:18.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:18 smithi192 ceph-mon[131661]: from='client.25159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:18.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:18 smithi135 ceph-mon[131403]: from='client.25159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:19.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:19 smithi192 ceph-mon[131661]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:19.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 -- 172.21.15.89:0/881695986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c98072a20 msgr2=0x7f1c98072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:19.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 --2- 172.21.15.89:0/881695986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c98072a20 0x7f1c98072e00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009990 tx=0x7f1c88009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 -- 172.21.15.89:0/881695986 shutdown_connections 2023-10-26T06:46:19.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 --2- 172.21.15.89:0/881695986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c98072a20 0x7f1c98072e00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 --2- 172.21.15.89:0/881695986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c9810d250 0x7f1c9810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 --2- 172.21.15.89:0/881695986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c98073340 0x7f1c9810cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 -- 172.21.15.89:0/881695986 >> 172.21.15.89:0/881695986 conn(0x7f1c980779e0 msgr2=0x7f1c98077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:19.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 -- 172.21.15.89:0/881695986 shutdown_connections 2023-10-26T06:46:19.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.441+0000 7f1c9d662700 1 -- 172.21.15.89:0/881695986 wait complete. 2023-10-26T06:46:19.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.442+0000 7f1c9d662700 1 Processor -- start 2023-10-26T06:46:19.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.442+0000 7f1c9d662700 1 -- start start 2023-10-26T06:46:19.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c9d662700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98072a20 0x7f1c98198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:19.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c9d662700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c98073340 0x7f1c98199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:19.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c9d662700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9810d250 0x7f1c9819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:19.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c9d662700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9819dde0 con 0x7f1c9810d250 2023-10-26T06:46:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c9d662700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9819df50 con 0x7f1c98072a20 2023-10-26T06:46:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c9d662700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9819e110 con 0x7f1c98073340 2023-10-26T06:46:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9810d250 0x7f1c9819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c977fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9810d250 0x7f1c9819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41128/0 (socket says 172.21.15.89:41128) 2023-10-26T06:46:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c977fe700 1 -- 172.21.15.89:0/2722878201 learned_addr learned my addr 172.21.15.89:0/2722878201 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c96ffd700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98072a20 0x7f1c98198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c967fc700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c98073340 0x7f1c98199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c977fe700 1 -- 172.21.15.89:0/2722878201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c98073340 msgr2=0x7f1c98199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c977fe700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c98073340 0x7f1c98199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.443+0000 7f1c977fe700 1 -- 172.21.15.89:0/2722878201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98072a20 msgr2=0x7f1c98198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c977fe700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98072a20 0x7f1c98198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c977fe700 1 -- 172.21.15.89:0/2722878201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c880095f0 con 0x7f1c9810d250 2023-10-26T06:46:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c967fc700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c98073340 0x7f1c98199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c977fe700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9810d250 0x7f1c9819d7d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c90007f10 tx=0x7f1c9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c7ffff700 1 -- 172.21.15.89:0/2722878201 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90009ea0 con 0x7f1c9810d250 2023-10-26T06:46:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c7ffff700 1 -- 172.21.15.89:0/2722878201 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c9000f040 con 0x7f1c9810d250 2023-10-26T06:46:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9819e3f0 con 0x7f1c9810d250 2023-10-26T06:46:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c7ffff700 1 -- 172.21.15.89:0/2722878201 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90012680 con 0x7f1c9810d250 2023-10-26T06:46:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.444+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c981a5c50 con 0x7f1c9810d250 2023-10-26T06:46:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.445+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c98108460 con 0x7f1c9810d250 2023-10-26T06:46:19.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.447+0000 7f1c7ffff700 1 -- 172.21.15.89:0/2722878201 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f1c900139b0 con 0x7f1c9810d250 2023-10-26T06:46:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.449+0000 7f1c7ffff700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1c800a15e0 0x7f1c800a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.450+0000 7f1c7ffff700 1 -- 172.21.15.89:0/2722878201 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1c900ccc30 con 0x7f1c9810d250 2023-10-26T06:46:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.450+0000 7f1c96ffd700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1c800a15e0 0x7f1c800a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.450+0000 7f1c96ffd700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1c800a15e0 0x7f1c800a3aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b5c0 tx=0x7f1c88005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:19.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.454+0000 7f1c7ffff700 1 -- 172.21.15.89:0/2722878201 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1c90095ff0 con 0x7f1c9810d250 2023-10-26T06:46:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:19 smithi135 ceph-mon[131403]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:19 smithi089 ceph-mon[95031]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:19.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.662+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c9819a1f0 con 0x7f1c800a15e0 2023-10-26T06:46:19.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.668+0000 7f1c7ffff700 1 -- 172.21.15.89:0/2722878201 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f1c9819a1f0 con 0x7f1c800a15e0 2023-10-26T06:46:19.670 INFO:teuthology.orchestra.run.smithi089.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-26T06:46:19.670 INFO:teuthology.orchestra.run.smithi089.stdout:mds.all 12/12 9s ago 110s smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2023-10-26T06:46:19.671 INFO:teuthology.orchestra.run.smithi089.stdout:mgr 3/3 9s ago 7m smithi089=x;smithi135=y;smithi192=z;count:3 2023-10-26T06:46:19.671 INFO:teuthology.orchestra.run.smithi089.stdout:mon 3/3 9s ago 7m smithi089:172.21.15.89=a;smithi135:172.21.15.135=b;smithi192:172.21.15.192=c;count:3 2023-10-26T06:46:19.671 INFO:teuthology.orchestra.run.smithi089.stdout:osd 12 9s ago - 2023-10-26T06:46:19.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1c800a15e0 msgr2=0x7f1c800a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:19.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1c800a15e0 0x7f1c800a3aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800b5c0 tx=0x7f1c88005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9810d250 msgr2=0x7f1c9819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9810d250 0x7f1c9819d7d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1c90007f10 tx=0x7f1c9000d3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 shutdown_connections 2023-10-26T06:46:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1c800a15e0 0x7f1c800a3aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9810d250 0x7f1c9819d7d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98072a20 0x7f1c98198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 --2- 172.21.15.89:0/2722878201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c98073340 0x7f1c98199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 >> 172.21.15.89:0/2722878201 conn(0x7f1c980779e0 msgr2=0x7f1c980fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 shutdown_connections 2023-10-26T06:46:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:19.670+0000 7f1c9d662700 1 -- 172.21.15.89:0/2722878201 wait complete. 2023-10-26T06:46:20.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-26T06:46:20.876 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:21.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:21 smithi192 ceph-mon[131661]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:21.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:21 smithi192 ceph-mon[131661]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:21.428 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:21 smithi089 ceph-mon[95031]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:21.428 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:21 smithi089 ceph-mon[95031]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:21 smithi135 ceph-mon[131403]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:21 smithi135 ceph-mon[131403]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 -- 172.21.15.89:0/750902292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c1008a0 msgr2=0x7f294c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 --2- 172.21.15.89:0/750902292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c1008a0 0x7f294c100d20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2938009990 tx=0x7f2938009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 -- 172.21.15.89:0/750902292 shutdown_connections 2023-10-26T06:46:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 --2- 172.21.15.89:0/750902292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c101260 0x7f294c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 --2- 172.21.15.89:0/750902292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f294c1057c0 0x7f294c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 --2- 172.21.15.89:0/750902292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c1008a0 0x7f294c100d20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 -- 172.21.15.89:0/750902292 >> 172.21.15.89:0/750902292 conn(0x7f294c0779e0 msgr2=0x7f294c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 -- 172.21.15.89:0/750902292 shutdown_connections 2023-10-26T06:46:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.366+0000 7f2952c6c700 1 -- 172.21.15.89:0/750902292 wait complete. 2023-10-26T06:46:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.367+0000 7f2952c6c700 1 Processor -- start 2023-10-26T06:46:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.367+0000 7f2952c6c700 1 -- start start 2023-10-26T06:46:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2952c6c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c1008a0 0x7f294c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2952c6c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c101260 0x7f294c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2952c6c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f294c1057c0 0x7f294c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2952c6c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c19dd30 con 0x7f294c101260 2023-10-26T06:46:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2952c6c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c19dea0 con 0x7f294c1057c0 2023-10-26T06:46:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2952c6c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c19e060 con 0x7f294c1008a0 2023-10-26T06:46:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c101260 0x7f294c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2950a08700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c1008a0 0x7f294c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c101260 0x7f294c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42626/0 (socket says 172.21.15.89:42626) 2023-10-26T06:46:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 -- 172.21.15.89:0/2612552797 learned_addr learned my addr 172.21.15.89:0/2612552797 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f2951209700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f294c1057c0 0x7f294c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 -- 172.21.15.89:0/2612552797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c1008a0 msgr2=0x7f294c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c1008a0 0x7f294c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 -- 172.21.15.89:0/2612552797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f294c1057c0 msgr2=0x7f294c19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f294c1057c0 0x7f294c19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.368+0000 7f294bfff700 1 -- 172.21.15.89:0/2612552797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380095f0 con 0x7f294c101260 2023-10-26T06:46:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.369+0000 7f294bfff700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c101260 0x7f294c199370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2938005910 tx=0x7f2938004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.369+0000 7f2949ffb700 1 -- 172.21.15.89:0/2612552797 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293801d070 con 0x7f294c101260 2023-10-26T06:46:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.369+0000 7f2949ffb700 1 -- 172.21.15.89:0/2612552797 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2938022470 con 0x7f294c101260 2023-10-26T06:46:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.369+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c19e2e0 con 0x7f294c101260 2023-10-26T06:46:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.369+0000 7f2949ffb700 1 -- 172.21.15.89:0/2612552797 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938017650 con 0x7f294c101260 2023-10-26T06:46:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.369+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c1a5ba0 con 0x7f294c101260 2023-10-26T06:46:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.370+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c101f30 con 0x7f294c101260 2023-10-26T06:46:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.372+0000 7f2949ffb700 1 -- 172.21.15.89:0/2612552797 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f29380225e0 con 0x7f294c101260 2023-10-26T06:46:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.373+0000 7f2949ffb700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f293c0a1560 0x7f293c0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.373+0000 7f2949ffb700 1 -- 172.21.15.89:0/2612552797 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f29380ce320 con 0x7f294c101260 2023-10-26T06:46:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.374+0000 7f2950a08700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f293c0a1560 0x7f293c0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.374+0000 7f2950a08700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f293c0a1560 0x7f293c0a3a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f294c101930 tx=0x7f2940009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:22.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.377+0000 7f2949ffb700 1 -- 172.21.15.89:0/2612552797 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2938097770 con 0x7f294c101260 2023-10-26T06:46:22.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.581+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f294c19a140 con 0x7f293c0a1560 2023-10-26T06:46:22.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.583+0000 7f2949ffb700 1 -- 172.21.15.89:0/2612552797 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f294c19a140 con 0x7f293c0a1560 2023-10-26T06:46:22.584 INFO:teuthology.orchestra.run.smithi089.stdout:HOST ADDR LABELS STATUS 2023-10-26T06:46:22.584 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089 172.21.15.89 2023-10-26T06:46:22.584 INFO:teuthology.orchestra.run.smithi089.stdout:smithi135 172.21.15.135 2023-10-26T06:46:22.585 INFO:teuthology.orchestra.run.smithi089.stdout:smithi192 172.21.15.192 2023-10-26T06:46:22.585 INFO:teuthology.orchestra.run.smithi089.stdout:3 hosts in cluster 2023-10-26T06:46:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f293c0a1560 msgr2=0x7f293c0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:22.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f293c0a1560 0x7f293c0a3a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f294c101930 tx=0x7f2940009260 comp rx=0 tx=0).stop 2023-10-26T06:46:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c101260 msgr2=0x7f294c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c101260 0x7f294c199370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2938005910 tx=0x7f2938004730 comp rx=0 tx=0).stop 2023-10-26T06:46:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 shutdown_connections 2023-10-26T06:46:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f293c0a1560 0x7f293c0a3a20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f294c1057c0 0x7f294c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c1008a0 0x7f294c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 --2- 172.21.15.89:0/2612552797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f294c101260 0x7f294c199370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 >> 172.21.15.89:0/2612552797 conn(0x7f294c0779e0 msgr2=0x7f294c10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 shutdown_connections 2023-10-26T06:46:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:22.585+0000 7f2952c6c700 1 -- 172.21.15.89:0/2612552797 wait complete. 2023-10-26T06:46:23.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-26T06:46:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:23 smithi192 ceph-mon[131661]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:23 smithi135 ceph-mon[131403]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:23.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:23 smithi089 ceph-mon[95031]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:23.794 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:24.336 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:24 smithi089 ceph-mon[95031]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:24.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:24 smithi192 ceph-mon[131661]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:24.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:24 smithi135 ceph-mon[131403]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:25.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.183+0000 7f01024b2700 1 -- 172.21.15.89:0/1978507804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 msgr2=0x7f00fc105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:25.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.183+0000 7f01024b2700 1 --2- 172.21.15.89:0/1978507804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc105ba0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009990 tx=0x7f00ec009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.183+0000 7f01024b2700 1 -- 172.21.15.89:0/1978507804 shutdown_connections 2023-10-26T06:46:25.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.183+0000 7f01024b2700 1 --2- 172.21.15.89:0/1978507804 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc105ba0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.183+0000 7f01024b2700 1 --2- 172.21.15.89:0/1978507804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00fc101260 0x7f00fc10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.183+0000 7f01024b2700 1 --2- 172.21.15.89:0/1978507804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00fc1008a0 0x7f00fc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.183+0000 7f01024b2700 1 -- 172.21.15.89:0/1978507804 >> 172.21.15.89:0/1978507804 conn(0x7f00fc0779e0 msgr2=0x7f00fc077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:25.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.184+0000 7f01024b2700 1 -- 172.21.15.89:0/1978507804 shutdown_connections 2023-10-26T06:46:25.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.184+0000 7f01024b2700 1 -- 172.21.15.89:0/1978507804 wait complete. 2023-10-26T06:46:25.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 Processor -- start 2023-10-26T06:46:25.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 -- start start 2023-10-26T06:46:25.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00fc1008a0 0x7f00fc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:25.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00fc101260 0x7f00fc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:25.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:25.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc19dd20 con 0x7f00fc1057c0 2023-10-26T06:46:25.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc19de90 con 0x7f00fc1008a0 2023-10-26T06:46:25.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.185+0000 7f01024b2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc19e050 con 0x7f00fc101260 2023-10-26T06:46:25.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f00fb7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00fc101260 0x7f00fc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:25.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:25.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42658/0 (socket says 172.21.15.89:42658) 2023-10-26T06:46:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 -- 172.21.15.89:0/3092358599 learned_addr learned my addr 172.21.15.89:0/3092358599 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f00fb7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00fc101260 0x7f00fc199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46304/0 (socket says 172.21.15.89:46304) 2023-10-26T06:46:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f00fbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00fc1008a0 0x7f00fc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 -- 172.21.15.89:0/3092358599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00fc101260 msgr2=0x7f00fc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00fc101260 0x7f00fc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 -- 172.21.15.89:0/3092358599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00fc1008a0 msgr2=0x7f00fc198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:25.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00fc1008a0 0x7f00fc198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 -- 172.21.15.89:0/3092358599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec0095f0 con 0x7f00fc1057c0 2023-10-26T06:46:25.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f0100a4f700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc19d710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f00f4007f10 tx=0x7f00f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:25.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f00f97fa700 1 -- 172.21.15.89:0/3092358599 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4017070 con 0x7f00fc1057c0 2023-10-26T06:46:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f00f97fa700 1 -- 172.21.15.89:0/3092358599 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f400f040 con 0x7f00fc1057c0 2023-10-26T06:46:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc19e330 con 0x7f00fc1057c0 2023-10-26T06:46:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.186+0000 7f00f97fa700 1 -- 172.21.15.89:0/3092358599 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f40054b0 con 0x7f00fc1057c0 2023-10-26T06:46:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.187+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc1a5b90 con 0x7f00fc1057c0 2023-10-26T06:46:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.188+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc101f30 con 0x7f00fc1057c0 2023-10-26T06:46:25.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.190+0000 7f00f97fa700 1 -- 172.21.15.89:0/3092358599 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f00f4004ad0 con 0x7f00fc1057c0 2023-10-26T06:46:25.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.191+0000 7f00f97fa700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00e40a1560 0x7f00e40a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.191+0000 7f00f97fa700 1 -- 172.21.15.89:0/3092358599 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f00f40cc590 con 0x7f00fc1057c0 2023-10-26T06:46:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.191+0000 7f00fbfff700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00e40a1560 0x7f00e40a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.192+0000 7f00fbfff700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00e40a1560 0x7f00e40a3a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006010 tx=0x7f00ec005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.195+0000 7f00f97fa700 1 -- 172.21.15.89:0/3092358599 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f00f40959b0 con 0x7f00fc1057c0 2023-10-26T06:46:25.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:25 smithi089 ceph-mon[95031]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:25 smithi192 ceph-mon[131661]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:25.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.401+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f00fc110710 con 0x7f00e40a1560 2023-10-26T06:46:25.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.402+0000 7f00f97fa700 1 -- 172.21.15.89:0/3092358599 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f00fc110710 con 0x7f00e40a1560 2023-10-26T06:46:25.404 INFO:teuthology.orchestra.run.smithi089.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-26T06:46:25.404 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-26T06:46:25.404 INFO:teuthology.orchestra.run.smithi089.stdout:smithi135 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401BA400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-26T06:46:25.404 INFO:teuthology.orchestra.run.smithi089.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-26T06:46:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00e40a1560 msgr2=0x7f00e40a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00e40a1560 0x7f00e40a3a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00ec006010 tx=0x7f00ec005960 comp rx=0 tx=0).stop 2023-10-26T06:46:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 msgr2=0x7f00fc19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:25.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc19d710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f00f4007f10 tx=0x7f00f400d3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 shutdown_connections 2023-10-26T06:46:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00e40a1560 0x7f00e40a3a20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00fc1057c0 0x7f00fc19d710 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00fc1008a0 0x7f00fc198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 --2- 172.21.15.89:0/3092358599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00fc101260 0x7f00fc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 >> 172.21.15.89:0/3092358599 conn(0x7f00fc0779e0 msgr2=0x7f00fc10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 shutdown_connections 2023-10-26T06:46:25.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:25.405+0000 7f01024b2700 1 -- 172.21.15.89:0/3092358599 wait complete. 2023-10-26T06:46:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:25 smithi135 ceph-mon[131403]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:25.871 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-26T06:46:25.880 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T06:46:25.881 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-26T06:46:26.617 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:27.161 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:27 smithi089 ceph-mon[95031]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:27.161 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:27 smithi089 ceph-mon[95031]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:27.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:27 smithi192 ceph-mon[131661]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:27.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:27 smithi192 ceph-mon[131661]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:27.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:27 smithi135 ceph-mon[131403]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:27.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:27 smithi135 ceph-mon[131403]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:28.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 -- 172.21.15.89:0/809494490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8288101260 msgr2=0x7f828810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:28.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 --2- 172.21.15.89:0/809494490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8288101260 0x7f828810d650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f828000b0a0 tx=0x7f828000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 -- 172.21.15.89:0/809494490 shutdown_connections 2023-10-26T06:46:28.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 --2- 172.21.15.89:0/809494490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8288101260 0x7f828810d650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 --2- 172.21.15.89:0/809494490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82881057c0 0x7f8288105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 --2- 172.21.15.89:0/809494490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82881008a0 0x7f8288100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 -- 172.21.15.89:0/809494490 >> 172.21.15.89:0/809494490 conn(0x7f82880779e0 msgr2=0x7f8288077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:28.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 -- 172.21.15.89:0/809494490 shutdown_connections 2023-10-26T06:46:28.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.064+0000 7f828ebe6700 1 -- 172.21.15.89:0/809494490 wait complete. 2023-10-26T06:46:28.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.065+0000 7f828ebe6700 1 Processor -- start 2023-10-26T06:46:28.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.065+0000 7f828ebe6700 1 -- start start 2023-10-26T06:46:28.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.065+0000 7f828ebe6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82881008a0 0x7f8288198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:28.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.065+0000 7f828ebe6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288101260 0x7f8288199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:28.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828ebe6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82881057c0 0x7f828819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:28.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828ebe6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828819de40 con 0x7f82881057c0 2023-10-26T06:46:28.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828ebe6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828819dfb0 con 0x7f8288101260 2023-10-26T06:46:28.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828ebe6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828819e170 con 0x7f82881008a0 2023-10-26T06:46:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82881057c0 0x7f828819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82881057c0 0x7f828819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42668/0 (socket says 172.21.15.89:42668) 2023-10-26T06:46:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 -- 172.21.15.89:0/2429032085 learned_addr learned my addr 172.21.15.89:0/2429032085 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828c982700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82881008a0 0x7f8288198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f8287fff700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288101260 0x7f8288199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:28.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 -- 172.21.15.89:0/2429032085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82881008a0 msgr2=0x7f8288198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:28.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82881008a0 0x7f8288198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 -- 172.21.15.89:0/2429032085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288101260 msgr2=0x7f8288199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:28.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288101260 0x7f8288199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 -- 172.21.15.89:0/2429032085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009d00 con 0x7f82881057c0 2023-10-26T06:46:28.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.066+0000 7f828d183700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82881057c0 0x7f828819d830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f828000b070 tx=0x7f82800078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:28.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.067+0000 7f8285ffb700 1 -- 172.21.15.89:0/2429032085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000e040 con 0x7f82881057c0 2023-10-26T06:46:28.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.067+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828819e3f0 con 0x7f82881057c0 2023-10-26T06:46:28.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.067+0000 7f8285ffb700 1 -- 172.21.15.89:0/2429032085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f828001d470 con 0x7f82881057c0 2023-10-26T06:46:28.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.067+0000 7f8285ffb700 1 -- 172.21.15.89:0/2429032085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800052a0 con 0x7f82881057c0 2023-10-26T06:46:28.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.067+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82881a5cb0 con 0x7f82881057c0 2023-10-26T06:46:28.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.068+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828819e660 con 0x7f82881057c0 2023-10-26T06:46:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.070+0000 7f8285ffb700 1 -- 172.21.15.89:0/2429032085 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f828001d850 con 0x7f82881057c0 2023-10-26T06:46:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.071+0000 7f8285ffb700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82780a1560 0x7f82780a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.071+0000 7f8285ffb700 1 -- 172.21.15.89:0/2429032085 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f82800ce4d0 con 0x7f82881057c0 2023-10-26T06:46:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.071+0000 7f828c982700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82780a1560 0x7f82780a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:28.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.072+0000 7f828c982700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82780a1560 0x7f82780a3a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8288101930 tx=0x7f827c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:28.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.073+0000 7f8285ffb700 1 -- 172.21.15.89:0/2429032085 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8280097920 con 0x7f82881057c0 2023-10-26T06:46:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.313+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8288101f30 con 0x7f82881057c0 2023-10-26T06:46:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.314+0000 7f8285ffb700 1 -- 172.21.15.89:0/2429032085 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+2915 (secure 0 0 0) 0x7f828001d5e0 con 0x7f82881057c0 2023-10-26T06:46:28.315 INFO:teuthology.orchestra.run.smithi089.stdout:e20 2023-10-26T06:46:28.316 INFO:teuthology.orchestra.run.smithi089.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-26T06:46:28.316 INFO:teuthology.orchestra.run.smithi089.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-26T06:46:28.316 INFO:teuthology.orchestra.run.smithi089.stdout:legacy client fscid: 1 2023-10-26T06:46:28.316 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:28.316 INFO:teuthology.orchestra.run.smithi089.stdout:Filesystem 'cephfs' (1) 2023-10-26T06:46:28.317 INFO:teuthology.orchestra.run.smithi089.stdout:fs_name cephfs 2023-10-26T06:46:28.317 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 20 2023-10-26T06:46:28.317 INFO:teuthology.orchestra.run.smithi089.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-26T06:46:28.317 INFO:teuthology.orchestra.run.smithi089.stdout:created 2023-10-26T06:45:05.445103+0000 2023-10-26T06:46:28.318 INFO:teuthology.orchestra.run.smithi089.stdout:modified 2023-10-26T06:45:09.558391+0000 2023-10-26T06:46:28.318 INFO:teuthology.orchestra.run.smithi089.stdout:tableserver 0 2023-10-26T06:46:28.318 INFO:teuthology.orchestra.run.smithi089.stdout:root 0 2023-10-26T06:46:28.318 INFO:teuthology.orchestra.run.smithi089.stdout:session_timeout 300 2023-10-26T06:46:28.318 INFO:teuthology.orchestra.run.smithi089.stdout:session_autoclose 300 2023-10-26T06:46:28.319 INFO:teuthology.orchestra.run.smithi089.stdout:max_file_size 1099511627776 2023-10-26T06:46:28.319 INFO:teuthology.orchestra.run.smithi089.stdout:max_xattr_size 65536 2023-10-26T06:46:28.319 INFO:teuthology.orchestra.run.smithi089.stdout:required_client_features {} 2023-10-26T06:46:28.319 INFO:teuthology.orchestra.run.smithi089.stdout:last_failure 0 2023-10-26T06:46:28.319 INFO:teuthology.orchestra.run.smithi089.stdout:last_failure_osd_epoch 0 2023-10-26T06:46:28.321 INFO:teuthology.orchestra.run.smithi089.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-26T06:46:28.321 INFO:teuthology.orchestra.run.smithi089.stdout:max_mds 1 2023-10-26T06:46:28.321 INFO:teuthology.orchestra.run.smithi089.stdout:in 0 2023-10-26T06:46:28.321 INFO:teuthology.orchestra.run.smithi089.stdout:up {0=24743} 2023-10-26T06:46:28.322 INFO:teuthology.orchestra.run.smithi089.stdout:failed 2023-10-26T06:46:28.322 INFO:teuthology.orchestra.run.smithi089.stdout:damaged 2023-10-26T06:46:28.323 INFO:teuthology.orchestra.run.smithi089.stdout:stopped 2023-10-26T06:46:28.323 INFO:teuthology.orchestra.run.smithi089.stdout:data_pools [3] 2023-10-26T06:46:28.323 INFO:teuthology.orchestra.run.smithi089.stdout:metadata_pool 2 2023-10-26T06:46:28.323 INFO:teuthology.orchestra.run.smithi089.stdout:inline_data disabled 2023-10-26T06:46:28.323 INFO:teuthology.orchestra.run.smithi089.stdout:balancer 2023-10-26T06:46:28.324 INFO:teuthology.orchestra.run.smithi089.stdout:bal_rank_mask -1 2023-10-26T06:46:28.324 INFO:teuthology.orchestra.run.smithi089.stdout:standby_count_wanted 0 2023-10-26T06:46:28.324 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b{0:24743} state up:active seq 3 addr [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.324 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.i{0:24703} state up:standby-replay seq 1 addr [v2:172.21.15.192:6838/2442523019,v1:172.21.15.192:6839/2442523019] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.324 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:28.325 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:28.325 INFO:teuthology.orchestra.run.smithi089.stdout:Standby daemons: 2023-10-26T06:46:28.325 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:28.325 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.j{-1:14625} state up:standby seq 1 addr [v2:172.21.15.89:6834/229884175,v1:172.21.15.89:6835/229884175] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.325 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.g{-1:14697} state up:standby seq 1 addr [v2:172.21.15.89:6838/1053435135,v1:172.21.15.89:6839/1053435135] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.325 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a{-1:14709} state up:standby seq 1 addr [v2:172.21.15.89:6840/3761987618,v1:172.21.15.89:6841/3761987618] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.326 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.l{-1:24616} state up:standby seq 1 addr [v2:172.21.15.192:6832/1842410141,v1:172.21.15.192:6833/1842410141] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.326 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.k{-1:24635} state up:standby seq 1 addr [v2:172.21.15.135:6832/3774068069,v1:172.21.15.135:6833/3774068069] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.326 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d{-1:24652} state up:standby seq 1 addr [v2:172.21.15.89:6836/104209914,v1:172.21.15.89:6837/104209914] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.326 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.e{-1:24671} state up:standby seq 1 addr [v2:172.21.15.135:6834/4067855863,v1:172.21.15.135:6835/4067855863] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.326 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.f{-1:24676} state up:standby seq 1 addr [v2:172.21.15.192:6834/2073006190,v1:172.21.15.192:6835/2073006190] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.327 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c{-1:24688} state up:standby seq 1 addr [v2:172.21.15.192:6836/1332439881,v1:172.21.15.192:6837/1332439881] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.327 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.h{-1:24692} state up:standby seq 1 addr [v2:172.21.15.135:6836/1484649325,v1:172.21.15.135:6837/1484649325] compat {c=[1],r=[1],i=[fff]}] 2023-10-26T06:46:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82780a1560 msgr2=0x7f82780a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82780a1560 0x7f82780a3a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8288101930 tx=0x7f827c009260 comp rx=0 tx=0).stop 2023-10-26T06:46:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82881057c0 msgr2=0x7f828819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82881057c0 0x7f828819d830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f828000b070 tx=0x7f82800078c0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 shutdown_connections 2023-10-26T06:46:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82780a1560 0x7f82780a3a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82881057c0 0x7f828819d830 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82881008a0 0x7f8288198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 --2- 172.21.15.89:0/2429032085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288101260 0x7f8288199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.316+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 >> 172.21.15.89:0/2429032085 conn(0x7f82880779e0 msgr2=0x7f828810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.317+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 shutdown_connections 2023-10-26T06:46:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:28.317+0000 7f828ebe6700 1 -- 172.21.15.89:0/2429032085 wait complete. 2023-10-26T06:46:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:28.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-26T06:46:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:29 smithi192 ceph-mon[131661]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:29 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2429032085' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-26T06:46:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:29 smithi135 ceph-mon[131403]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:29 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2429032085' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-26T06:46:29.532 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:29 smithi089 ceph-mon[95031]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:46:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:29 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2429032085' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-26T06:46:31.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 -- 172.21.15.89:0/533991598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec101260 msgr2=0x7f72ec10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 --2- 172.21.15.89:0/533991598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec101260 0x7f72ec10d650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b0a0 tx=0x7f72e400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 -- 172.21.15.89:0/533991598 shutdown_connections 2023-10-26T06:46:31.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 --2- 172.21.15.89:0/533991598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec101260 0x7f72ec10d650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 --2- 172.21.15.89:0/533991598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec1057c0 0x7f72ec105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 --2- 172.21.15.89:0/533991598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72ec1008a0 0x7f72ec100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 -- 172.21.15.89:0/533991598 >> 172.21.15.89:0/533991598 conn(0x7f72ec0779e0 msgr2=0x7f72ec077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:31.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 -- 172.21.15.89:0/533991598 shutdown_connections 2023-10-26T06:46:31.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.014+0000 7f72f2d37700 1 -- 172.21.15.89:0/533991598 wait complete. 2023-10-26T06:46:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.015+0000 7f72f2d37700 1 Processor -- start 2023-10-26T06:46:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.015+0000 7f72f2d37700 1 -- start start 2023-10-26T06:46:31.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f2d37700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec1008a0 0x7f72ec198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f2d37700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72ec101260 0x7f72ec199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f2d37700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec1057c0 0x7f72ec19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f2d37700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec19de40 con 0x7f72ec1057c0 2023-10-26T06:46:31.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f2d37700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec19dfb0 con 0x7f72ec1008a0 2023-10-26T06:46:31.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f2d37700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec19e170 con 0x7f72ec101260 2023-10-26T06:46:31.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec1057c0 0x7f72ec19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72ec101260 0x7f72ec199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72ebfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72ec101260 0x7f72ec199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:34856/0 (socket says 172.21.15.89:34856) 2023-10-26T06:46:31.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72ebfff700 1 -- 172.21.15.89:0/1408282758 learned_addr learned my addr 172.21.15.89:0/1408282758 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec1057c0 0x7f72ec19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45586/0 (socket says 172.21.15.89:45586) 2023-10-26T06:46:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f0ad3700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec1008a0 0x7f72ec198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 -- 172.21.15.89:0/1408282758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72ec101260 msgr2=0x7f72ec199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72ec101260 0x7f72ec199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 -- 172.21.15.89:0/1408282758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec1008a0 msgr2=0x7f72ec198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec1008a0 0x7f72ec198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 -- 172.21.15.89:0/1408282758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4009d00 con 0x7f72ec1057c0 2023-10-26T06:46:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f0ad3700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec1008a0 0x7f72ec198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.016+0000 7f72f12d4700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec1057c0 0x7f72ec19d830 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b070 tx=0x7f72e400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:31.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.017+0000 7f72e9ffb700 1 -- 172.21.15.89:0/1408282758 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400e040 con 0x7f72ec1057c0 2023-10-26T06:46:31.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.017+0000 7f72e9ffb700 1 -- 172.21.15.89:0/1408282758 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e401d480 con 0x7f72ec1057c0 2023-10-26T06:46:31.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.017+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec19e3f0 con 0x7f72ec1057c0 2023-10-26T06:46:31.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.017+0000 7f72e9ffb700 1 -- 172.21.15.89:0/1408282758 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40052a0 con 0x7f72ec1057c0 2023-10-26T06:46:31.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.017+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec1a5cb0 con 0x7f72ec1057c0 2023-10-26T06:46:31.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.018+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec101f30 con 0x7f72ec1057c0 2023-10-26T06:46:31.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.020+0000 7f72e9ffb700 1 -- 172.21.15.89:0/1408282758 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f72e401d880 con 0x7f72ec1057c0 2023-10-26T06:46:31.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.021+0000 7f72e9ffb700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72dc0a15e0 0x7f72dc0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.021+0000 7f72e9ffb700 1 -- 172.21.15.89:0/1408282758 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f72e40ce0b0 con 0x7f72ec1057c0 2023-10-26T06:46:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.021+0000 7f72f0ad3700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72dc0a15e0 0x7f72dc0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.022+0000 7f72f0ad3700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72dc0a15e0 0x7f72dc0a3aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72ec101930 tx=0x7f72e000b570 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:31.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.025+0000 7f72e9ffb700 1 -- 172.21.15.89:0/1408282758 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f72e4097500 con 0x7f72ec1057c0 2023-10-26T06:46:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.229+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f72ec04e4c0 con 0x7f72ec1057c0 2023-10-26T06:46:31.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.230+0000 7f72e9ffb700 1 -- 172.21.15.89:0/1408282758 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) v1 ==== 56+0+5683 (secure 0 0 0) 0x7f72e4024310 con 0x7f72ec1057c0 2023-10-26T06:46:31.231 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 86 2023-10-26T06:46:31.231 INFO:teuthology.orchestra.run.smithi089.stdout:fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T06:46:31.231 INFO:teuthology.orchestra.run.smithi089.stdout:created 2023-10-26T06:36:45.901565+0000 2023-10-26T06:46:31.232 INFO:teuthology.orchestra.run.smithi089.stdout:modified 2023-10-26T06:46:03.220293+0000 2023-10-26T06:46:31.232 INFO:teuthology.orchestra.run.smithi089.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-26T06:46:31.232 INFO:teuthology.orchestra.run.smithi089.stdout:crush_version 26 2023-10-26T06:46:31.232 INFO:teuthology.orchestra.run.smithi089.stdout:full_ratio 0.95 2023-10-26T06:46:31.232 INFO:teuthology.orchestra.run.smithi089.stdout:backfillfull_ratio 0.9 2023-10-26T06:46:31.233 INFO:teuthology.orchestra.run.smithi089.stdout:nearfull_ratio 0.85 2023-10-26T06:46:31.233 INFO:teuthology.orchestra.run.smithi089.stdout:require_min_compat_client luminous 2023-10-26T06:46:31.233 INFO:teuthology.orchestra.run.smithi089.stdout:min_compat_client luminous 2023-10-26T06:46:31.233 INFO:teuthology.orchestra.run.smithi089.stdout:require_osd_release reef 2023-10-26T06:46:31.233 INFO:teuthology.orchestra.run.smithi089.stdout:stretch_mode_enabled false 2023-10-26T06:46:31.234 INFO:teuthology.orchestra.run.smithi089.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-26T06:46:31.234 INFO:teuthology.orchestra.run.smithi089.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-26T06:46:31.234 INFO:teuthology.orchestra.run.smithi089.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-10-26T06:46:31.234 INFO:teuthology.orchestra.run.smithi089.stdout:max_osd 12 2023-10-26T06:46:31.234 INFO:teuthology.orchestra.run.smithi089.stdout:osd.0 up in weight 1 up_from 9 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6802/2463649651,v1:172.21.15.89:6803/2463649651] [v2:172.21.15.89:6804/2463649651,v1:172.21.15.89:6805/2463649651] exists,up 54028a78-a197-4bc3-b38b-f3e792bb42b5 2023-10-26T06:46:31.235 INFO:teuthology.orchestra.run.smithi089.stdout:osd.1 up in weight 1 up_from 14 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/77037565,v1:172.21.15.135:6801/77037565] [v2:172.21.15.135:6802/77037565,v1:172.21.15.135:6803/77037565] exists,up e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3 2023-10-26T06:46:31.235 INFO:teuthology.orchestra.run.smithi089.stdout:osd.2 up in weight 1 up_from 19 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/2357909960,v1:172.21.15.192:6801/2357909960] [v2:172.21.15.192:6802/2357909960,v1:172.21.15.192:6803/2357909960] exists,up f05df73a-080f-410a-84ad-50df3f2717f7 2023-10-26T06:46:31.235 INFO:teuthology.orchestra.run.smithi089.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6810/1365528390,v1:172.21.15.89:6811/1365528390] [v2:172.21.15.89:6812/1365528390,v1:172.21.15.89:6813/1365528390] exists,up 4b4e4079-2075-4d49-9a5b-457e210aa444 2023-10-26T06:46:31.235 INFO:teuthology.orchestra.run.smithi089.stdout:osd.4 up in weight 1 up_from 34 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/3181790377,v1:172.21.15.135:6809/3181790377] [v2:172.21.15.135:6810/3181790377,v1:172.21.15.135:6811/3181790377] exists,up 391e09b6-9475-4b8e-914f-34ed06f36e07 2023-10-26T06:46:31.235 INFO:teuthology.orchestra.run.smithi089.stdout:osd.5 up in weight 1 up_from 39 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/2176647372,v1:172.21.15.192:6809/2176647372] [v2:172.21.15.192:6810/2176647372,v1:172.21.15.192:6811/2176647372] exists,up 529984c4-0e50-4d78-b4ba-572a2f7df23c 2023-10-26T06:46:31.236 INFO:teuthology.orchestra.run.smithi089.stdout:osd.6 up in weight 1 up_from 47 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6818/2393166048,v1:172.21.15.89:6819/2393166048] [v2:172.21.15.89:6820/2393166048,v1:172.21.15.89:6821/2393166048] exists,up 0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984 2023-10-26T06:46:31.236 INFO:teuthology.orchestra.run.smithi089.stdout:osd.7 up in weight 1 up_from 52 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/1440944869,v1:172.21.15.135:6817/1440944869] [v2:172.21.15.135:6818/1440944869,v1:172.21.15.135:6819/1440944869] exists,up d9fc3b27-d54b-49e6-accb-8e02a7782a6b 2023-10-26T06:46:31.236 INFO:teuthology.orchestra.run.smithi089.stdout:osd.8 up in weight 1 up_from 59 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/87624122,v1:172.21.15.192:6817/87624122] [v2:172.21.15.192:6818/87624122,v1:172.21.15.192:6819/87624122] exists,up 809944cc-518e-49bc-b6fb-b56823f81148 2023-10-26T06:46:31.236 INFO:teuthology.orchestra.run.smithi089.stdout:osd.9 up in weight 1 up_from 67 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6826/781872938,v1:172.21.15.89:6827/781872938] [v2:172.21.15.89:6828/781872938,v1:172.21.15.89:6829/781872938] exists,up 0f38fdca-e054-402a-b88e-7abc1659d361 2023-10-26T06:46:31.236 INFO:teuthology.orchestra.run.smithi089.stdout:osd.10 up in weight 1 up_from 72 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6824/3703256177,v1:172.21.15.135:6825/3703256177] [v2:172.21.15.135:6826/3703256177,v1:172.21.15.135:6827/3703256177] exists,up bd567e81-805b-489f-8005-fb636fa97372 2023-10-26T06:46:31.236 INFO:teuthology.orchestra.run.smithi089.stdout:osd.11 up in weight 1 up_from 79 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/2369975497,v1:172.21.15.192:6825/2369975497] [v2:172.21.15.192:6826/2369975497,v1:172.21.15.192:6827/2369975497] exists,up 1551f98d-9cc1-45b1-9b7c-5a5b2f814669 2023-10-26T06:46:31.237 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.4 [2,11] 2023-10-26T06:46:31.237 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.10 [2,4] 2023-10-26T06:46:31.237 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.1a [8,1] 2023-10-26T06:46:31.237 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.1c [8,11] 2023-10-26T06:46:31.237 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.e [4,8] 2023-10-26T06:46:31.238 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.1a [5,6] 2023-10-26T06:46:31.238 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.22 [3,9] 2023-10-26T06:46:31.238 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.34 [4,9] 2023-10-26T06:46:31.238 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.36 [4,9] 2023-10-26T06:46:31.238 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6800/692487046 expires 2023-10-27T06:39:01.639977+0000 2023-10-26T06:46:31.238 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/1796653810 expires 2023-10-27T06:37:50.043554+0000 2023-10-26T06:46:31.239 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2906183572 expires 2023-10-27T06:37:50.043554+0000 2023-10-26T06:46:31.239 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6801/925229622 expires 2023-10-27T06:37:50.043554+0000 2023-10-26T06:46:31.239 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/4177523596 expires 2023-10-27T06:39:01.639977+0000 2023-10-26T06:46:31.239 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6801/2105103929 expires 2023-10-27T06:37:23.538291+0000 2023-10-26T06:46:31.239 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/746092036 expires 2023-10-27T06:39:01.639977+0000 2023-10-26T06:46:31.240 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6800/2105103929 expires 2023-10-27T06:37:23.538291+0000 2023-10-26T06:46:31.240 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/1900326324 expires 2023-10-27T06:37:23.538291+0000 2023-10-26T06:46:31.240 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6801/692487046 expires 2023-10-27T06:39:01.639977+0000 2023-10-26T06:46:31.240 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2757333567 expires 2023-10-27T06:37:50.043554+0000 2023-10-26T06:46:31.240 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/516070855 expires 2023-10-27T06:37:23.538291+0000 2023-10-26T06:46:31.240 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2016135884 expires 2023-10-27T06:39:01.639977+0000 2023-10-26T06:46:31.241 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2689379544 expires 2023-10-27T06:37:23.538291+0000 2023-10-26T06:46:31.241 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6800/925229622 expires 2023-10-27T06:37:50.043554+0000 2023-10-26T06:46:31.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72dc0a15e0 msgr2=0x7f72dc0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72dc0a15e0 0x7f72dc0a3aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72ec101930 tx=0x7f72e000b570 comp rx=0 tx=0).stop 2023-10-26T06:46:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec1057c0 msgr2=0x7f72ec19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec1057c0 0x7f72ec19d830 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b070 tx=0x7f72e400ce70 comp rx=0 tx=0).stop 2023-10-26T06:46:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 shutdown_connections 2023-10-26T06:46:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72dc0a15e0 0x7f72dc0a3aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f72ec1057c0 0x7f72ec19d830 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec1008a0 0x7f72ec198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 --2- 172.21.15.89:0/1408282758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72ec101260 0x7f72ec199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 >> 172.21.15.89:0/1408282758 conn(0x7f72ec0779e0 msgr2=0x7f72ec10ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 shutdown_connections 2023-10-26T06:46:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.232+0000 7f72f2d37700 1 -- 172.21.15.89:0/1408282758 wait complete. 2023-10-26T06:46:31.368 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:31 smithi089 ceph-mon[95031]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:31 smithi192 ceph-mon[131661]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:31.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:31 smithi135 ceph-mon[131403]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:31.710 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-26T06:46:31.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:31.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 -- 172.21.15.89:0/879310667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f81006a0 msgr2=0x7f91f8100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/879310667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f81006a0 0x7f91f8100b20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009990 tx=0x7f91e4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 -- 172.21.15.89:0/879310667 shutdown_connections 2023-10-26T06:46:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/879310667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f8101060 0x7f91f810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/879310667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f81066b0 0x7f91f8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/879310667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f81006a0 0x7f91f8100b20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 -- 172.21.15.89:0/879310667 >> 172.21.15.89:0/879310667 conn(0x7f91f80fc170 msgr2=0x7f91f80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:31.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 -- 172.21.15.89:0/879310667 shutdown_connections 2023-10-26T06:46:31.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.950+0000 7f91ff1ff700 1 -- 172.21.15.89:0/879310667 wait complete. 2023-10-26T06:46:31.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 Processor -- start 2023-10-26T06:46:31.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 -- start start 2023-10-26T06:46:31.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f81006a0 0x7f91f8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f8101060 0x7f91f8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f81066b0 0x7f91f8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f81a4ef0 con 0x7f91f8101060 2023-10-26T06:46:31.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f81a5060 con 0x7f91f81066b0 2023-10-26T06:46:31.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.951+0000 7f91ff1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f81a5220 con 0x7f91f81006a0 2023-10-26T06:46:31.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f8101060 0x7f91f8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91fcf9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f81006a0 0x7f91f8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f8101060 0x7f91f8073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45602/0 (socket says 172.21.15.89:45602) 2023-10-26T06:46:31.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91fcf9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f81006a0 0x7f91f8072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:34872/0 (socket says 172.21.15.89:34872) 2023-10-26T06:46:31.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 -- 172.21.15.89:0/627089600 learned_addr learned my addr 172.21.15.89:0/627089600 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:31.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91fd79c700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f81066b0 0x7f91f8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 -- 172.21.15.89:0/627089600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f81006a0 msgr2=0x7f91f8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f81006a0 0x7f91f8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 -- 172.21.15.89:0/627089600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f81066b0 msgr2=0x7f91f8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:31.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f81066b0 0x7f91f8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:31.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 -- 172.21.15.89:0/627089600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40095f0 con 0x7f91f8101060 2023-10-26T06:46:31.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f7fff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f8101060 0x7f91f8073030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000c00 tx=0x7f91e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:31.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e401d070 con 0x7f91f8101060 2023-10-26T06:46:31.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e4022470 con 0x7f91f8101060 2023-10-26T06:46:31.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.952+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4017650 con 0x7f91f8101060 2023-10-26T06:46:31.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.953+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f81a54a0 con 0x7f91f8101060 2023-10-26T06:46:31.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.953+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f81a5980 con 0x7f91f8101060 2023-10-26T06:46:31.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.954+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f8108c30 con 0x7f91f8101060 2023-10-26T06:46:31.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.956+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) v1 ==== 136071+0+0 (secure 0 0 0) 0x7f91e40225e0 con 0x7f91f8101060 2023-10-26T06:46:31.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.958+0000 7f91f5ffb700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91e80a15e0 0x7f91e80a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:31.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.958+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f91e40ce940 con 0x7f91f8101060 2023-10-26T06:46:31.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.958+0000 7f91fcf9b700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91e80a15e0 0x7f91e80a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:31.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.959+0000 7f91fcf9b700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91e80a15e0 0x7f91e80a3aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009bc0 tx=0x7f91ec009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:31.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:31.960+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91e4097d90 con 0x7f91f8101060 2023-10-26T06:46:32.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.109+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f91e4098160 con 0x7f91f8101060 2023-10-26T06:46:32.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.170+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91f8073fd0 con 0x7f91f8101060 2023-10-26T06:46:32.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.172+0000 7f91f5ffb700 1 -- 172.21.15.89:0/627089600 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7f91e408b560 con 0x7f91f8101060 2023-10-26T06:46:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:32.174 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:32.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91e80a15e0 msgr2=0x7f91e80a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91e80a15e0 0x7f91e80a3aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009bc0 tx=0x7f91ec009260 comp rx=0 tx=0).stop 2023-10-26T06:46:32.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f8101060 msgr2=0x7f91f8073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f8101060 0x7f91f8073030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f91e4000c00 tx=0x7f91e4005740 comp rx=0 tx=0).stop 2023-10-26T06:46:32.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 shutdown_connections 2023-10-26T06:46:32.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91e80a15e0 0x7f91e80a3aa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f81066b0 0x7f91f8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f81006a0 0x7f91f8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 --2- 172.21.15.89:0/627089600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91f8101060 0x7f91f8073030 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 >> 172.21.15.89:0/627089600 conn(0x7f91f80fc170 msgr2=0x7f91f80fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 shutdown_connections 2023-10-26T06:46:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.174+0000 7f91ff1ff700 1 -- 172.21.15.89:0/627089600 wait complete. 2023-10-26T06:46:32.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:46:32.195 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:46:32.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:32 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1408282758' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-26T06:46:32.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 -- 172.21.15.89:0/385733623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 msgr2=0x7f7dd0100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/385733623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd0100ba0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40098f0 tx=0x7f7dc4009c00 comp rx=0 tx=0).stop 2023-10-26T06:46:32.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 -- 172.21.15.89:0/385733623 shutdown_connections 2023-10-26T06:46:32.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/385733623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd01010e0 0x7f7dd010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/385733623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0106730 0x7f7dd0106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/385733623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd0100ba0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 -- 172.21.15.89:0/385733623 >> 172.21.15.89:0/385733623 conn(0x7f7dd00fc1b0 msgr2=0x7f7dd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 -- 172.21.15.89:0/385733623 shutdown_connections 2023-10-26T06:46:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.396+0000 7f7dd55ff700 1 -- 172.21.15.89:0/385733623 wait complete. 2023-10-26T06:46:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.397+0000 7f7dd55ff700 1 Processor -- start 2023-10-26T06:46:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.397+0000 7f7dd55ff700 1 -- start start 2023-10-26T06:46:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dd55ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0100720 0x7f7dd0198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dd55ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dd55ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd0106730 0x7f7dd019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dd55ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019dcf0 con 0x7f7dd01010e0 2023-10-26T06:46:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dd55ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019de60 con 0x7f7dd0106730 2023-10-26T06:46:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dd55ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019e020 con 0x7f7dd0100720 2023-10-26T06:46:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dce7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dce7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd0199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45620/0 (socket says 172.21.15.89:45620) 2023-10-26T06:46:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dce7fc700 1 -- 172.21.15.89:0/4245922716 learned_addr learned my addr 172.21.15.89:0/4245922716 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dceffd700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0100720 0x7f7dd0198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dcf7fe700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd0106730 0x7f7dd019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dce7fc700 1 -- 172.21.15.89:0/4245922716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0100720 msgr2=0x7f7dd0198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dce7fc700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0100720 0x7f7dd0198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dce7fc700 1 -- 172.21.15.89:0/4245922716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd0106730 msgr2=0x7f7dd019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.398+0000 7f7dce7fc700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd0106730 0x7f7dd019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.399+0000 7f7dce7fc700 1 -- 172.21.15.89:0/4245922716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4009520 con 0x7f7dd01010e0 2023-10-26T06:46:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.399+0000 7f7dce7fc700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd0199360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005950 tx=0x7f7dc400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.399+0000 7f7db7fff700 1 -- 172.21.15.89:0/4245922716 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401c070 con 0x7f7dd01010e0 2023-10-26T06:46:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.399+0000 7f7db7fff700 1 -- 172.21.15.89:0/4245922716 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc400bb20 con 0x7f7dd01010e0 2023-10-26T06:46:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.399+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd019e2a0 con 0x7f7dd01010e0 2023-10-26T06:46:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.399+0000 7f7db7fff700 1 -- 172.21.15.89:0/4245922716 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40038e0 con 0x7f7dd01010e0 2023-10-26T06:46:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.399+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd01a5b40 con 0x7f7dd01010e0 2023-10-26T06:46:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.400+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd0101e60 con 0x7f7dd01010e0 2023-10-26T06:46:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.402+0000 7f7db7fff700 1 -- 172.21.15.89:0/4245922716 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f7dc4020030 con 0x7f7dd01010e0 2023-10-26T06:46:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.403+0000 7f7db7fff700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a14d0 0x7f7db80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.403+0000 7f7db7fff700 1 -- 172.21.15.89:0/4245922716 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7dc40cdfc0 con 0x7f7dd01010e0 2023-10-26T06:46:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.404+0000 7f7dceffd700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a14d0 0x7f7db80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.404+0000 7f7dceffd700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a14d0 0x7f7db80a3990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009be0 tx=0x7f7dc0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:32.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.406+0000 7f7db7fff700 1 -- 172.21.15.89:0/4245922716 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7dc4097570 con 0x7f7dd01010e0 2023-10-26T06:46:32.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:32 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1408282758' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-26T06:46:32.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:32 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1408282758' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-26T06:46:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.608+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7dd019a0d0 con 0x7f7dd01010e0 2023-10-26T06:46:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.608+0000 7f7db7fff700 1 -- 172.21.15.89:0/4245922716 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dc409ab90 con 0x7f7dd01010e0 2023-10-26T06:46:32.610 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:46:32.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.610+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a14d0 msgr2=0x7f7db80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a14d0 0x7f7db80a3990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009be0 tx=0x7f7dc0009260 comp rx=0 tx=0).stop 2023-10-26T06:46:32.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 msgr2=0x7f7dd0199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd0199360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4005950 tx=0x7f7dc400fa00 comp rx=0 tx=0).stop 2023-10-26T06:46:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 shutdown_connections 2023-10-26T06:46:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a14d0 0x7f7db80a3990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd0106730 0x7f7dd019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0100720 0x7f7dd0198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 --2- 172.21.15.89:0/4245922716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd0199360 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 >> 172.21.15.89:0/4245922716 conn(0x7f7dd00fc1b0 msgr2=0x7f7dd00fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 shutdown_connections 2023-10-26T06:46:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.611+0000 7f7dd55ff700 1 -- 172.21.15.89:0/4245922716 wait complete. 2023-10-26T06:46:32.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:46:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 -- 172.21.15.89:0/1601066222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258106750 msgr2=0x7f3258106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 --2- 172.21.15.89:0/1601066222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258106750 0x7f3258106b30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3248009990 tx=0x7f3248009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 -- 172.21.15.89:0/1601066222 shutdown_connections 2023-10-26T06:46:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 --2- 172.21.15.89:0/1601066222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258106750 0x7f3258106b30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 --2- 172.21.15.89:0/1601066222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3258101100 0x7f325810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 --2- 172.21.15.89:0/1601066222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3258100740 0x7f3258100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 -- 172.21.15.89:0/1601066222 >> 172.21.15.89:0/1601066222 conn(0x7f32580fc230 msgr2=0x7f32580fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 -- 172.21.15.89:0/1601066222 shutdown_connections 2023-10-26T06:46:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.831+0000 7f325d9ff700 1 -- 172.21.15.89:0/1601066222 wait complete. 2023-10-26T06:46:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 Processor -- start 2023-10-26T06:46:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 -- start start 2023-10-26T06:46:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258100740 0x7f3258198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3258101100 0x7f32581992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3258106750 0x7f325819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325819dcb0 con 0x7f3258100740 2023-10-26T06:46:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325819de20 con 0x7f3258101100 2023-10-26T06:46:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.832+0000 7f325d9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325819dfe0 con 0x7f3258106750 2023-10-26T06:46:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258100740 0x7f3258198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258100740 0x7f3258198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45646/0 (socket says 172.21.15.89:45646) 2023-10-26T06:46:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f32567fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3258101100 0x7f32581992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 -- 172.21.15.89:0/4190642738 learned_addr learned my addr 172.21.15.89:0/4190642738 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f32577fe700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3258106750 0x7f325819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 -- 172.21.15.89:0/4190642738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3258106750 msgr2=0x7f325819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3258106750 0x7f325819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 -- 172.21.15.89:0/4190642738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3258101100 msgr2=0x7f32581992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3258101100 0x7f32581992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 -- 172.21.15.89:0/4190642738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480095f0 con 0x7f3258100740 2023-10-26T06:46:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f32567fc700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3258101100 0x7f32581992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f3256ffd700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258100740 0x7f3258198d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3248000c00 tx=0x7f3248005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f325c9fd700 1 -- 172.21.15.89:0/4190642738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324801d070 con 0x7f3258100740 2023-10-26T06:46:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.833+0000 7f325c9fd700 1 -- 172.21.15.89:0/4190642738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3248022470 con 0x7f3258100740 2023-10-26T06:46:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.834+0000 7f325c9fd700 1 -- 172.21.15.89:0/4190642738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248017650 con 0x7f3258100740 2023-10-26T06:46:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.834+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325819e260 con 0x7f3258100740 2023-10-26T06:46:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.834+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32581a5b00 con 0x7f3258100740 2023-10-26T06:46:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.835+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3258101e80 con 0x7f3258100740 2023-10-26T06:46:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.836+0000 7f325c9fd700 1 -- 172.21.15.89:0/4190642738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f32480225e0 con 0x7f3258100740 2023-10-26T06:46:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.837+0000 7f325c9fd700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32400a1550 0x7f32400a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.838+0000 7f325c9fd700 1 -- 172.21.15.89:0/4190642738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f32480cdfe0 con 0x7f3258100740 2023-10-26T06:46:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.838+0000 7f32567fc700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32400a1550 0x7f32400a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.838+0000 7f32567fc700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32400a1550 0x7f32400a3a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f325819a2e0 tx=0x7f324c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:32.841+0000 7f325c9fd700 1 -- 172.21.15.89:0/4190642738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3248097710 con 0x7f3258100740 2023-10-26T06:46:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.041+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f325819a090 con 0x7f3258100740 2023-10-26T06:46:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.041+0000 7f325c9fd700 1 -- 172.21.15.89:0/4190642738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f324809ad30 con 0x7f3258100740 2023-10-26T06:46:33.043 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:46:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.043+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32400a1550 msgr2=0x7f32400a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.043+0000 7f325d9ff700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32400a1550 0x7f32400a3a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f325819a2e0 tx=0x7f324c009260 comp rx=0 tx=0).stop 2023-10-26T06:46:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.043+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258100740 msgr2=0x7f3258198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.043+0000 7f325d9ff700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258100740 0x7f3258198d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3248000c00 tx=0x7f3248005740 comp rx=0 tx=0).stop 2023-10-26T06:46:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 shutdown_connections 2023-10-26T06:46:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f32400a1550 0x7f32400a3a10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3258100740 0x7f3258198d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3258106750 0x7f325819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 --2- 172.21.15.89:0/4190642738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3258101100 0x7f32581992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 >> 172.21.15.89:0/4190642738 conn(0x7f32580fc230 msgr2=0x7f325810be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 shutdown_connections 2023-10-26T06:46:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.044+0000 7f325d9ff700 1 -- 172.21.15.89:0/4190642738 wait complete. 2023-10-26T06:46:33.064 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:46:33.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 -- 172.21.15.89:0/949932311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa8106730 msgr2=0x7f2aa8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/949932311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa8106730 0x7f2aa8106b10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009990 tx=0x7f2a98009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 -- 172.21.15.89:0/949932311 shutdown_connections 2023-10-26T06:46:33.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/949932311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa8106730 0x7f2aa8106b10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/949932311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2aa81010e0 0x7f2aa810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/949932311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa8100720 0x7f2aa8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 -- 172.21.15.89:0/949932311 >> 172.21.15.89:0/949932311 conn(0x7f2aa80fc1b0 msgr2=0x7f2aa80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:33.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 -- 172.21.15.89:0/949932311 shutdown_connections 2023-10-26T06:46:33.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.264+0000 7f2aae7ff700 1 -- 172.21.15.89:0/949932311 wait complete. 2023-10-26T06:46:33.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.265+0000 7f2aae7ff700 1 Processor -- start 2023-10-26T06:46:33.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.266+0000 7f2aae7ff700 1 -- start start 2023-10-26T06:46:33.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.266+0000 7f2aae7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2aa8100720 0x7f2aa8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.266+0000 7f2aae7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa81010e0 0x7f2aa8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.266+0000 7f2aae7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa8106730 0x7f2aa8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.266+0000 7f2aae7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa81a4f30 con 0x7f2aa81010e0 2023-10-26T06:46:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.266+0000 7f2aae7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa81a50a0 con 0x7f2aa8100720 2023-10-26T06:46:33.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.266+0000 7f2aae7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa81a5260 con 0x7f2aa8106730 2023-10-26T06:46:33.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa81010e0 0x7f2aa8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa81010e0 0x7f2aa8073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45654/0 (socket says 172.21.15.89:45654) 2023-10-26T06:46:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 -- 172.21.15.89:0/2969960380 learned_addr learned my addr 172.21.15.89:0/2969960380 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa7fff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2aa8100720 0x7f2aa8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aacd9c700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa8106730 0x7f2aa8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 -- 172.21.15.89:0/2969960380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa8106730 msgr2=0x7f2aa8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa8106730 0x7f2aa8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 -- 172.21.15.89:0/2969960380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2aa8100720 msgr2=0x7f2aa8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2aa8100720 0x7f2aa8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 -- 172.21.15.89:0/2969960380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a980095f0 con 0x7f2aa81010e0 2023-10-26T06:46:33.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa77fe700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa81010e0 0x7f2aa8073030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00d980 tx=0x7f2a9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:33.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa57fa700 1 -- 172.21.15.89:0/2969960380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c004220 con 0x7f2aa81010e0 2023-10-26T06:46:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa81a5540 con 0x7f2aa81010e0 2023-10-26T06:46:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa57fa700 1 -- 172.21.15.89:0/2969960380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a9c007640 con 0x7f2aa81010e0 2023-10-26T06:46:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.267+0000 7f2aa57fa700 1 -- 172.21.15.89:0/2969960380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c005610 con 0x7f2aa81010e0 2023-10-26T06:46:33.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.268+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa81a5a50 con 0x7f2aa81010e0 2023-10-26T06:46:33.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.269+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa8108d20 con 0x7f2aa81010e0 2023-10-26T06:46:33.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.271+0000 7f2aa57fa700 1 -- 172.21.15.89:0/2969960380 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f2a9c004750 con 0x7f2aa81010e0 2023-10-26T06:46:33.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.272+0000 7f2aa57fa700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a900a14d0 0x7f2a900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.272+0000 7f2aa57fa700 1 -- 172.21.15.89:0/2969960380 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2a9c0ccd60 con 0x7f2aa81010e0 2023-10-26T06:46:33.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.272+0000 7f2aa7fff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a900a14d0 0x7f2a900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.273+0000 7f2aa7fff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a900a14d0 0x7f2a900a3990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006010 tx=0x7f2a9800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:33.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.276+0000 7f2aa57fa700 1 -- 172.21.15.89:0/2969960380 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2a9c096280 con 0x7f2aa81010e0 2023-10-26T06:46:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:33 smithi192 ceph-mon[131661]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:33 smithi192 ceph-mon[131661]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-26T06:46:33.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:33 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/627089600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:33.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:33 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4245922716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:33.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:33 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4190642738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:33.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:33 smithi135 ceph-mon[131403]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:33.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:33 smithi135 ceph-mon[131403]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-26T06:46:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:33 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/627089600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:33 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4245922716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:33.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:33 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4190642738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:33.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.477+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2aa8073fd0 con 0x7f2aa81010e0 2023-10-26T06:46:33.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.477+0000 7f2aa57fa700 1 -- 172.21.15.89:0/2969960380 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a9c016020 con 0x7f2aa81010e0 2023-10-26T06:46:33.479 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:46:33.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a900a14d0 msgr2=0x7f2a900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a900a14d0 0x7f2a900a3990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006010 tx=0x7f2a9800b540 comp rx=0 tx=0).stop 2023-10-26T06:46:33.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa81010e0 msgr2=0x7f2aa8073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa81010e0 0x7f2aa8073030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00d980 tx=0x7f2a9c00dc90 comp rx=0 tx=0).stop 2023-10-26T06:46:33.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 shutdown_connections 2023-10-26T06:46:33.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a900a14d0 0x7f2a900a3990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa8106730 0x7f2aa8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2aa8100720 0x7f2aa8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 --2- 172.21.15.89:0/2969960380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2aa81010e0 0x7f2aa8073030 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 >> 172.21.15.89:0/2969960380 conn(0x7f2aa80fc1b0 msgr2=0x7f2aa80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:33.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 shutdown_connections 2023-10-26T06:46:33.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.480+0000 7f2aae7ff700 1 -- 172.21.15.89:0/2969960380 wait complete. 2023-10-26T06:46:33.497 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:33 smithi089 ceph-mon[95031]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:46:33.497 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:33 smithi089 ceph-mon[95031]: mgrmap e22: x(active, since 7m), standbys: z, y 2023-10-26T06:46:33.498 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:33 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/627089600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:33.498 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:33 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4245922716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:33.498 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:33 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4190642738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:33.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:46:33.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 -- 172.21.15.89:0/2189596759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc101060 msgr2=0x7fd5cc10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/2189596759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc101060 0x7fd5cc10d570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b0a0 tx=0x7fd5c400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 -- 172.21.15.89:0/2189596759 shutdown_connections 2023-10-26T06:46:33.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/2189596759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc101060 0x7fd5cc10d570 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/2189596759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1066b0 0x7fd5cc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/2189596759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5cc1006a0 0x7fd5cc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 -- 172.21.15.89:0/2189596759 >> 172.21.15.89:0/2189596759 conn(0x7fd5cc0fc1d0 msgr2=0x7fd5cc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 -- 172.21.15.89:0/2189596759 shutdown_connections 2023-10-26T06:46:33.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.697+0000 7fd5d2dff700 1 -- 172.21.15.89:0/2189596759 wait complete. 2023-10-26T06:46:33.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 Processor -- start 2023-10-26T06:46:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 -- start start 2023-10-26T06:46:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1006a0 0x7fd5cc072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5cc101060 0x7fd5cc073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc1066b0 0x7fd5cc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc1a4ee0 con 0x7fd5cc1066b0 2023-10-26T06:46:33.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc1a5050 con 0x7fd5cc1006a0 2023-10-26T06:46:33.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.698+0000 7fd5d2dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc1a5210 con 0x7fd5cc101060 2023-10-26T06:46:33.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d139c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc1066b0 0x7fd5cc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d0b9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1006a0 0x7fd5cc072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d139c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc1066b0 0x7fd5cc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45668/0 (socket says 172.21.15.89:45668) 2023-10-26T06:46:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d139c700 1 -- 172.21.15.89:0/536306086 learned_addr learned my addr 172.21.15.89:0/536306086 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5cbfff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5cc101060 0x7fd5cc073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d0b9b700 1 -- 172.21.15.89:0/536306086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5cc101060 msgr2=0x7fd5cc073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d0b9b700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5cc101060 0x7fd5cc073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d0b9b700 1 -- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc1066b0 msgr2=0x7fd5cc073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d0b9b700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc1066b0 0x7fd5cc073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d0b9b700 1 -- 172.21.15.89:0/536306086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4009d00 con 0x7fd5cc1006a0 2023-10-26T06:46:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d139c700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc1066b0 0x7fd5cc073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5d0b9b700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1006a0 0x7fd5cc072af0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e9f0 tx=0x7fd5c000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:33.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.699+0000 7fd5c9ffb700 1 -- 172.21.15.89:0/536306086 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000c9c0 con 0x7fd5cc1006a0 2023-10-26T06:46:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.700+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5cc1a54f0 con 0x7fd5cc1006a0 2023-10-26T06:46:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.700+0000 7fd5c9ffb700 1 -- 172.21.15.89:0/536306086 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5c0010430 con 0x7fd5cc1006a0 2023-10-26T06:46:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.700+0000 7fd5c9ffb700 1 -- 172.21.15.89:0/536306086 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0018680 con 0x7fd5cc1006a0 2023-10-26T06:46:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.700+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc1a5a50 con 0x7fd5cc1006a0 2023-10-26T06:46:33.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.700+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5cc108cc0 con 0x7fd5cc1006a0 2023-10-26T06:46:33.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.703+0000 7fd5c9ffb700 1 -- 172.21.15.89:0/536306086 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd5c0010860 con 0x7fd5cc1006a0 2023-10-26T06:46:33.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.704+0000 7fd5c9ffb700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a14d0 0x7fd5bc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:33.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.704+0000 7fd5c9ffb700 1 -- 172.21.15.89:0/536306086 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd5c0014070 con 0x7fd5cc1006a0 2023-10-26T06:46:33.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.704+0000 7fd5cbfff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a14d0 0x7fd5bc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:33.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.705+0000 7fd5cbfff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a14d0 0x7fd5bc0a3990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc074220 tx=0x7fd5b8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:33.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.706+0000 7fd5c9ffb700 1 -- 172.21.15.89:0/536306086 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5c0096610 con 0x7fd5cc1006a0 2023-10-26T06:46:33.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.954+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5cc101de0 con 0x7fd5cc1006a0 2023-10-26T06:46:33.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.955+0000 7fd5c9ffb700 1 -- 172.21.15.89:0/536306086 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fd5c0099c30 con 0x7fd5cc1006a0 2023-10-26T06:46:33.956 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:33.956 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:46:33.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a14d0 msgr2=0x7fd5bc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a14d0 0x7fd5bc0a3990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc074220 tx=0x7fd5b8009260 comp rx=0 tx=0).stop 2023-10-26T06:46:33.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1006a0 msgr2=0x7fd5cc072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:33.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1006a0 0x7fd5cc072af0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e9f0 tx=0x7fd5c000ed00 comp rx=0 tx=0).stop 2023-10-26T06:46:33.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 shutdown_connections 2023-10-26T06:46:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5bc0a14d0 0x7fd5bc0a3990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5cc1066b0 0x7fd5cc073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5cc1006a0 0x7fd5cc072af0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 --2- 172.21.15.89:0/536306086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5cc101060 0x7fd5cc073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:33.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.957+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 >> 172.21.15.89:0/536306086 conn(0x7fd5cc0fc1d0 msgr2=0x7fd5cc0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.958+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 shutdown_connections 2023-10-26T06:46:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:33.958+0000 7fd5d2dff700 1 -- 172.21.15.89:0/536306086 wait complete. 2023-10-26T06:46:33.961 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:33.978 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:34.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 -- 172.21.15.89:0/3484425125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 msgr2=0x7f27f8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 --2- 172.21.15.89:0/3484425125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f8106a90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009990 tx=0x7f27e8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 -- 172.21.15.89:0/3484425125 shutdown_connections 2023-10-26T06:46:34.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 --2- 172.21.15.89:0/3484425125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f8106a90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 --2- 172.21.15.89:0/3484425125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27f8101060 0x7f27f810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 --2- 172.21.15.89:0/3484425125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f81006a0 0x7f27f8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 -- 172.21.15.89:0/3484425125 >> 172.21.15.89:0/3484425125 conn(0x7f27f80fc1d0 msgr2=0x7f27f80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 -- 172.21.15.89:0/3484425125 shutdown_connections 2023-10-26T06:46:34.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.176+0000 7f27fddff700 1 -- 172.21.15.89:0/3484425125 wait complete. 2023-10-26T06:46:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.177+0000 7f27fddff700 1 Processor -- start 2023-10-26T06:46:34.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.177+0000 7f27fddff700 1 -- start start 2023-10-26T06:46:34.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27fddff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27f81006a0 0x7f27f819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27fddff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f8101060 0x7f27f819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27fddff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f81969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27fddff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8197010 con 0x7f27f81066b0 2023-10-26T06:46:34.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27fddff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8197180 con 0x7f27f81006a0 2023-10-26T06:46:34.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27fddff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8197340 con 0x7f27f8101060 2023-10-26T06:46:34.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f81969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f81969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45676/0 (socket says 172.21.15.89:45676) 2023-10-26T06:46:34.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 -- 172.21.15.89:0/2470484036 learned_addr learned my addr 172.21.15.89:0/2470484036 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:34.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f6ffd700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f8101060 0x7f27f819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f77fe700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27f81006a0 0x7f27f819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 -- 172.21.15.89:0/2470484036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f8101060 msgr2=0x7f27f819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f8101060 0x7f27f819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 -- 172.21.15.89:0/2470484036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27f81006a0 msgr2=0x7f27f819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27f81006a0 0x7f27f819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.178+0000 7f27f7fff700 1 -- 172.21.15.89:0/2470484036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80095f0 con 0x7f27f81066b0 2023-10-26T06:46:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.179+0000 7f27f7fff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f81969b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f27f0007ee0 tx=0x7f27f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.179+0000 7f27f4ff9700 1 -- 172.21.15.89:0/2470484036 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000dd20 con 0x7f27f81066b0 2023-10-26T06:46:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.179+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f8197620 con 0x7f27f81066b0 2023-10-26T06:46:34.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.179+0000 7f27f4ff9700 1 -- 172.21.15.89:0/2470484036 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f000f040 con 0x7f27f81066b0 2023-10-26T06:46:34.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.179+0000 7f27f4ff9700 1 -- 172.21.15.89:0/2470484036 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f00048a0 con 0x7f27f81066b0 2023-10-26T06:46:34.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.179+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f8072c00 con 0x7f27f81066b0 2023-10-26T06:46:34.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.180+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f8108ca0 con 0x7f27f81066b0 2023-10-26T06:46:34.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.182+0000 7f27f4ff9700 1 -- 172.21.15.89:0/2470484036 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f27f0003a00 con 0x7f27f81066b0 2023-10-26T06:46:34.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.183+0000 7f27f4ff9700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f27e00a14d0 0x7f27e00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.183+0000 7f27f4ff9700 1 -- 172.21.15.89:0/2470484036 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f27f00cc630 con 0x7f27f81066b0 2023-10-26T06:46:34.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.183+0000 7f27f77fe700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f27e00a14d0 0x7f27e00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.184+0000 7f27f77fe700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f27e00a14d0 0x7f27e00a3990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f27e800b2e0 tx=0x7f27e8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:34.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.187+0000 7f27f4ff9700 1 -- 172.21.15.89:0/2470484036 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f27f004f100 con 0x7f27f81066b0 2023-10-26T06:46:34.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:34 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2969960380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:34.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:34 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/536306086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:34.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.390+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27f8197e70 con 0x7f27f81066b0 2023-10-26T06:46:34.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.392+0000 7f27f4ff9700 1 -- 172.21.15.89:0/2470484036 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7f27f0095bb0 con 0x7f27f81066b0 2023-10-26T06:46:34.393 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:34.394 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:34.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:34 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2969960380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:34.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:34 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/536306086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:34.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f27e00a14d0 msgr2=0x7f27e00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f27e00a14d0 0x7f27e00a3990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f27e800b2e0 tx=0x7f27e8005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 msgr2=0x7f27f81969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f81969b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f27f0007ee0 tx=0x7f27f000d3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 shutdown_connections 2023-10-26T06:46:34.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f27e00a14d0 0x7f27e00a3990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27f81066b0 0x7f27f81969b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27f81006a0 0x7f27f819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 --2- 172.21.15.89:0/2470484036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f8101060 0x7f27f819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.394+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 >> 172.21.15.89:0/2470484036 conn(0x7f27f80fc1d0 msgr2=0x7f27f80fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:34.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.395+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 shutdown_connections 2023-10-26T06:46:34.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.395+0000 7f27fddff700 1 -- 172.21.15.89:0/2470484036 wait complete. 2023-10-26T06:46:34.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:46:34.416 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:46:34.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:34 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2969960380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:34.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:34 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/536306086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 -- 172.21.15.89:0/2598253860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c100710 msgr2=0x7fe56c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 --2- 172.21.15.89:0/2598253860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c100710 0x7fe56c100b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe560009990 tx=0x7fe560009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 -- 172.21.15.89:0/2598253860 shutdown_connections 2023-10-26T06:46:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 --2- 172.21.15.89:0/2598253860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe56c1010d0 0x7fe56c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 --2- 172.21.15.89:0/2598253860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c106720 0x7fe56c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 --2- 172.21.15.89:0/2598253860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c100710 0x7fe56c100b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 -- 172.21.15.89:0/2598253860 >> 172.21.15.89:0/2598253860 conn(0x7fe56c0fc200 msgr2=0x7fe56c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:34.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 -- 172.21.15.89:0/2598253860 shutdown_connections 2023-10-26T06:46:34.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.616+0000 7fe5721ff700 1 -- 172.21.15.89:0/2598253860 wait complete. 2023-10-26T06:46:34.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.617+0000 7fe5721ff700 1 Processor -- start 2023-10-26T06:46:34.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe5721ff700 1 -- start start 2023-10-26T06:46:34.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe5721ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe56c100710 0x7fe56c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe5721ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c1010d0 0x7fe56c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe5721ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c106720 0x7fe56c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe5721ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c199250 con 0x7fe56c106720 2023-10-26T06:46:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe5721ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c1993c0 con 0x7fe56c100710 2023-10-26T06:46:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe5721ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c199580 con 0x7fe56c1010d0 2023-10-26T06:46:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe56bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c106720 0x7fe56c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe56b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe56c100710 0x7fe56c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe56bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c106720 0x7fe56c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45688/0 (socket says 172.21.15.89:45688) 2023-10-26T06:46:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe56bfff700 1 -- 172.21.15.89:0/3261121676 learned_addr learned my addr 172.21.15.89:0/3261121676 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.618+0000 7fe56affd700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c1010d0 0x7fe56c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe56bfff700 1 -- 172.21.15.89:0/3261121676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c1010d0 msgr2=0x7fe56c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe56bfff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c1010d0 0x7fe56c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe56bfff700 1 -- 172.21.15.89:0/3261121676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe56c100710 msgr2=0x7fe56c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe56bfff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe56c100710 0x7fe56c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe56bfff700 1 -- 172.21.15.89:0/3261121676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600095f0 con 0x7fe56c106720 2023-10-26T06:46:34.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe56bfff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c106720 0x7fe56c198bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe564007ee0 tx=0x7fe56400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:34.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe568ff9700 1 -- 172.21.15.89:0/3261121676 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56400dd20 con 0x7fe56c106720 2023-10-26T06:46:34.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe568ff9700 1 -- 172.21.15.89:0/3261121676 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe56400f040 con 0x7fe56c106720 2023-10-26T06:46:34.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c199860 con 0x7fe56c106720 2023-10-26T06:46:34.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe568ff9700 1 -- 172.21.15.89:0/3261121676 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640048a0 con 0x7fe56c106720 2023-10-26T06:46:34.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.619+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c1a5a60 con 0x7fe56c106720 2023-10-26T06:46:34.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.620+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56c101e50 con 0x7fe56c106720 2023-10-26T06:46:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.622+0000 7fe568ff9700 1 -- 172.21.15.89:0/3261121676 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe564003a00 con 0x7fe56c106720 2023-10-26T06:46:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.624+0000 7fe568ff9700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe5540a1550 0x7fe5540a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.624+0000 7fe568ff9700 1 -- 172.21.15.89:0/3261121676 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fe5640cc690 con 0x7fe56c106720 2023-10-26T06:46:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.624+0000 7fe56b7fe700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe5540a1550 0x7fe5540a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.624+0000 7fe56b7fe700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe5540a1550 0x7fe5540a3a10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005950 tx=0x7fe55c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.627+0000 7fe568ff9700 1 -- 172.21.15.89:0/3261121676 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe56404f100 con 0x7fe56c106720 2023-10-26T06:46:34.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.829+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe56c1106e0 con 0x7fe56c106720 2023-10-26T06:46:34.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.829+0000 7fe568ff9700 1 -- 172.21.15.89:0/3261121676 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe564095bb0 con 0x7fe56c106720 2023-10-26T06:46:34.831 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:46:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe5540a1550 msgr2=0x7fe5540a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe5540a1550 0x7fe5540a3a10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005950 tx=0x7fe55c00b410 comp rx=0 tx=0).stop 2023-10-26T06:46:34.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c106720 msgr2=0x7fe56c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c106720 0x7fe56c198bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe564007ee0 tx=0x7fe56400d3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 shutdown_connections 2023-10-26T06:46:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe5540a1550 0x7fe5540a3a10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe56c106720 0x7fe56c198bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe56c100710 0x7fe56c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 --2- 172.21.15.89:0/3261121676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c1010d0 0x7fe56c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 >> 172.21.15.89:0/3261121676 conn(0x7fe56c0fc200 msgr2=0x7fe56c072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 shutdown_connections 2023-10-26T06:46:34.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:34.832+0000 7fe5721ff700 1 -- 172.21.15.89:0/3261121676 wait complete. 2023-10-26T06:46:34.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:46:35.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.050+0000 7f8350dff700 1 -- 172.21.15.89:0/974038019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c10cf60 msgr2=0x7f834c10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.050+0000 7f8350dff700 1 --2- 172.21.15.89:0/974038019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c10cf60 0x7f834c10f3d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f834400b0a0 tx=0x7f834400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.050+0000 7f8350dff700 1 -- 172.21.15.89:0/974038019 shutdown_connections 2023-10-26T06:46:35.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.050+0000 7f8350dff700 1 --2- 172.21.15.89:0/974038019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c10cf60 0x7f834c10f3d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.050+0000 7f8350dff700 1 --2- 172.21.15.89:0/974038019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f834c072c20 0x7f834c073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.050+0000 7f8350dff700 1 --2- 172.21.15.89:0/974038019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f834c073540 0x7f834c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.050+0000 7f8350dff700 1 -- 172.21.15.89:0/974038019 >> 172.21.15.89:0/974038019 conn(0x7f834c0fc1b0 msgr2=0x7f834c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:35.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.051+0000 7f8350dff700 1 -- 172.21.15.89:0/974038019 shutdown_connections 2023-10-26T06:46:35.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.051+0000 7f8350dff700 1 -- 172.21.15.89:0/974038019 wait complete. 2023-10-26T06:46:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.051+0000 7f8350dff700 1 Processor -- start 2023-10-26T06:46:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8350dff700 1 -- start start 2023-10-26T06:46:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8350dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f834c072c20 0x7f834c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8350dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c073540 0x7f834c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8350dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f834c10cf60 0x7f834c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8350dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c19de50 con 0x7f834c073540 2023-10-26T06:46:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8350dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c19dfc0 con 0x7f834c10cf60 2023-10-26T06:46:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8350dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c19e180 con 0x7f834c072c20 2023-10-26T06:46:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8349d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c073540 0x7f834c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8349d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c073540 0x7f834c199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45694/0 (socket says 172.21.15.89:45694) 2023-10-26T06:46:35.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f8349d9b700 1 -- 172.21.15.89:0/2955493699 learned_addr learned my addr 172.21.15.89:0/2955493699 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:35.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f834a59c700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f834c072c20 0x7f834c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.052+0000 7f834ad9d700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f834c10cf60 0x7f834c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8349d9b700 1 -- 172.21.15.89:0/2955493699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f834c072c20 msgr2=0x7f834c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8349d9b700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f834c072c20 0x7f834c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8349d9b700 1 -- 172.21.15.89:0/2955493699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f834c10cf60 msgr2=0x7f834c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8349d9b700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f834c10cf60 0x7f834c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8349d9b700 1 -- 172.21.15.89:0/2955493699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340009520 con 0x7f834c073540 2023-10-26T06:46:35.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8349d9b700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c073540 0x7f834c199470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f834000eb30 tx=0x7f834000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f833b7fe700 1 -- 172.21.15.89:0/2955493699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340009880 con 0x7f834c073540 2023-10-26T06:46:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f833b7fe700 1 -- 172.21.15.89:0/2955493699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83400044e0 con 0x7f834c073540 2023-10-26T06:46:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8344009d00 con 0x7f834c073540 2023-10-26T06:46:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f833b7fe700 1 -- 172.21.15.89:0/2955493699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340003d80 con 0x7f834c073540 2023-10-26T06:46:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.053+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c19e650 con 0x7f834c073540 2023-10-26T06:46:35.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.054+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c108170 con 0x7f834c073540 2023-10-26T06:46:35.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.056+0000 7f833b7fe700 1 -- 172.21.15.89:0/2955493699 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8340003f20 con 0x7f834c073540 2023-10-26T06:46:35.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.058+0000 7f833b7fe700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f83340a1550 0x7f83340a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.058+0000 7f833b7fe700 1 -- 172.21.15.89:0/2955493699 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f8340014070 con 0x7f834c073540 2023-10-26T06:46:35.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.058+0000 7f834a59c700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f83340a1550 0x7f83340a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.058+0000 7f834a59c700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f83340a1550 0x7f83340a3a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f833c009c60 tx=0x7f833c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:35.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.061+0000 7f833b7fe700 1 -- 172.21.15.89:0/2955493699 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8340096820 con 0x7f834c073540 2023-10-26T06:46:35.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.266+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f834c19a230 con 0x7f834c073540 2023-10-26T06:46:35.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.266+0000 7f833b7fe700 1 -- 172.21.15.89:0/2955493699 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8340099e40 con 0x7f834c073540 2023-10-26T06:46:35.268 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:46:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.268+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f83340a1550 msgr2=0x7f83340a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.268+0000 7f8350dff700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f83340a1550 0x7f83340a3a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f833c009c60 tx=0x7f833c009260 comp rx=0 tx=0).stop 2023-10-26T06:46:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.268+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c073540 msgr2=0x7f834c199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.268+0000 7f8350dff700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c073540 0x7f834c199470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f834000eb30 tx=0x7f834000ee40 comp rx=0 tx=0).stop 2023-10-26T06:46:35.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 shutdown_connections 2023-10-26T06:46:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f83340a1550 0x7f83340a3a10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f834c10cf60 0x7f834c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f834c072c20 0x7f834c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 --2- 172.21.15.89:0/2955493699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f834c073540 0x7f834c199470 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 >> 172.21.15.89:0/2955493699 conn(0x7f834c0fc1b0 msgr2=0x7f834c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 shutdown_connections 2023-10-26T06:46:35.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.269+0000 7f8350dff700 1 -- 172.21.15.89:0/2955493699 wait complete. 2023-10-26T06:46:35.290 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:46:35.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:35 smithi192 ceph-mon[131661]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:35.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:35 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2470484036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:35.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:35 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3261121676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:35 smithi135 ceph-mon[131403]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:35 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2470484036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:35 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3261121676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:35.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.489+0000 7f713b1ff700 1 -- 172.21.15.89:0/290831880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134106730 msgr2=0x7f7134106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.489+0000 7f713b1ff700 1 --2- 172.21.15.89:0/290831880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134106730 0x7f7134106b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7128009990 tx=0x7f7128009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.489+0000 7f713b1ff700 1 -- 172.21.15.89:0/290831880 shutdown_connections 2023-10-26T06:46:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.489+0000 7f713b1ff700 1 --2- 172.21.15.89:0/290831880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134106730 0x7f7134106b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.489+0000 7f713b1ff700 1 --2- 172.21.15.89:0/290831880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71341010e0 0x7f713410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.489+0000 7f713b1ff700 1 --2- 172.21.15.89:0/290831880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134100720 0x7f7134100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.489+0000 7f713b1ff700 1 -- 172.21.15.89:0/290831880 >> 172.21.15.89:0/290831880 conn(0x7f71340fc1b0 msgr2=0x7f71340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.490+0000 7f713b1ff700 1 -- 172.21.15.89:0/290831880 shutdown_connections 2023-10-26T06:46:35.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.490+0000 7f713b1ff700 1 -- 172.21.15.89:0/290831880 wait complete. 2023-10-26T06:46:35.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.490+0000 7f713b1ff700 1 Processor -- start 2023-10-26T06:46:35.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f713b1ff700 1 -- start start 2023-10-26T06:46:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f713b1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134100720 0x7f713419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f713b1ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71341010e0 0x7f713419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f713b1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7134106730 0x7f7134198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f713b1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134199250 con 0x7f7134100720 2023-10-26T06:46:35.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f713b1ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71341993c0 con 0x7f71341010e0 2023-10-26T06:46:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f713b1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134199580 con 0x7f7134106730 2023-10-26T06:46:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f7138f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134100720 0x7f713419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f7138f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134100720 0x7f713419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45712/0 (socket says 172.21.15.89:45712) 2023-10-26T06:46:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f7138f9b700 1 -- 172.21.15.89:0/2862624542 learned_addr learned my addr 172.21.15.89:0/2862624542 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:35.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.491+0000 7f7133fff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71341010e0 0x7f713419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f713979c700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7134106730 0x7f7134198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7138f9b700 1 -- 172.21.15.89:0/2862624542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7134106730 msgr2=0x7f7134198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7138f9b700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7134106730 0x7f7134198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7138f9b700 1 -- 172.21.15.89:0/2862624542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71341010e0 msgr2=0x7f713419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7138f9b700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71341010e0 0x7f713419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7138f9b700 1 -- 172.21.15.89:0/2862624542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280095f0 con 0x7f7134100720 2023-10-26T06:46:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f713979c700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7134106730 0x7f7134198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:46:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7133fff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71341010e0 0x7f713419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7138f9b700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134100720 0x7f713419ea70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7128005910 tx=0x7f7128004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7131ffb700 1 -- 172.21.15.89:0/2862624542 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712801d070 con 0x7f7134100720 2023-10-26T06:46:35.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7131ffb700 1 -- 172.21.15.89:0/2862624542 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7128022470 con 0x7f7134100720 2023-10-26T06:46:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7134199800 con 0x7f7134100720 2023-10-26T06:46:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f7131ffb700 1 -- 172.21.15.89:0/2862624542 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128017650 con 0x7f7134100720 2023-10-26T06:46:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.492+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71341a5a60 con 0x7f7134100720 2023-10-26T06:46:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.493+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7134101e60 con 0x7f7134100720 2023-10-26T06:46:35.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.495+0000 7f7131ffb700 1 -- 172.21.15.89:0/2862624542 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f71280225e0 con 0x7f7134100720 2023-10-26T06:46:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.497+0000 7f7131ffb700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f71240a14d0 0x7f71240a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.497+0000 7f7131ffb700 1 -- 172.21.15.89:0/2862624542 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f71280ce4d0 con 0x7f7134100720 2023-10-26T06:46:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.497+0000 7f7133fff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f71240a14d0 0x7f71240a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.497+0000 7f7133fff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f71240a14d0 0x7f71240a3990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f713419a240 tx=0x7f7120009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:35.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.499+0000 7f7131ffb700 1 -- 172.21.15.89:0/2862624542 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7128097ae0 con 0x7f7134100720 2023-10-26T06:46:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:35 smithi089 ceph-mon[95031]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:35 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2470484036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:35 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3261121676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:35.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.701+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7134199ff0 con 0x7f7134100720 2023-10-26T06:46:35.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.701+0000 7f7131ffb700 1 -- 172.21.15.89:0/2862624542 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f712809b100 con 0x7f7134100720 2023-10-26T06:46:35.703 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:46:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f71240a14d0 msgr2=0x7f71240a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f71240a14d0 0x7f71240a3990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f713419a240 tx=0x7f7120009260 comp rx=0 tx=0).stop 2023-10-26T06:46:35.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134100720 msgr2=0x7f713419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134100720 0x7f713419ea70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7128005910 tx=0x7f7128004730 comp rx=0 tx=0).stop 2023-10-26T06:46:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 shutdown_connections 2023-10-26T06:46:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f71240a14d0 0x7f71240a3990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7134100720 0x7f713419ea70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7134106730 0x7f7134198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 --2- 172.21.15.89:0/2862624542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71341010e0 0x7f713419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 >> 172.21.15.89:0/2862624542 conn(0x7f71340fc1b0 msgr2=0x7f71340fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 shutdown_connections 2023-10-26T06:46:35.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.704+0000 7f713b1ff700 1 -- 172.21.15.89:0/2862624542 wait complete. 2023-10-26T06:46:35.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:46:35.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 -- 172.21.15.89:0/2958218800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281028c0 msgr2=0x7f3828102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 --2- 172.21.15.89:0/2958218800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281028c0 0x7f3828102d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f381c009990 tx=0x7f381c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 -- 172.21.15.89:0/2958218800 shutdown_connections 2023-10-26T06:46:35.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 --2- 172.21.15.89:0/2958218800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3828103280 0x7f382810f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 --2- 172.21.15.89:0/2958218800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38281088d0 0x7f3828108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 --2- 172.21.15.89:0/2958218800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281028c0 0x7f3828102d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 -- 172.21.15.89:0/2958218800 >> 172.21.15.89:0/2958218800 conn(0x7f38280fe370 msgr2=0x7f3828100790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 -- 172.21.15.89:0/2958218800 shutdown_connections 2023-10-26T06:46:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.923+0000 7f382dfff700 1 -- 172.21.15.89:0/2958218800 wait complete. 2023-10-26T06:46:35.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 Processor -- start 2023-10-26T06:46:35.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 -- start start 2023-10-26T06:46:35.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38281028c0 0x7f382819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3828103280 0x7f382819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281088d0 0x7f3828198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3828199320 con 0x7f38281028c0 2023-10-26T06:46:35.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3828199490 con 0x7f3828103280 2023-10-26T06:46:35.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.924+0000 7f382dfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3828199650 con 0x7f38281088d0 2023-10-26T06:46:35.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38281028c0 0x7f382819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38281028c0 0x7f382819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45726/0 (socket says 172.21.15.89:45726) 2023-10-26T06:46:35.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 -- 172.21.15.89:0/971869466 learned_addr learned my addr 172.21.15.89:0/971869466 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:35.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f3827fff700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281088d0 0x7f3828198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f3826ffd700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3828103280 0x7f382819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 -- 172.21.15.89:0/971869466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281088d0 msgr2=0x7f3828198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281088d0 0x7f3828198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 -- 172.21.15.89:0/971869466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3828103280 msgr2=0x7f382819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3828103280 0x7f382819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:35.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 -- 172.21.15.89:0/971869466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0095f0 con 0x7f38281028c0 2023-10-26T06:46:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f38277fe700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38281028c0 0x7f382819ea60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f381800e9a0 tx=0x7f381800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f3824ff9700 1 -- 172.21.15.89:0/971869466 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818009790 con 0x7f38281028c0 2023-10-26T06:46:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f3824ff9700 1 -- 172.21.15.89:0/971869466 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3818010430 con 0x7f38281028c0 2023-10-26T06:46:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.925+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3828199930 con 0x7f38281028c0 2023-10-26T06:46:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.926+0000 7f3824ff9700 1 -- 172.21.15.89:0/971869466 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818018680 con 0x7f38281028c0 2023-10-26T06:46:35.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.926+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38281a5a30 con 0x7f38281028c0 2023-10-26T06:46:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.927+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382810aec0 con 0x7f38281028c0 2023-10-26T06:46:35.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.928+0000 7f3824ff9700 1 -- 172.21.15.89:0/971869466 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f38180105a0 con 0x7f38281028c0 2023-10-26T06:46:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.930+0000 7f3824ff9700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f38100a1550 0x7f38100a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.930+0000 7f3824ff9700 1 -- 172.21.15.89:0/971869466 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3818014070 con 0x7f38281028c0 2023-10-26T06:46:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.930+0000 7f3826ffd700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f38100a1550 0x7f38100a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:35.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.931+0000 7f3826ffd700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f38100a1550 0x7f38100a3a10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f382819a270 tx=0x7f381c005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:35.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:35.933+0000 7f3824ff9700 1 -- 172.21.15.89:0/971869466 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3818096470 con 0x7f38281028c0 2023-10-26T06:46:36.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.181+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3828199fe0 con 0x7f38281028c0 2023-10-26T06:46:36.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.182+0000 7f3824ff9700 1 -- 172.21.15.89:0/971869466 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f3818099a90 con 0x7f38281028c0 2023-10-26T06:46:36.183 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:36.183 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:46:36.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f38100a1550 msgr2=0x7f38100a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f38100a1550 0x7f38100a3a10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f382819a270 tx=0x7f381c005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38281028c0 msgr2=0x7f382819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38281028c0 0x7f382819ea60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f381800e9a0 tx=0x7f381800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 shutdown_connections 2023-10-26T06:46:36.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f38100a1550 0x7f38100a3a10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38281028c0 0x7f382819ea60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38281088d0 0x7f3828198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 --2- 172.21.15.89:0/971869466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3828103280 0x7f382819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 >> 172.21.15.89:0/971869466 conn(0x7f38280fe370 msgr2=0x7f38280fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 shutdown_connections 2023-10-26T06:46:36.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.184+0000 7f382dfff700 1 -- 172.21.15.89:0/971869466 wait complete. 2023-10-26T06:46:36.188 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:36.204 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:36.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:36 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2955493699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:36.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:36 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2862624542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 -- 172.21.15.89:0/1399059205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41006a0 msgr2=0x7f09f4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1399059205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41006a0 0x7f09f4100b20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009990 tx=0x7f09e8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 -- 172.21.15.89:0/1399059205 shutdown_connections 2023-10-26T06:46:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1399059205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f4101060 0x7f09f410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1399059205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09f41066b0 0x7f09f4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1399059205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41006a0 0x7f09f4100b20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 -- 172.21.15.89:0/1399059205 >> 172.21.15.89:0/1399059205 conn(0x7f09f40fc170 msgr2=0x7f09f40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 -- 172.21.15.89:0/1399059205 shutdown_connections 2023-10-26T06:46:36.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.401+0000 7f09f8dff700 1 -- 172.21.15.89:0/1399059205 wait complete. 2023-10-26T06:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.402+0000 7f09f8dff700 1 Processor -- start 2023-10-26T06:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.402+0000 7f09f8dff700 1 -- start start 2023-10-26T06:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.402+0000 7f09f8dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f41006a0 0x7f09f419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.402+0000 7f09f8dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09f4101060 0x7f09f419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.402+0000 7f09f8dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41066b0 0x7f09f4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.402+0000 7f09f8dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4199190 con 0x7f09f41066b0 2023-10-26T06:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f8dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4199300 con 0x7f09f41006a0 2023-10-26T06:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f8dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f41994c0 con 0x7f09f4101060 2023-10-26T06:46:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41066b0 0x7f09f4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41066b0 0x7f09f4198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45742/0 (socket says 172.21.15.89:45742) 2023-10-26T06:46:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 -- 172.21.15.89:0/1210115939 learned_addr learned my addr 172.21.15.89:0/1210115939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f259c700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f41006a0 0x7f09f419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f1d9b700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09f4101060 0x7f09f419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 -- 172.21.15.89:0/1210115939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09f4101060 msgr2=0x7f09f419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09f4101060 0x7f09f419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 -- 172.21.15.89:0/1210115939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f41006a0 msgr2=0x7f09f419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f41006a0 0x7f09f419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 -- 172.21.15.89:0/1210115939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e80095f0 con 0x7f09f41066b0 2023-10-26T06:46:36.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09f2d9d700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41066b0 0x7f09f4198b30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00ef30 tx=0x7f09ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.403+0000 7f09e37fe700 1 -- 172.21.15.89:0/1210115939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec018070 con 0x7f09f41066b0 2023-10-26T06:46:36.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.404+0000 7f09e37fe700 1 -- 172.21.15.89:0/1210115939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09ec00f040 con 0x7f09f41066b0 2023-10-26T06:46:36.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.404+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f4199740 con 0x7f09f41066b0 2023-10-26T06:46:36.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.404+0000 7f09e37fe700 1 -- 172.21.15.89:0/1210115939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00cdb0 con 0x7f09f41066b0 2023-10-26T06:46:36.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.404+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f41a5a40 con 0x7f09f41066b0 2023-10-26T06:46:36.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.405+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f4101de0 con 0x7f09f41066b0 2023-10-26T06:46:36.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.406+0000 7f09e37fe700 1 -- 172.21.15.89:0/1210115939 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f09ec003a00 con 0x7f09f41066b0 2023-10-26T06:46:36.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.408+0000 7f09e37fe700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f09dc0a1550 0x7f09dc0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.408+0000 7f09e37fe700 1 -- 172.21.15.89:0/1210115939 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f09ec0cc9b0 con 0x7f09f41066b0 2023-10-26T06:46:36.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.408+0000 7f09f259c700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f09dc0a1550 0x7f09dc0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.408+0000 7f09f259c700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f09dc0a1550 0x7f09dc0a3a10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007b60 tx=0x7f09e40073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.411+0000 7f09e37fe700 1 -- 172.21.15.89:0/1210115939 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f09ec095fc0 con 0x7f09f41066b0 2023-10-26T06:46:36.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:36 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2955493699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:36.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:36 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2862624542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:36.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:36 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2955493699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:36.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:36 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2862624542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:36.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.612+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09f4110620 con 0x7f09f41066b0 2023-10-26T06:46:36.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.613+0000 7f09e37fe700 1 -- 172.21.15.89:0/1210115939 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7f09ec0995e0 con 0x7f09f41066b0 2023-10-26T06:46:36.615 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:36.615 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:36.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f09dc0a1550 msgr2=0x7f09dc0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f09dc0a1550 0x7f09dc0a3a10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007b60 tx=0x7f09e40073d0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41066b0 msgr2=0x7f09f4198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41066b0 0x7f09f4198b30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00ef30 tx=0x7f09ec00c3a0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 shutdown_connections 2023-10-26T06:46:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f09dc0a1550 0x7f09dc0a3a10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09f41066b0 0x7f09f4198b30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f41006a0 0x7f09f419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 --2- 172.21.15.89:0/1210115939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09f4101060 0x7f09f419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 >> 172.21.15.89:0/1210115939 conn(0x7f09f40fc170 msgr2=0x7f09f410bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 shutdown_connections 2023-10-26T06:46:36.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.616+0000 7f09f8dff700 1 -- 172.21.15.89:0/1210115939 wait complete. 2023-10-26T06:46:36.637 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-26T06:45:05.445103+0000', 'modified': '2023-10-26T06:45:09.558391+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24703': {'gid': 24703, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6839/2442523019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 2442523019}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 2442523019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6839/609246385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 609246385}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 609246385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T06:46:36.637 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T06:46:36.638 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-26T06:46:36.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 -- 172.21.15.89:0/1733397053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c84101060 msgr2=0x7f8c8410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 --2- 172.21.15.89:0/1733397053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c84101060 0x7f8c8410d570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b0a0 tx=0x7f8c7c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 -- 172.21.15.89:0/1733397053 shutdown_connections 2023-10-26T06:46:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 --2- 172.21.15.89:0/1733397053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c84101060 0x7f8c8410d570 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 --2- 172.21.15.89:0/1733397053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c841066b0 0x7f8c84106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 --2- 172.21.15.89:0/1733397053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c841006a0 0x7f8c84100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 -- 172.21.15.89:0/1733397053 >> 172.21.15.89:0/1733397053 conn(0x7f8c840fc170 msgr2=0x7f8c840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.833+0000 7f8c88fff700 1 -- 172.21.15.89:0/1733397053 shutdown_connections 2023-10-26T06:46:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.834+0000 7f8c88fff700 1 -- 172.21.15.89:0/1733397053 wait complete. 2023-10-26T06:46:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.834+0000 7f8c88fff700 1 Processor -- start 2023-10-26T06:46:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c88fff700 1 -- start start 2023-10-26T06:46:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c88fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c841006a0 0x7f8c8419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c88fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c84101060 0x7f8c8419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c88fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c841066b0 0x7f8c84198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c88fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841990c0 con 0x7f8c841006a0 2023-10-26T06:46:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c88fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c84199230 con 0x7f8c84101060 2023-10-26T06:46:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c88fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841993f0 con 0x7f8c841066b0 2023-10-26T06:46:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c8259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c841006a0 0x7f8c8419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c8259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c841006a0 0x7f8c8419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45770/0 (socket says 172.21.15.89:45770) 2023-10-26T06:46:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c8259c700 1 -- 172.21.15.89:0/4082617455 learned_addr learned my addr 172.21.15.89:0/4082617455 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.835+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c84101060 0x7f8c8419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c841066b0 0x7f8c84198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c8259c700 1 -- 172.21.15.89:0/4082617455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c841066b0 msgr2=0x7f8c84198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c8259c700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c841066b0 0x7f8c84198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c8259c700 1 -- 172.21.15.89:0/4082617455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c84101060 msgr2=0x7f8c8419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c8259c700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c84101060 0x7f8c8419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c8259c700 1 -- 172.21.15.89:0/4082617455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c009d00 con 0x7f8c841006a0 2023-10-26T06:46:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c84101060 0x7f8c8419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c8259c700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c841006a0 0x7f8c8419e860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400e9a0 tx=0x7f8c7400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c737fe700 1 -- 172.21.15.89:0/4082617455 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74009790 con 0x7f8c841006a0 2023-10-26T06:46:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c737fe700 1 -- 172.21.15.89:0/4082617455 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c7400cbd0 con 0x7f8c841006a0 2023-10-26T06:46:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c737fe700 1 -- 172.21.15.89:0/4082617455 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74018680 con 0x7f8c841006a0 2023-10-26T06:46:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c841996d0 con 0x7f8c841006a0 2023-10-26T06:46:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.836+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c841a5900 con 0x7f8c841006a0 2023-10-26T06:46:36.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.839+0000 7f8c737fe700 1 -- 172.21.15.89:0/4082617455 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f8c7400cd40 con 0x7f8c841006a0 2023-10-26T06:46:36.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.841+0000 7f8c9c93bb80 1 Processor -- start 2023-10-26T06:46:36.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.841+0000 7f8c9c93bb80 1 -- start start 2023-10-26T06:46:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.841+0000 7f8c737fe700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8c6c0a1550 0x7f8c6c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c737fe700 1 -- 172.21.15.89:0/4082617455 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f8c74014070 con 0x7f8c841006a0 2023-10-26T06:46:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e3663690 con 0x55e9e36f64b0 2023-10-26T06:46:36.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e3649c10 con 0x55e9e36f5b90 2023-10-26T06:46:36.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e3647530 con 0x55e9e36f0580 2023-10-26T06:46:36.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8c6c0a1550 0x7f8c6c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45782/0 (socket says 172.21.15.89:45782) 2023-10-26T06:46:36.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 -- 172.21.15.89:0/654963854 learned_addr learned my addr 172.21.15.89:0/654963854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:36.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c8259c700 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 -- 172.21.15.89:0/654963854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 msgr2=0x55e9e36f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 -- 172.21.15.89:0/654963854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 msgr2=0x55e9e36f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8c6c0a1550 0x7f8c6c0a3a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009610 tx=0x7f8c7c007c20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c81d9b700 1 -- 172.21.15.89:0/654963854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e36003a0 con 0x55e9e36f64b0 2023-10-26T06:46:36.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.842+0000 7f8c8259c700 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:36.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.843+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f0040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d80 tx=0x7f8c780116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9471968788df3d7a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.843+0000 7f8c70ff9700 1 -- 172.21.15.89:0/654963854 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78011e20 con 0x55e9e36f64b0 2023-10-26T06:46:36.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.843+0000 7f8c70ff9700 1 -- 172.21.15.89:0/654963854 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c78012910 con 0x55e9e36f64b0 2023-10-26T06:46:36.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.843+0000 7f8c70ff9700 1 -- 172.21.15.89:0/654963854 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800f840 con 0x55e9e36f64b0 2023-10-26T06:46:36.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/654963854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 msgr2=0x55e9e36f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f0040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d80 tx=0x7f8c780116c0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/654963854 shutdown_connections 2023-10-26T06:46:36.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/654963854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f0040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/654963854 >> 172.21.15.89:0/654963854 conn(0x55e9e36646d0 msgr2=0x55e9e36eab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/654963854 shutdown_connections 2023-10-26T06:46:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/654963854 wait complete. 2023-10-26T06:46:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.844+0000 7f8c9c93bb80 1 -- start start 2023-10-26T06:46:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 1 Processor -- start 2023-10-26T06:46:36.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f5b90 0x55e9e36f9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e3642e20 con 0x55e9e36f5b90 2023-10-26T06:46:36.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e35df6c0 con 0x55e9e36f64b0 2023-10-26T06:46:36.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e35df360 con 0x55e9e36f0580 2023-10-26T06:46:36.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c9c93bb80 10 client.0 _ll_register_callbacks cb 0x55e9e3529a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:36.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c81d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f5b90 0x55e9e36f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c81d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f5b90 0x55e9e36f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45798/0 (socket says 172.21.15.89:45798) 2023-10-26T06:46:36.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c81d9b700 1 -- 172.21.15.89:0/374592567 learned_addr learned my addr 172.21.15.89:0/374592567 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:36.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c8259c700 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.845+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c81d9b700 1 -- 172.21.15.89:0/374592567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 msgr2=0x55e9e36fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c81d9b700 1 -- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f64b0 msgr2=0x55e9e36f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c81d9b700 1 -- 172.21.15.89:0/374592567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c841a5900 con 0x55e9e36f5b90 2023-10-26T06:46:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f5b90 0x55e9e36f9790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8c7801c040 tx=0x7f8c78012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb2c68ebb4aae9b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:46:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c69ffb700 1 -- 172.21.15.89:0/374592567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78012660 con 0x55e9e36f5b90 2023-10-26T06:46:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c69ffb700 1 -- 172.21.15.89:0/374592567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f8c78004040 con 0x55e9e36f5b90 2023-10-26T06:46:36.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9e3647ea0 con 0x55e9e36f5b90 2023-10-26T06:46:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c69ffb700 1 -- 172.21.15.89:0/374592567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c78004200 con 0x55e9e36f5b90 2023-10-26T06:46:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c9c93bb80 10 client.15243 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.846+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f8c780099b0 con 0x55e9e36f5b90 2023-10-26T06:46:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c69ffb700 1 -- 172.21.15.89:0/374592567 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9e3647ea0 con 0x55e9e36f5b90 2023-10-26T06:46:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c69ffb700 1 -- 172.21.15.89:0/374592567 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c78022070 con 0x55e9e36f5b90 2023-10-26T06:46:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c9c93bb80 10 client.15243 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c9c93bb80 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T06:46:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c9c93bb80 20 client.15243 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e3709e90 0x55e9e3708950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c9c93bb80 4 client.15243 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:46:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9e36bf860 con 0x55e9e3709e90 2023-10-26T06:46:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.847+0000 7f8c8259c700 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e3709e90 0x55e9e3708950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.848+0000 7f8c8259c700 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e3709e90 0x55e9e3708950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ccb320a2d58ea42 server_cookie=fc648af6e830656d in_seq=0 out_seq=0 2023-10-26T06:46:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.848+0000 7f8c69ffb700 10 client.15243 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.848+0000 7f8c69ffb700 1 -- 172.21.15.89:0/374592567 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9e36bf860 con 0x55e9e3709e90 2023-10-26T06:46:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.848+0000 7f8c69ffb700 10 client.15243 handle_command_reply: tid=0 2023-10-26T06:46:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.848+0000 7f8c9c93bb80 1 client.15243 shutdown 2023-10-26T06:46:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f5b90 msgr2=0x55e9e36f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f5b90 0x55e9e36f9790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8c7801c040 tx=0x7f8c78012d30 comp rx=0 tx=0).stop 2023-10-26T06:46:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 shutdown_connections 2023-10-26T06:46:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e3709e90 0x55e9e3708950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c69ffb700 0 client.15243 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/374592567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f5b90 0x55e9e36f9790 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 >> 172.21.15.89:0/374592567 conn(0x55e9e36646d0 msgr2=0x55e9e36ef030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 shutdown_connections 2023-10-26T06:46:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/374592567 wait complete. 2023-10-26T06:46:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.849+0000 7f8c9c93bb80 20 client.15243 trim_cache size 0 max 16384 2023-10-26T06:46:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 Processor -- start 2023-10-26T06:46:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 -- start start 2023-10-26T06:46:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e3712800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e34c38d0 con 0x55e9e36f64b0 2023-10-26T06:46:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e363ef40 con 0x55e9e36f5b90 2023-10-26T06:46:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.867+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e35e94d0 con 0x55e9e36f0580 2023-10-26T06:46:36.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c8259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45802/0 (socket says 172.21.15.89:45802) 2023-10-26T06:46:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 -- 172.21.15.89:0/1029614626 learned_addr learned my addr 172.21.15.89:0/1029614626 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e3712800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 -- 172.21.15.89:0/1029614626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 msgr2=0x55e9e36f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 -- 172.21.15.89:0/1029614626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 msgr2=0x55e9e3712800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e3712800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 -- 172.21.15.89:0/1029614626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e346dd90 con 0x55e9e36f64b0 2023-10-26T06:46:36.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e3712800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c82d9d700 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9730 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c007f70 tx=0x7f8c7c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43748a622281721e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c717fa700 1 -- 172.21.15.89:0/1029614626 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00e040 con 0x55e9e36f64b0 2023-10-26T06:46:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.868+0000 7f8c717fa700 1 -- 172.21.15.89:0/1029614626 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c7c007960 con 0x55e9e36f64b0 2023-10-26T06:46:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c717fa700 1 -- 172.21.15.89:0/1029614626 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0056c0 con 0x55e9e36f64b0 2023-10-26T06:46:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/1029614626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 msgr2=0x55e9e36f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9730 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c007f70 tx=0x7f8c7c020000 comp rx=0 tx=0).stop 2023-10-26T06:46:36.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/1029614626 shutdown_connections 2023-10-26T06:46:36.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e36f9730 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/1029614626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e3712800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/1029614626 >> 172.21.15.89:0/1029614626 conn(0x55e9e36646d0 msgr2=0x55e9e36f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/1029614626 shutdown_connections 2023-10-26T06:46:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.869+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/1029614626 wait complete. 2023-10-26T06:46:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 -- start start 2023-10-26T06:46:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 Processor -- start 2023-10-26T06:46:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e3711dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e3712310 con 0x55e9e36f64b0 2023-10-26T06:46:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e3712480 con 0x55e9e36f5b90 2023-10-26T06:46:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e3712640 con 0x55e9e36f0580 2023-10-26T06:46:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c9c93bb80 10 client.0 _ll_register_callbacks cb 0x55e9e3529a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c81d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c82d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e3711dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c81d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:56752/0 (socket says 172.21.15.89:56752) 2023-10-26T06:46:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.870+0000 7f8c81d9b700 1 -- 172.21.15.89:0/3924847084 learned_addr learned my addr 172.21.15.89:0/3924847084 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c8259c700 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c81d9b700 1 -- 172.21.15.89:0/3924847084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 msgr2=0x55e9e36f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c81d9b700 1 -- 172.21.15.89:0/3924847084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 msgr2=0x55e9e3711dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e3711dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c81d9b700 1 -- 172.21.15.89:0/3924847084 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c780099b0 con 0x55e9e36f5b90 2023-10-26T06:46:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c8259c700 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c81d9b700 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f93d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e9e36e3050 tx=0x7f8c78012890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c3ffaf4c8221f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:46:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c69ffb700 1 -- 172.21.15.89:0/3924847084 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780052b0 con 0x55e9e36f5b90 2023-10-26T06:46:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c78012910 con 0x55e9e36f5b90 2023-10-26T06:46:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.871+0000 7f8c69ffb700 1 -- 172.21.15.89:0/3924847084 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f8c78005410 con 0x55e9e36f5b90 2023-10-26T06:46:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c69ffb700 1 -- 172.21.15.89:0/3924847084 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c780055d0 con 0x55e9e36f5b90 2023-10-26T06:46:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 10 client.25265 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f8c8410de70 con 0x55e9e36f5b90 2023-10-26T06:46:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c69ffb700 1 -- 172.21.15.89:0/3924847084 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c78012910 con 0x55e9e36f5b90 2023-10-26T06:46:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c69ffb700 1 -- 172.21.15.89:0/3924847084 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c78022070 con 0x55e9e36f5b90 2023-10-26T06:46:36.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 10 client.25265 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:36.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 10 client.25265 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T06:46:36.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 20 client.25265 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:36.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e37631c0 0x55e9e375f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:36.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 4 client.25265 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T06:46:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.872+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e9e363be00 con 0x55e9e37631c0 2023-10-26T06:46:36.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c8259c700 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e37631c0 0x55e9e375f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c69ffb700 10 client.25265 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c8259c700 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e37631c0 0x55e9e375f640 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173c030dece96588 server_cookie=3d3505e3d35821fd in_seq=0 out_seq=0 2023-10-26T06:46:36.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c69ffb700 1 -- 172.21.15.89:0/3924847084 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e9e363be00 con 0x55e9e37631c0 2023-10-26T06:46:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c69ffb700 10 client.25265 handle_command_reply: tid=0 2023-10-26T06:46:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c9c93bb80 1 client.25265 shutdown 2023-10-26T06:46:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 msgr2=0x55e9e36f93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.873+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f93d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e9e36e3050 tx=0x7f8c78012890 comp rx=0 tx=0).stop 2023-10-26T06:46:36.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 shutdown_connections 2023-10-26T06:46:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e9e36f64b0 0x55e9e3711dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e9e37631c0 0x55e9e375f640 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e9e36f0580 0x55e9e36f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 --2- 172.21.15.89:0/3924847084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9e36f5b90 0x55e9e36f93d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c69ffb700 0 client.25265 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:36.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 >> 172.21.15.89:0/3924847084 conn(0x55e9e36646d0 msgr2=0x55e9e36f4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 shutdown_connections 2023-10-26T06:46:36.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 1 -- 172.21.15.89:0/3924847084 wait complete. 2023-10-26T06:46:36.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c9c93bb80 20 client.25265 trim_cache size 0 max 16384 2023-10-26T06:46:36.891 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:46:36.891 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "e22b2d1a-73c9-11ee-8db9-212e2dc638e7", 2023-10-26T06:46:36.891 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2023-10-26T06:46:36.891 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24743, 2023-10-26T06:46:36.891 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2023-10-26T06:46:36.892 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2023-10-26T06:46:36.892 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2023-10-26T06:46:36.892 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 90.822168098999995, 2023-10-26T06:46:36.892 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 20, 2023-10-26T06:46:36.892 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 86, 2023-10-26T06:46:36.892 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 84, 2023-10-26T06:46:36.893 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 97.934738620999994 2023-10-26T06:46:36.893 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:46:36.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8c6c0a1550 msgr2=0x7f8c6c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c88fff700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8c6c0a1550 0x7f8c6c0a3a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009610 tx=0x7f8c7c007c20 comp rx=0 tx=0).stop 2023-10-26T06:46:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c841006a0 msgr2=0x7f8c8419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c88fff700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c841006a0 0x7f8c8419e860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400e9a0 tx=0x7f8c7400ed60 comp rx=0 tx=0).stop 2023-10-26T06:46:36.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 shutdown_connections 2023-10-26T06:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c88fff700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8c6c0a1550 0x7f8c6c0a3a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.874+0000 7f8c88fff700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c841006a0 0x7f8c8419e860 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.875+0000 7f8c88fff700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c841066b0 0x7f8c84198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.875+0000 7f8c88fff700 1 --2- 172.21.15.89:0/4082617455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c84101060 0x7f8c8419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.875+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 >> 172.21.15.89:0/4082617455 conn(0x7f8c840fc170 msgr2=0x7f8c8410c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:36.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.875+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 shutdown_connections 2023-10-26T06:46:36.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:36.875+0000 7f8c88fff700 1 -- 172.21.15.89:0/4082617455 wait complete. 2023-10-26T06:46:36.897 INFO:teuthology.run_tasks:Running task kclient... 2023-10-26T06:46:36.920 INFO:tasks.kclient:Mounting kernel clients... 2023-10-26T06:46:36.920 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-26T06:46:36.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:36.921 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-26T06:46:36.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:36.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-26T06:46:37.078 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2023-10-26T06:46:37.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:46:37.081 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-10-26T06:46:37.081 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T06:46:37.082 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:37.082 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph fs ls 2023-10-26T06:46:37.144 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:37 smithi089 ceph-mon[95031]: pgmap v301: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:37.145 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:37 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/971869466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:37.145 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:37 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1210115939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:37 smithi192 ceph-mon[131661]: pgmap v301: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:37 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/971869466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:37 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1210115939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:37.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:37 smithi135 ceph-mon[131403]: pgmap v301: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:37.518 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:37 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/971869466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:37.519 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:37 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1210115939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:46:39.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 -- 172.21.15.89:0/2567490906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381057c0 msgr2=0x7f4638105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:39.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 --2- 172.21.15.89:0/2567490906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381057c0 0x7f4638105ba0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4628009990 tx=0x7f4628009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 -- 172.21.15.89:0/2567490906 shutdown_connections 2023-10-26T06:46:39.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 --2- 172.21.15.89:0/2567490906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381057c0 0x7f4638105ba0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 --2- 172.21.15.89:0/2567490906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4638101260 0x7f463810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 --2- 172.21.15.89:0/2567490906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46381008a0 0x7f4638100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 -- 172.21.15.89:0/2567490906 >> 172.21.15.89:0/2567490906 conn(0x7f46380779e0 msgr2=0x7f4638077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 -- 172.21.15.89:0/2567490906 shutdown_connections 2023-10-26T06:46:39.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.276+0000 7f463cf82700 1 -- 172.21.15.89:0/2567490906 wait complete. 2023-10-26T06:46:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.277+0000 7f463cf82700 1 Processor -- start 2023-10-26T06:46:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463cf82700 1 -- start start 2023-10-26T06:46:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463cf82700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381008a0 0x7f4638198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:39.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463cf82700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4638101260 0x7f4638199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:39.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463cf82700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46381057c0 0x7f463819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:39.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463cf82700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463819dd50 con 0x7f46381008a0 2023-10-26T06:46:39.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463cf82700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463819dec0 con 0x7f46381057c0 2023-10-26T06:46:39.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463cf82700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463819e080 con 0x7f4638101260 2023-10-26T06:46:39.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381008a0 0x7f4638198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381008a0 0x7f4638198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45830/0 (socket says 172.21.15.89:45830) 2023-10-26T06:46:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f463659c700 1 -- 172.21.15.89:0/1561898870 learned_addr learned my addr 172.21.15.89:0/1561898870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f4636d9d700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46381057c0 0x7f463819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.278+0000 7f4635d9b700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4638101260 0x7f4638199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463659c700 1 -- 172.21.15.89:0/1561898870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4638101260 msgr2=0x7f4638199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463659c700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4638101260 0x7f4638199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463659c700 1 -- 172.21.15.89:0/1561898870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46381057c0 msgr2=0x7f463819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463659c700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46381057c0 0x7f463819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463659c700 1 -- 172.21.15.89:0/1561898870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46280095f0 con 0x7f46381008a0 2023-10-26T06:46:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f4635d9b700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4638101260 0x7f4638199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:39.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463659c700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381008a0 0x7f4638198e50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4628000c00 tx=0x7f4628004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:39.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f46277fe700 1 -- 172.21.15.89:0/1561898870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462801d070 con 0x7f46381008a0 2023-10-26T06:46:39.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f46277fe700 1 -- 172.21.15.89:0/1561898870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4628022470 con 0x7f46381008a0 2023-10-26T06:46:39.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463819e300 con 0x7f46381008a0 2023-10-26T06:46:39.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f46277fe700 1 -- 172.21.15.89:0/1561898870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628017650 con 0x7f46381008a0 2023-10-26T06:46:39.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.279+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46381a5bc0 con 0x7f46381008a0 2023-10-26T06:46:39.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.280+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4638101f30 con 0x7f46381008a0 2023-10-26T06:46:39.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.282+0000 7f46277fe700 1 -- 172.21.15.89:0/1561898870 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f46280225e0 con 0x7f46381008a0 2023-10-26T06:46:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.284+0000 7f46277fe700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46200a1550 0x7f46200a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.285+0000 7f46277fe700 1 -- 172.21.15.89:0/1561898870 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f46280ce0e0 con 0x7f46381008a0 2023-10-26T06:46:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.285+0000 7f4635d9b700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46200a1550 0x7f46200a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:39.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.285+0000 7f4635d9b700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46200a1550 0x7f46200a3a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f463819a3b0 tx=0x7f462c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:39.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.288+0000 7f46277fe700 1 -- 172.21.15.89:0/1561898870 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4628097690 con 0x7f46381008a0 2023-10-26T06:46:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:39 smithi192 ceph-mon[131661]: pgmap v302: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:39 smithi135 ceph-mon[131403]: pgmap v302: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.531+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f463819a160 con 0x7f46381008a0 2023-10-26T06:46:39.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.531+0000 7f46277fe700 1 -- 172.21.15.89:0/1561898870 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7f46280228e0 con 0x7f46381008a0 2023-10-26T06:46:39.534 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T06:46:39.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46200a1550 msgr2=0x7f46200a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:39.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46200a1550 0x7f46200a3a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f463819a3b0 tx=0x7f462c009260 comp rx=0 tx=0).stop 2023-10-26T06:46:39.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381008a0 msgr2=0x7f4638198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:39.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381008a0 0x7f4638198e50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4628000c00 tx=0x7f4628004730 comp rx=0 tx=0).stop 2023-10-26T06:46:39.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 shutdown_connections 2023-10-26T06:46:39.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f46200a1550 0x7f46200a3a10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f46381008a0 0x7f4638198e50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46381057c0 0x7f463819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 --2- 172.21.15.89:0/1561898870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4638101260 0x7f4638199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:39.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.533+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 >> 172.21.15.89:0/1561898870 conn(0x7f46380779e0 msgr2=0x7f463810ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:39.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.534+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 shutdown_connections 2023-10-26T06:46:39.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:39.534+0000 7f463cf82700 1 -- 172.21.15.89:0/1561898870 wait complete. 2023-10-26T06:46:39.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:39 smithi089 ceph-mon[95031]: pgmap v302: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:39.992 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T06:46:39.993 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T06:46:39.993 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2023-10-26T06:46:39.993 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T06:46:39.993 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:39.994 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T06:46:39.994 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T06:46:39.994 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T06:46:39.995 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T06:46:39.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:39.995 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2023-10-26T06:46:40.042 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T06:46:40.042 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T06:46:40.042 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T06:46:40.043 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-26T06:46:40.043 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2023-10-26T06:46:40.043 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-26T06:46:40.043 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T06:46:40.043 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2023-10-26T06:46:40.043 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2023-10-26T06:46:40.044 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T06:46:40.044 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2023-10-26T06:46:40.044 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2023-10-26T06:46:40.044 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T06:46:40.044 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2023-10-26T06:46:40.045 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T06:46:40.045 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2023-10-26T06:46:40.045 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T06:46:40.045 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 41509sec preferred_lft 41509sec 2023-10-26T06:46:40.046 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::e999:b82c:300:16e6/64 scope link noprefixroute 2023-10-26T06:46:40.046 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2023-10-26T06:46:40.046 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-26T06:46:40.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:40.047 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-26T06:46:40.047 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-26T06:46:40.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2023-10-26T06:46:40.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2023-10-26T06:46:40.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2023-10-26T06:46:40.049 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-26T06:46:40.049 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-26T06:46:40.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:40 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1561898870' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-26T06:46:40.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:40 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1561898870' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-26T06:46:40.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:40 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1561898870' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-26T06:46:40.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:40.513 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T06:46:40.584 INFO:teuthology.orchestra.run.smithi089.stdout:1 2023-10-26T06:46:40.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:40.587 DEBUG:teuthology.orchestra.run.smithi089:> route 2023-10-26T06:46:40.650 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2023-10-26T06:46:40.650 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T06:46:40.650 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T06:46:40.651 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T06:46:40.651 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T06:46:40.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:40.652 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-26T06:46:40.653 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-26T06:46:40.653 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T06:46:40.653 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T06:46:40.654 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T06:46:40.654 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-26T06:46:41.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:41 smithi089 ceph-mon[95031]: pgmap v303: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:41.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:41.392 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-26T06:46:41.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:41 smithi192 ceph-mon[131661]: pgmap v303: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:41.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:41.437 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2023-10-26T06:46:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:41 smithi135 ceph-mon[131403]: pgmap v303: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:41.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:41.521 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-26T06:46:41.521 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-26T06:46:41.522 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T06:46:41.522 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-26T06:46:41.522 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-26T06:46:41.706 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-26T06:46:41.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:41.707 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-26T06:46:41.707 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-26T06:46:41.707 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-26T06:46:41.707 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-26T06:46:41.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-26T06:46:41.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-26T06:46:41.708 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-26T06:46:41.709 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-26T06:46:41.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:41.986 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec bash -c ' 2023-10-26T06:46:41.987 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2023-10-26T06:46:41.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2023-10-26T06:46:41.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2023-10-26T06:46:41.987 DEBUG:teuthology.orchestra.run.smithi089:> ') 2023-10-26T06:46:42.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T06:46:42.163 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:42.206 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T06:46:42.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T06:46:42.207 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:42.290 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-26T06:46:42.291 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-26T06:46:42.291 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/4456288b-9087-4ca7-95ed-920f9d752dee 2023-10-26T06:46:42.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:42.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/4456288b-9087-4ca7-95ed-920f9d752dee /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-10-26T06:46:42.441 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-10-26T06:46:42.442 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-10-26T06:46:42.889 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-10-26T06:46:42.890 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-10-26T06:46:42.890 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.135:6789,172.21.15.192:6789" 2023-10-26T06:46:42.890 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with new device syntax: 0@e22b2d1a-73c9-11ee-8db9-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/4456288b-9087-4ca7-95ed-920f9d752dee 2023-10-26T06:46:42.890 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.89:6789/172.21.15.135:6789/172.21.15.192:6789" will pass to kernel 2023-10-26T06:46:42.896 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T06:46:42.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:42.896 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:42.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:42.969 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T06:46:43.067 INFO:teuthology.orchestra.run.smithi089.stdout:client15258 2023-10-26T06:46:43.068 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:46:43.068 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T06:46:43.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:43.113 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T06:46:43.239 INFO:teuthology.orchestra.run.smithi089.stdout:client15258 2023-10-26T06:46:43.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:43.240 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T06:46:43.327 INFO:teuthology.orchestra.run.smithi089.stdout:client15258 2023-10-26T06:46:43.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:46:43.328 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7.client15258/status') 2023-10-26T06:46:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:43 smithi192 ceph-mon[131661]: pgmap v304: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:43.430 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:43 smithi089 ceph-mon[95031]: pgmap v304: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:43.437 INFO:teuthology.orchestra.run.smithi089.stdout:instance: client.15258 (3)192.168.0.1:0/2177709330 2023-10-26T06:46:43.438 INFO:teuthology.orchestra.run.smithi089.stdout:blocklisted: false 2023-10-26T06:46:43.438 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records in 2023-10-26T06:46:43.439 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records out 2023-10-26T06:46:43.439 INFO:teuthology.orchestra.run.smithi089.stderr:70 bytes copied, 0.000103043 s, 679 kB/s 2023-10-26T06:46:43.440 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-26T06:46:43.453 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-26T06:46:43.497 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:43.498 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:43 smithi135 ceph-mon[131403]: pgmap v304: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:43.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 -- 172.21.15.89:0/2512262992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc1066b0 msgr2=0x7f53fc106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:43.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 --2- 172.21.15.89:0/2512262992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc1066b0 0x7f53fc106a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009990 tx=0x7f53ec009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 -- 172.21.15.89:0/2512262992 shutdown_connections 2023-10-26T06:46:43.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 --2- 172.21.15.89:0/2512262992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc1066b0 0x7f53fc106a90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 --2- 172.21.15.89:0/2512262992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53fc101060 0x7f53fc10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 --2- 172.21.15.89:0/2512262992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53fc1006a0 0x7f53fc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 -- 172.21.15.89:0/2512262992 >> 172.21.15.89:0/2512262992 conn(0x7f53fc0fc170 msgr2=0x7f53fc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:43.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 -- 172.21.15.89:0/2512262992 shutdown_connections 2023-10-26T06:46:43.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.695+0000 7f5400bff700 1 -- 172.21.15.89:0/2512262992 wait complete. 2023-10-26T06:46:43.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 Processor -- start 2023-10-26T06:46:43.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 -- start start 2023-10-26T06:46:43.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53fc1006a0 0x7f53fc19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:43.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc101060 0x7f53fc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:43.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53fc1066b0 0x7f53fc196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:43.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc196fa0 con 0x7f53fc101060 2023-10-26T06:46:43.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc197110 con 0x7f53fc1006a0 2023-10-26T06:46:43.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.696+0000 7f5400bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc1972d0 con 0x7f53fc1066b0 2023-10-26T06:46:43.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc101060 0x7f53fc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:43.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc101060 0x7f53fc19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53948/0 (socket says 172.21.15.89:53948) 2023-10-26T06:46:43.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 -- 172.21.15.89:0/2522731411 learned_addr learned my addr 172.21.15.89:0/2522731411 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:43.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53fa59c700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53fc1006a0 0x7f53fc19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:43.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53fad9d700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53fc1066b0 0x7f53fc196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:43.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 -- 172.21.15.89:0/2522731411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53fc1066b0 msgr2=0x7f53fc196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:43.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53fc1066b0 0x7f53fc196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 -- 172.21.15.89:0/2522731411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53fc1006a0 msgr2=0x7f53fc19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:43.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53fc1006a0 0x7f53fc19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 -- 172.21.15.89:0/2522731411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0095f0 con 0x7f53fc101060 2023-10-26T06:46:43.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53f9d9b700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc101060 0x7f53fc19cd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d980 tx=0x7f53f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:43.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53eb7fe700 1 -- 172.21.15.89:0/2522731411 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0004220 con 0x7f53fc101060 2023-10-26T06:46:43.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.697+0000 7f53eb7fe700 1 -- 172.21.15.89:0/2522731411 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f0007640 con 0x7f53fc101060 2023-10-26T06:46:43.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.698+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc1975b0 con 0x7f53fc101060 2023-10-26T06:46:43.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.698+0000 7f53eb7fe700 1 -- 172.21.15.89:0/2522731411 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0005690 con 0x7f53fc101060 2023-10-26T06:46:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.698+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc1a3800 con 0x7f53fc101060 2023-10-26T06:46:43.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.699+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53fc108ca0 con 0x7f53fc101060 2023-10-26T06:46:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.701+0000 7f53eb7fe700 1 -- 172.21.15.89:0/2522731411 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f53f0004750 con 0x7f53fc101060 2023-10-26T06:46:43.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.702+0000 7f53eb7fe700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53e40a14d0 0x7f53e40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.702+0000 7f53eb7fe700 1 -- 172.21.15.89:0/2522731411 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f53f00cd140 con 0x7f53fc101060 2023-10-26T06:46:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.702+0000 7f53fa59c700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53e40a14d0 0x7f53e40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.703+0000 7f53fa59c700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53e40a14d0 0x7f53e40a3990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006010 tx=0x7f53ec00b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:43.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.704+0000 7f53eb7fe700 1 -- 172.21.15.89:0/2522731411 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f53f0096660 con 0x7f53fc101060 2023-10-26T06:46:43.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.909+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53fc101de0 con 0x7f53fc101060 2023-10-26T06:46:43.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.910+0000 7f53eb7fe700 1 -- 172.21.15.89:0/2522731411 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7f53f0016090 con 0x7f53fc101060 2023-10-26T06:46:43.913 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:43.913 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:43.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53e40a14d0 msgr2=0x7f53e40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:43.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53e40a14d0 0x7f53e40a3990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006010 tx=0x7f53ec00b260 comp rx=0 tx=0).stop 2023-10-26T06:46:43.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc101060 msgr2=0x7f53fc19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:43.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc101060 0x7f53fc19cd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d980 tx=0x7f53f000dc90 comp rx=0 tx=0).stop 2023-10-26T06:46:43.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 shutdown_connections 2023-10-26T06:46:43.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f53e40a14d0 0x7f53e40a3990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53fc1066b0 0x7f53fc196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53fc1006a0 0x7f53fc19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 --2- 172.21.15.89:0/2522731411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53fc101060 0x7f53fc19cd50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:43.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 >> 172.21.15.89:0/2522731411 conn(0x7f53fc0fc170 msgr2=0x7f53fc0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:43.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 shutdown_connections 2023-10-26T06:46:43.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:43.913+0000 7f5400bff700 1 -- 172.21.15.89:0/2522731411 wait complete. 2023-10-26T06:46:43.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:46:43.935 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:46:44.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 -- 172.21.15.89:0/4275217882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 msgr2=0x7f9504100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4275217882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f9504100ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009990 tx=0x7f94ec009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 -- 172.21.15.89:0/4275217882 shutdown_connections 2023-10-26T06:46:44.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4275217882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95041010e0 0x7f950410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4275217882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9504106730 0x7f9504106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4275217882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f9504100ba0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 -- 172.21.15.89:0/4275217882 >> 172.21.15.89:0/4275217882 conn(0x7f95040fc1b0 msgr2=0x7f95040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:44.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.136+0000 7f950a1ff700 1 -- 172.21.15.89:0/4275217882 shutdown_connections 2023-10-26T06:46:44.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.137+0000 7f950a1ff700 1 -- 172.21.15.89:0/4275217882 wait complete. 2023-10-26T06:46:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.138+0000 7f950a1ff700 1 Processor -- start 2023-10-26T06:46:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.138+0000 7f950a1ff700 1 -- start start 2023-10-26T06:46:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f950a1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f950419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f950a1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95041010e0 0x7f950419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f950a1ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9504106730 0x7f9504196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f950a1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9504197070 con 0x7f9504100720 2023-10-26T06:46:44.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f950a1ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95041971e0 con 0x7f9504106730 2023-10-26T06:46:44.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f950a1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95041973a0 con 0x7f95041010e0 2023-10-26T06:46:44.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f950419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f950419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53954/0 (socket says 172.21.15.89:53954) 2023-10-26T06:46:44.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 -- 172.21.15.89:0/4114066664 learned_addr learned my addr 172.21.15.89:0/4114066664 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:44.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f9503fff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9504106730 0x7f9504196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f94fbfff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95041010e0 0x7f950419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 -- 172.21.15.89:0/4114066664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95041010e0 msgr2=0x7f950419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95041010e0 0x7f950419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 -- 172.21.15.89:0/4114066664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9504106730 msgr2=0x7f9504196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9504106730 0x7f9504196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f95037fe700 1 -- 172.21.15.89:0/4114066664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec0095f0 con 0x7f9504100720 2023-10-26T06:46:44.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.139+0000 7f9503fff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9504106730 0x7f9504196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:44.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.140+0000 7f95037fe700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f950419c890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94f400ef10 tx=0x7f94f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:44.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.140+0000 7f95017fa700 1 -- 172.21.15.89:0/4114066664 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f400c9c0 con 0x7f9504100720 2023-10-26T06:46:44.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.140+0000 7f95017fa700 1 -- 172.21.15.89:0/4114066664 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f400f040 con 0x7f9504100720 2023-10-26T06:46:44.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.140+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9504197620 con 0x7f9504100720 2023-10-26T06:46:44.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.140+0000 7f95017fa700 1 -- 172.21.15.89:0/4114066664 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4014480 con 0x7f9504100720 2023-10-26T06:46:44.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.140+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95041a3880 con 0x7f9504100720 2023-10-26T06:46:44.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.141+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9504108d40 con 0x7f9504100720 2023-10-26T06:46:44.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.143+0000 7f95017fa700 1 -- 172.21.15.89:0/4114066664 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f94f4007690 con 0x7f9504100720 2023-10-26T06:46:44.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.144+0000 7f95017fa700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f94f00a1550 0x7f94f00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.144+0000 7f95017fa700 1 -- 172.21.15.89:0/4114066664 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f94f40ccff0 con 0x7f9504100720 2023-10-26T06:46:44.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.145+0000 7f94fbfff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f94f00a1550 0x7f94f00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.145+0000 7f94fbfff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f94f00a1550 0x7f94f00a3a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9504197b80 tx=0x7f94ec005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:44.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.148+0000 7f95017fa700 1 -- 172.21.15.89:0/4114066664 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f94f4100100 con 0x7f9504100720 2023-10-26T06:46:44.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:44 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2522731411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:44.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.350+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9504197e80 con 0x7f9504100720 2023-10-26T06:46:44.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.350+0000 7f95017fa700 1 -- 172.21.15.89:0/4114066664 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94f4096530 con 0x7f9504100720 2023-10-26T06:46:44.353 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:46:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.352+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f94f00a1550 msgr2=0x7f94f00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.352+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f94f00a1550 0x7f94f00a3a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9504197b80 tx=0x7f94ec005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.352+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 msgr2=0x7f950419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.352+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f950419c890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94f400ef10 tx=0x7f94f400c3a0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 shutdown_connections 2023-10-26T06:46:44.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f94f00a1550 0x7f94f00a3a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9504100720 0x7f950419c890 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9504106730 0x7f9504196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 --2- 172.21.15.89:0/4114066664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95041010e0 0x7f950419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 >> 172.21.15.89:0/4114066664 conn(0x7f95040fc1b0 msgr2=0x7f95040fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 shutdown_connections 2023-10-26T06:46:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.353+0000 7f950a1ff700 1 -- 172.21.15.89:0/4114066664 wait complete. 2023-10-26T06:46:44.374 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:46:44.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:44 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2522731411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:44.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:44 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2522731411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:44.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 -- 172.21.15.89:0/1734928417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 msgr2=0x7ff12c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 --2- 172.21.15.89:0/1734928417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c100ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff120009990 tx=0x7ff120009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 -- 172.21.15.89:0/1734928417 shutdown_connections 2023-10-26T06:46:44.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 --2- 172.21.15.89:0/1734928417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c1010e0 0x7ff12c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 --2- 172.21.15.89:0/1734928417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff12c106730 0x7ff12c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 --2- 172.21.15.89:0/1734928417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c100ba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 -- 172.21.15.89:0/1734928417 >> 172.21.15.89:0/1734928417 conn(0x7ff12c0fc1b0 msgr2=0x7ff12c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:44.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 -- 172.21.15.89:0/1734928417 shutdown_connections 2023-10-26T06:46:44.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.573+0000 7ff1329ff700 1 -- 172.21.15.89:0/1734928417 wait complete. 2023-10-26T06:46:44.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.574+0000 7ff1329ff700 1 Processor -- start 2023-10-26T06:46:44.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff1329ff700 1 -- start start 2023-10-26T06:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff1329ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff1329ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff12c1010e0 0x7ff12c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff1329ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c106730 0x7ff12c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff1329ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c19dcf0 con 0x7ff12c100720 2023-10-26T06:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff1329ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c19de60 con 0x7ff12c1010e0 2023-10-26T06:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff1329ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c19e020 con 0x7ff12c106730 2023-10-26T06:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.575+0000 7ff12bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53974/0 (socket says 172.21.15.89:53974) 2023-10-26T06:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 -- 172.21.15.89:0/2006027723 learned_addr learned my addr 172.21.15.89:0/2006027723 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12b7fe700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff12c1010e0 0x7ff12c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff130f9c700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c106730 0x7ff12c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 -- 172.21.15.89:0/2006027723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c106730 msgr2=0x7ff12c19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c106730 0x7ff12c19d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 -- 172.21.15.89:0/2006027723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff12c1010e0 msgr2=0x7ff12c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff12c1010e0 0x7ff12c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 -- 172.21.15.89:0/2006027723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200095f0 con 0x7ff12c100720 2023-10-26T06:46:44.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12b7fe700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff12c1010e0 0x7ff12c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:44.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff12bfff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c198dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9a0 tx=0x7ff11c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:44.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff1297fa700 1 -- 172.21.15.89:0/2006027723 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c009790 con 0x7ff12c100720 2023-10-26T06:46:44.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff1297fa700 1 -- 172.21.15.89:0/2006027723 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff11c010430 con 0x7ff12c100720 2023-10-26T06:46:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c19e300 con 0x7ff12c100720 2023-10-26T06:46:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.576+0000 7ff1297fa700 1 -- 172.21.15.89:0/2006027723 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c018680 con 0x7ff12c100720 2023-10-26T06:46:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.577+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c1a5b40 con 0x7ff12c100720 2023-10-26T06:46:44.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.578+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c101e60 con 0x7ff12c100720 2023-10-26T06:46:44.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.579+0000 7ff1297fa700 1 -- 172.21.15.89:0/2006027723 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff11c0105a0 con 0x7ff12c100720 2023-10-26T06:46:44.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.581+0000 7ff1297fa700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff1140a14d0 0x7ff1140a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:44.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.581+0000 7ff1297fa700 1 -- 172.21.15.89:0/2006027723 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff11c014070 con 0x7ff12c100720 2023-10-26T06:46:44.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.581+0000 7ff12b7fe700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff1140a14d0 0x7ff1140a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:44.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.582+0000 7ff12b7fe700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff1140a14d0 0x7ff1140a3990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff12c19a320 tx=0x7ff120005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:44.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.585+0000 7ff1297fa700 1 -- 172.21.15.89:0/2006027723 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff11c052100 con 0x7ff12c100720 2023-10-26T06:46:44.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.785+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff12c19a0d0 con 0x7ff12c100720 2023-10-26T06:46:44.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.785+0000 7ff1297fa700 1 -- 172.21.15.89:0/2006027723 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff11c096510 con 0x7ff12c100720 2023-10-26T06:46:44.788 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:46:44.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff1140a14d0 msgr2=0x7ff1140a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff1140a14d0 0x7ff1140a3990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff12c19a320 tx=0x7ff120005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 msgr2=0x7ff12c198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c198dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9a0 tx=0x7ff11c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 shutdown_connections 2023-10-26T06:46:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff1140a14d0 0x7ff1140a3990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c100720 0x7ff12c198dd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff12c106730 0x7ff12c19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 --2- 172.21.15.89:0/2006027723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff12c1010e0 0x7ff12c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 >> 172.21.15.89:0/2006027723 conn(0x7ff12c0fc1b0 msgr2=0x7ff12c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 shutdown_connections 2023-10-26T06:46:44.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:44.788+0000 7ff1329ff700 1 -- 172.21.15.89:0/2006027723 wait complete. 2023-10-26T06:46:44.809 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:46:45.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 -- 172.21.15.89:0/272309760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72641010e0 msgr2=0x7f726410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 --2- 172.21.15.89:0/272309760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72641010e0 0x7f726410d5f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f725c009f90 tx=0x7f725c00b330 comp rx=0 tx=0).stop 2023-10-26T06:46:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 -- 172.21.15.89:0/272309760 shutdown_connections 2023-10-26T06:46:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 --2- 172.21.15.89:0/272309760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72641010e0 0x7f726410d5f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 --2- 172.21.15.89:0/272309760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7264106730 0x7f7264106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 --2- 172.21.15.89:0/272309760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264100720 0x7f7264100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 -- 172.21.15.89:0/272309760 >> 172.21.15.89:0/272309760 conn(0x7f72640fc1b0 msgr2=0x7f72640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:45.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 -- 172.21.15.89:0/272309760 shutdown_connections 2023-10-26T06:46:45.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.008+0000 7f726b5ff700 1 -- 172.21.15.89:0/272309760 wait complete. 2023-10-26T06:46:45.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.009+0000 7f726b5ff700 1 Processor -- start 2023-10-26T06:46:45.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.009+0000 7f726b5ff700 1 -- start start 2023-10-26T06:46:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726b5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7264100720 0x7f7264198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726b5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72641010e0 0x7f7264199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726b5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264106730 0x7f726419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726b5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726419cb40 con 0x7f7264106730 2023-10-26T06:46:45.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726b5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726419ccb0 con 0x7f7264100720 2023-10-26T06:46:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726b5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726419ce70 con 0x7f72641010e0 2023-10-26T06:46:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7269b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264106730 0x7f726419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726939b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7264100720 0x7f7264198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726939b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7264100720 0x7f7264198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47182/0 (socket says 172.21.15.89:47182) 2023-10-26T06:46:45.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726939b700 1 -- 172.21.15.89:0/3023700447 learned_addr learned my addr 172.21.15.89:0/3023700447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7269b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264106730 0x7f726419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53996/0 (socket says 172.21.15.89:53996) 2023-10-26T06:46:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7268b9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72641010e0 0x7f7264199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7269b9c700 1 -- 172.21.15.89:0/3023700447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72641010e0 msgr2=0x7f7264199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7269b9c700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72641010e0 0x7f7264199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7269b9c700 1 -- 172.21.15.89:0/3023700447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7264100720 msgr2=0x7f7264198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7269b9c700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7264100720 0x7f7264198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f7269b9c700 1 -- 172.21.15.89:0/3023700447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009c40 con 0x7f7264106730 2023-10-26T06:46:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.010+0000 7f726939b700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7264100720 0x7f7264198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.011+0000 7f7269b9c700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264106730 0x7f726419c540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f725c00b6c0 tx=0x7f725c008fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:45.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.011+0000 7f725a7fc700 1 -- 172.21.15.89:0/3023700447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00e040 con 0x7f7264106730 2023-10-26T06:46:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.011+0000 7f725a7fc700 1 -- 172.21.15.89:0/3023700447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f725c01d470 con 0x7f7264106730 2023-10-26T06:46:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.011+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726419d0f0 con 0x7f7264106730 2023-10-26T06:46:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.011+0000 7f725a7fc700 1 -- 172.21.15.89:0/3023700447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0052a0 con 0x7f7264106730 2023-10-26T06:46:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.011+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72641a5ab0 con 0x7f7264106730 2023-10-26T06:46:45.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.012+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7264101e60 con 0x7f7264106730 2023-10-26T06:46:45.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.014+0000 7f725a7fc700 1 -- 172.21.15.89:0/3023700447 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f725c01d5e0 con 0x7f7264106730 2023-10-26T06:46:45.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.015+0000 7f725a7fc700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72500a1550 0x7f72500a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.015+0000 7f725a7fc700 1 -- 172.21.15.89:0/3023700447 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f725c0ce170 con 0x7f7264106730 2023-10-26T06:46:45.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.015+0000 7f726939b700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72500a1550 0x7f72500a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.016+0000 7f726939b700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72500a1550 0x7f72500a3a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7260011550 tx=0x7f7260009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:45.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.019+0000 7f725a7fc700 1 -- 172.21.15.89:0/3023700447 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f725c097720 con 0x7f7264106730 2023-10-26T06:46:45.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.224+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72641106f0 con 0x7f7264106730 2023-10-26T06:46:45.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.225+0000 7f725a7fc700 1 -- 172.21.15.89:0/3023700447 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f725c09ad40 con 0x7f7264106730 2023-10-26T06:46:45.227 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:46:45.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72500a1550 msgr2=0x7f72500a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72500a1550 0x7f72500a3a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7260011550 tx=0x7f7260009260 comp rx=0 tx=0).stop 2023-10-26T06:46:45.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264106730 msgr2=0x7f726419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264106730 0x7f726419c540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f725c00b6c0 tx=0x7f725c008fb0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 shutdown_connections 2023-10-26T06:46:45.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72500a1550 0x7f72500a3a10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7264106730 0x7f726419c540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7264100720 0x7f7264198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 --2- 172.21.15.89:0/3023700447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72641010e0 0x7f7264199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 >> 172.21.15.89:0/3023700447 conn(0x7f72640fc1b0 msgr2=0x7f7264072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:45.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 shutdown_connections 2023-10-26T06:46:45.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.227+0000 7f726b5ff700 1 -- 172.21.15.89:0/3023700447 wait complete. 2023-10-26T06:46:45.249 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:45.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:45 smithi192 ceph-mon[131661]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:45.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:45 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4114066664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:45.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:45 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2006027723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 -- 172.21.15.89:0/3270306214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 msgr2=0x7f91b810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/3270306214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 0x7f91b810c9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009990 tx=0x7f91ac009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 -- 172.21.15.89:0/3270306214 shutdown_connections 2023-10-26T06:46:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/3270306214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 0x7f91b810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/3270306214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b8072c20 0x7f91b8073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/3270306214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 0x7f91b810c9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 -- 172.21.15.89:0/3270306214 >> 172.21.15.89:0/3270306214 conn(0x7f91b80fc170 msgr2=0x7f91b80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 -- 172.21.15.89:0/3270306214 shutdown_connections 2023-10-26T06:46:45.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.445+0000 7f91bc9ff700 1 -- 172.21.15.89:0/3270306214 wait complete. 2023-10-26T06:46:45.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 Processor -- start 2023-10-26T06:46:45.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 -- start start 2023-10-26T06:46:45.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b8072c20 0x7f91b8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 0x7f91b81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 0x7f91b819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819dcd0 con 0x7f91b810cee0 2023-10-26T06:46:45.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819de40 con 0x7f91b8072c20 2023-10-26T06:46:45.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.446+0000 7f91bc9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819e000 con 0x7f91b8073540 2023-10-26T06:46:45.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 0x7f91b819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b5d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 0x7f91b81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 0x7f91b819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54022/0 (socket says 172.21.15.89:54022) 2023-10-26T06:46:45.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 -- 172.21.15.89:0/4283539772 learned_addr learned my addr 172.21.15.89:0/4283539772 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:45.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b659c700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b8072c20 0x7f91b8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 -- 172.21.15.89:0/4283539772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 msgr2=0x7f91b81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 0x7f91b81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 -- 172.21.15.89:0/4283539772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b8072c20 msgr2=0x7f91b8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b8072c20 0x7f91b8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 -- 172.21.15.89:0/4283539772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0095f0 con 0x7f91b810cee0 2023-10-26T06:46:45.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b5d9b700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 0x7f91b81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:45.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91b6d9d700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 0x7f91b819d690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f91b000ef30 tx=0x7f91b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:45.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91a77fe700 1 -- 172.21.15.89:0/4283539772 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0018070 con 0x7f91b810cee0 2023-10-26T06:46:45.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b819e280 con 0x7f91b810cee0 2023-10-26T06:46:45.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91a77fe700 1 -- 172.21.15.89:0/4283539772 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91b000f040 con 0x7f91b810cee0 2023-10-26T06:46:45.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.447+0000 7f91a77fe700 1 -- 172.21.15.89:0/4283539772 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000cd00 con 0x7f91b810cee0 2023-10-26T06:46:45.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.448+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b81a5b20 con 0x7f91b810cee0 2023-10-26T06:46:45.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.449+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b81080f0 con 0x7f91b810cee0 2023-10-26T06:46:45.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.450+0000 7f91a77fe700 1 -- 172.21.15.89:0/4283539772 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f91b0003a00 con 0x7f91b810cee0 2023-10-26T06:46:45.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.452+0000 7f91a77fe700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91a00a14d0 0x7f91a00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.452+0000 7f91a77fe700 1 -- 172.21.15.89:0/4283539772 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f91b00cc960 con 0x7f91b810cee0 2023-10-26T06:46:45.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.452+0000 7f91b659c700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91a00a14d0 0x7f91a00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.453+0000 7f91b659c700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91a00a14d0 0x7f91a00a3990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009c60 tx=0x7f91a8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:45.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.456+0000 7f91a77fe700 1 -- 172.21.15.89:0/4283539772 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91b0095f10 con 0x7f91b810cee0 2023-10-26T06:46:45.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:45 smithi135 ceph-mon[131403]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:45.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:45 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4114066664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:45.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:45 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2006027723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:45.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:45 smithi089 ceph-mon[95031]: pgmap v305: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2023-10-26T06:46:45.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:45 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4114066664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:45.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:45 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2006027723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:45.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.660+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91b819a0b0 con 0x7f91b810cee0 2023-10-26T06:46:45.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.661+0000 7f91a77fe700 1 -- 172.21.15.89:0/4283539772 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7f91b0099530 con 0x7f91b810cee0 2023-10-26T06:46:45.664 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:45.664 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91a00a14d0 msgr2=0x7f91a00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91a00a14d0 0x7f91a00a3990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009c60 tx=0x7f91a8009260 comp rx=0 tx=0).stop 2023-10-26T06:46:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 msgr2=0x7f91b819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 0x7f91b819d690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f91b000ef30 tx=0x7f91b000c3a0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 shutdown_connections 2023-10-26T06:46:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91a00a14d0 0x7f91a00a3990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b810cee0 0x7f91b819d690 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b8072c20 0x7f91b8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 --2- 172.21.15.89:0/4283539772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8073540 0x7f91b81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 >> 172.21.15.89:0/4283539772 conn(0x7f91b80fc170 msgr2=0x7f91b80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:45.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 shutdown_connections 2023-10-26T06:46:45.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.664+0000 7f91bc9ff700 1 -- 172.21.15.89:0/4283539772 wait complete. 2023-10-26T06:46:45.685 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:46:45.685 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:46:45.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 -- 172.21.15.89:0/3197331308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 msgr2=0x7f42cc100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3197331308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 0x7f42cc100ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42b40098f0 tx=0x7f42b4009c00 comp rx=0 tx=0).stop 2023-10-26T06:46:45.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 -- 172.21.15.89:0/3197331308 shutdown_connections 2023-10-26T06:46:45.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3197331308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 0x7f42cc106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3197331308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42cc1010e0 0x7f42cc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3197331308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 0x7f42cc100ba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 -- 172.21.15.89:0/3197331308 >> 172.21.15.89:0/3197331308 conn(0x7f42cc0fc1b0 msgr2=0x7f42cc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:45.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 -- 172.21.15.89:0/3197331308 shutdown_connections 2023-10-26T06:46:45.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.885+0000 7f42d1dff700 1 -- 172.21.15.89:0/3197331308 wait complete. 2023-10-26T06:46:45.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.886+0000 7f42d1dff700 1 Processor -- start 2023-10-26T06:46:45.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42d1dff700 1 -- start start 2023-10-26T06:46:45.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42d1dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 0x7f42cc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42d1dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42cc1010e0 0x7f42cc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42d1dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 0x7f42cc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42d1dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc19dcd0 con 0x7f42cc106730 2023-10-26T06:46:45.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42d1dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc19de40 con 0x7f42cc100720 2023-10-26T06:46:45.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42d1dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc19e000 con 0x7f42cc1010e0 2023-10-26T06:46:45.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 0x7f42cc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 0x7f42cc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 0x7f42cc19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54042/0 (socket says 172.21.15.89:54042) 2023-10-26T06:46:45.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.887+0000 7f42cbfff700 1 -- 172.21.15.89:0/3365391443 learned_addr learned my addr 172.21.15.89:0/3365391443 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:45.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42c3fff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42cc1010e0 0x7f42cc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42cbfff700 1 -- 172.21.15.89:0/3365391443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42cc1010e0 msgr2=0x7f42cc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42cbfff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42cc1010e0 0x7f42cc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42cbfff700 1 -- 172.21.15.89:0/3365391443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 msgr2=0x7f42cc198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:45.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42cbfff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 0x7f42cc198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:45.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42cbfff700 1 -- 172.21.15.89:0/3365391443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc0095f0 con 0x7f42cc106730 2023-10-26T06:46:45.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42cb7fe700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 0x7f42cc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:45.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42cbfff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 0x7f42cc19d690 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42c400ef30 tx=0x7f42c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:45.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42c97fa700 1 -- 172.21.15.89:0/3365391443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4018070 con 0x7f42cc106730 2023-10-26T06:46:45.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b4009520 con 0x7f42cc106730 2023-10-26T06:46:45.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42c97fa700 1 -- 172.21.15.89:0/3365391443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c400f040 con 0x7f42cc106730 2023-10-26T06:46:45.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42c97fa700 1 -- 172.21.15.89:0/3365391443 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400ccc0 con 0x7f42cc106730 2023-10-26T06:46:45.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.888+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc1a5940 con 0x7f42cc106730 2023-10-26T06:46:45.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.890+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc101e60 con 0x7f42cc106730 2023-10-26T06:46:45.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.892+0000 7f42c97fa700 1 -- 172.21.15.89:0/3365391443 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f42c4003a00 con 0x7f42cc106730 2023-10-26T06:46:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.893+0000 7f42c97fa700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f42b80a14d0 0x7f42b80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.893+0000 7f42c97fa700 1 -- 172.21.15.89:0/3365391443 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f42c40cc940 con 0x7f42cc106730 2023-10-26T06:46:45.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.893+0000 7f42cb7fe700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f42b80a14d0 0x7f42b80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.894+0000 7f42cb7fe700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f42b80a14d0 0x7f42b80a3990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005fd0 tx=0x7f42bc009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:45.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:45.897+0000 7f42c97fa700 1 -- 172.21.15.89:0/3365391443 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f42c4095f50 con 0x7f42cc106730 2023-10-26T06:46:46.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.098+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42cc19a150 con 0x7f42cc106730 2023-10-26T06:46:46.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.099+0000 7f42c97fa700 1 -- 172.21.15.89:0/3365391443 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42c4099570 con 0x7f42cc106730 2023-10-26T06:46:46.102 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:46:46.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f42b80a14d0 msgr2=0x7f42b80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f42b80a14d0 0x7f42b80a3990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005fd0 tx=0x7f42bc009310 comp rx=0 tx=0).stop 2023-10-26T06:46:46.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 msgr2=0x7f42cc19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 0x7f42cc19d690 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42c400ef30 tx=0x7f42c400c3a0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 shutdown_connections 2023-10-26T06:46:46.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f42b80a14d0 0x7f42b80a3990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42cc106730 0x7f42cc19d690 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42cc100720 0x7f42cc198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 --2- 172.21.15.89:0/3365391443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42cc1010e0 0x7f42cc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.101+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 >> 172.21.15.89:0/3365391443 conn(0x7f42cc0fc1b0 msgr2=0x7f42cc10be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:46.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.102+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 shutdown_connections 2023-10-26T06:46:46.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.102+0000 7f42d1dff700 1 -- 172.21.15.89:0/3365391443 wait complete. 2023-10-26T06:46:46.122 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:46:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2073671456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048100720 msgr2=0x7fb048100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2073671456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048100720 0x7fb048100ba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb038009990 tx=0x7fb038009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2073671456 shutdown_connections 2023-10-26T06:46:46.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2073671456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0481010e0 0x7fb04810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2073671456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb048106730 0x7fb048106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2073671456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048100720 0x7fb048100ba0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2073671456 >> 172.21.15.89:0/2073671456 conn(0x7fb0480fc1b0 msgr2=0x7fb0480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2073671456 shutdown_connections 2023-10-26T06:46:46.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.322+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2073671456 wait complete. 2023-10-26T06:46:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.323+0000 7fb04f9ff700 1 Processor -- start 2023-10-26T06:46:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04f9ff700 1 -- start start 2023-10-26T06:46:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04f9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb048100720 0x7fb04819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04f9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0481010e0 0x7fb04819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04f9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048106730 0x7fb048198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04f9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0481991f0 con 0x7fb048106730 2023-10-26T06:46:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04f9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048199360 con 0x7fb048100720 2023-10-26T06:46:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04f9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048199520 con 0x7fb0481010e0 2023-10-26T06:46:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04d79b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb048100720 0x7fb04819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04df9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048106730 0x7fb048198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0481010e0 0x7fb04819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04df9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048106730 0x7fb048198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54052/0 (socket says 172.21.15.89:54052) 2023-10-26T06:46:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04df9c700 1 -- 172.21.15.89:0/2799124327 learned_addr learned my addr 172.21.15.89:0/2799124327 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04d79b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb048100720 0x7fb04819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47252/0 (socket says 172.21.15.89:47252) 2023-10-26T06:46:46.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.324+0000 7fb04cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0481010e0 0x7fb04819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:35868/0 (socket says 172.21.15.89:35868) 2023-10-26T06:46:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04df9c700 1 -- 172.21.15.89:0/2799124327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0481010e0 msgr2=0x7fb04819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04df9c700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0481010e0 0x7fb04819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04df9c700 1 -- 172.21.15.89:0/2799124327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb048100720 msgr2=0x7fb04819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04df9c700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb048100720 0x7fb04819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04df9c700 1 -- 172.21.15.89:0/2799124327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380095f0 con 0x7fb048106730 2023-10-26T06:46:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04df9c700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048106730 0x7fb048198b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb04000ef30 tx=0x7fb04000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb03e7fc700 1 -- 172.21.15.89:0/2799124327 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040018070 con 0x7fb048106730 2023-10-26T06:46:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb03e7fc700 1 -- 172.21.15.89:0/2799124327 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04000f040 con 0x7fb048106730 2023-10-26T06:46:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0481997a0 con 0x7fb048106730 2023-10-26T06:46:46.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb03e7fc700 1 -- 172.21.15.89:0/2799124327 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000cd00 con 0x7fb048106730 2023-10-26T06:46:46.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.325+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0481a5a50 con 0x7fb048106730 2023-10-26T06:46:46.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.326+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb048101e60 con 0x7fb048106730 2023-10-26T06:46:46.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.328+0000 7fb03e7fc700 1 -- 172.21.15.89:0/2799124327 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb040003a00 con 0x7fb048106730 2023-10-26T06:46:46.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.330+0000 7fb03e7fc700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb0340a14d0 0x7fb0340a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.330+0000 7fb03e7fc700 1 -- 172.21.15.89:0/2799124327 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb0400cca10 con 0x7fb048106730 2023-10-26T06:46:46.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.330+0000 7fb04d79b700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb0340a14d0 0x7fb0340a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.330+0000 7fb04d79b700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb0340a14d0 0x7fb0340a3990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb044009be0 tx=0x7fb044009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:46.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.333+0000 7fb03e7fc700 1 -- 172.21.15.89:0/2799124327 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb040095fc0 con 0x7fb048106730 2023-10-26T06:46:46.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:46 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3023700447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:46.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:46 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4283539772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:46.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:46 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3365391443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.534+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb04819a080 con 0x7fb048106730 2023-10-26T06:46:46.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.535+0000 7fb03e7fc700 1 -- 172.21.15.89:0/2799124327 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0400995e0 con 0x7fb048106730 2023-10-26T06:46:46.538 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:46:46.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:46 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3023700447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:46.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:46 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4283539772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:46.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:46 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3365391443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:46.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.537+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb0340a14d0 msgr2=0x7fb0340a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.537+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb0340a14d0 0x7fb0340a3990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb044009be0 tx=0x7fb044009260 comp rx=0 tx=0).stop 2023-10-26T06:46:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.537+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048106730 msgr2=0x7fb048198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.537+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048106730 0x7fb048198b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb04000ef30 tx=0x7fb04000c3a0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.537+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 shutdown_connections 2023-10-26T06:46:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.538+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb0340a14d0 0x7fb0340a3990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.538+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb048106730 0x7fb048198b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.538+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb048100720 0x7fb04819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.538+0000 7fb04f9ff700 1 --2- 172.21.15.89:0/2799124327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0481010e0 0x7fb04819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.538+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 >> 172.21.15.89:0/2799124327 conn(0x7fb0480fc1b0 msgr2=0x7fb0480fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:46.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.538+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 shutdown_connections 2023-10-26T06:46:46.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.538+0000 7fb04f9ff700 1 -- 172.21.15.89:0/2799124327 wait complete. 2023-10-26T06:46:46.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:46:46.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:46 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3023700447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:46.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:46 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4283539772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:46.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:46 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3365391443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 -- 172.21.15.89:0/624332743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 msgr2=0x7fa67810d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 --2- 172.21.15.89:0/624332743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa67810d450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa67000b0d0 tx=0x7fa67000b3e0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 -- 172.21.15.89:0/624332743 shutdown_connections 2023-10-26T06:46:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 --2- 172.21.15.89:0/624332743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa67810d450 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 --2- 172.21.15.89:0/624332743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa678106690 0x7fa678106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 --2- 172.21.15.89:0/624332743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6781005d0 0x7fa678100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 -- 172.21.15.89:0/624332743 >> 172.21.15.89:0/624332743 conn(0x7fa6780fc0c0 msgr2=0x7fa6780fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 -- 172.21.15.89:0/624332743 shutdown_connections 2023-10-26T06:46:46.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.758+0000 7fa67f19d700 1 -- 172.21.15.89:0/624332743 wait complete. 2023-10-26T06:46:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.759+0000 7fa67f19d700 1 Processor -- start 2023-10-26T06:46:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.759+0000 7fa67f19d700 1 -- start start 2023-10-26T06:46:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67f19d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6781005d0 0x7fa678198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67f19d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa678199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67f19d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa678106690 0x7fa67819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67f19d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67819dce0 con 0x7fa678100f90 2023-10-26T06:46:46.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67f19d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67819de50 con 0x7fa678106690 2023-10-26T06:46:46.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67f19d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67819e010 con 0x7fa6781005d0 2023-10-26T06:46:46.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa678199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa678199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54064/0 (socket says 172.21.15.89:54064) 2023-10-26T06:46:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 -- 172.21.15.89:0/2977270177 learned_addr learned my addr 172.21.15.89:0/2977270177 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67e19b700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6781005d0 0x7fa678198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67e99c700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa678106690 0x7fa67819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 -- 172.21.15.89:0/2977270177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6781005d0 msgr2=0x7fa678198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6781005d0 0x7fa678198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 -- 172.21.15.89:0/2977270177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa678106690 msgr2=0x7fa67819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa678106690 0x7fa67819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.760+0000 7fa67d99a700 1 -- 172.21.15.89:0/2977270177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa670009d00 con 0x7fa678100f90 2023-10-26T06:46:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.761+0000 7fa67d99a700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa678199300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d950 tx=0x7fa66800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.761+0000 7fa66f7fe700 1 -- 172.21.15.89:0/2977270177 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800f890 con 0x7fa678100f90 2023-10-26T06:46:46.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.761+0000 7fa66f7fe700 1 -- 172.21.15.89:0/2977270177 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa668010460 con 0x7fa678100f90 2023-10-26T06:46:46.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.761+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67819e2f0 con 0x7fa678100f90 2023-10-26T06:46:46.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.761+0000 7fa66f7fe700 1 -- 172.21.15.89:0/2977270177 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800e610 con 0x7fa678100f90 2023-10-26T06:46:46.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.761+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6781a5b30 con 0x7fa678100f90 2023-10-26T06:46:46.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.762+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa678101cf0 con 0x7fa678100f90 2023-10-26T06:46:46.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.764+0000 7fa66f7fe700 1 -- 172.21.15.89:0/2977270177 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fa6680108d0 con 0x7fa678100f90 2023-10-26T06:46:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.766+0000 7fa66f7fe700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6640a14d0 0x7fa6640a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.766+0000 7fa66f7fe700 1 -- 172.21.15.89:0/2977270177 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa6680ccc30 con 0x7fa678100f90 2023-10-26T06:46:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.766+0000 7fa67e19b700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6640a14d0 0x7fa6640a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:46.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.767+0000 7fa67e19b700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6640a14d0 0x7fa6640a3990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa674007b60 tx=0x7fa6740073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:46.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.770+0000 7fa66f7fe700 1 -- 172.21.15.89:0/2977270177 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa668096150 con 0x7fa678100f90 2023-10-26T06:46:46.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.972+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa678110550 con 0x7fa678100f90 2023-10-26T06:46:46.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.973+0000 7fa66f7fe700 1 -- 172.21.15.89:0/2977270177 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6680160a0 con 0x7fa678100f90 2023-10-26T06:46:46.975 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:46:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6640a14d0 msgr2=0x7fa6640a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6640a14d0 0x7fa6640a3990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa674007b60 tx=0x7fa6740073d0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 msgr2=0x7fa678199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa678199300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d950 tx=0x7fa66800dc60 comp rx=0 tx=0).stop 2023-10-26T06:46:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 shutdown_connections 2023-10-26T06:46:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6640a14d0 0x7fa6640a3990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa678106690 0x7fa67819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6781005d0 0x7fa678198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 --2- 172.21.15.89:0/2977270177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa678100f90 0x7fa678199300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 >> 172.21.15.89:0/2977270177 conn(0x7fa6780fc0c0 msgr2=0x7fa6780fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 shutdown_connections 2023-10-26T06:46:46.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:46.975+0000 7fa67f19d700 1 -- 172.21.15.89:0/2977270177 wait complete. 2023-10-26T06:46:46.996 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-26T06:46:46.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:46:47.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.193+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/543245785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 msgr2=0x7fb8a4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.193+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/543245785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4100b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb894009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.194+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/543245785 shutdown_connections 2023-10-26T06:46:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.194+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/543245785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8a4101060 0x7fb8a410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.194+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/543245785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8a41066b0 0x7fb8a4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.194+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/543245785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4100b20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.194+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/543245785 >> 172.21.15.89:0/543245785 conn(0x7fb8a40fc1d0 msgr2=0x7fb8a40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:47.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.194+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/543245785 shutdown_connections 2023-10-26T06:46:47.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.194+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/543245785 wait complete. 2023-10-26T06:46:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 Processor -- start 2023-10-26T06:46:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 -- start start 2023-10-26T06:46:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8a4101060 0x7fb8a4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8a41066b0 0x7fb8a4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41a4f10 con 0x7fb8a41006a0 2023-10-26T06:46:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41a5080 con 0x7fb8a41066b0 2023-10-26T06:46:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8ab5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41a5240 con 0x7fb8a4101060 2023-10-26T06:46:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8a939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.195+0000 7fb8a939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54092/0 (socket says 172.21.15.89:54092) 2023-10-26T06:46:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a939b700 1 -- 172.21.15.89:0/2398269996 learned_addr learned my addr 172.21.15.89:0/2398269996 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a8b9a700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8a4101060 0x7fb8a4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a9b9c700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8a41066b0 0x7fb8a4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a939b700 1 -- 172.21.15.89:0/2398269996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8a4101060 msgr2=0x7fb8a4073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a939b700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8a4101060 0x7fb8a4073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a939b700 1 -- 172.21.15.89:0/2398269996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8a41066b0 msgr2=0x7fb8a4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a939b700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8a41066b0 0x7fb8a4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a939b700 1 -- 172.21.15.89:0/2398269996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940095f0 con 0x7fb8a41006a0 2023-10-26T06:46:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8a939b700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4072af0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000e9a0 tx=0x7fb8a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb89a7fc700 1 -- 172.21.15.89:0/2398269996 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0009790 con 0x7fb8a41006a0 2023-10-26T06:46:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a41a5520 con 0x7fb8a41006a0 2023-10-26T06:46:47.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb89a7fc700 1 -- 172.21.15.89:0/2398269996 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8a0010430 con 0x7fb8a41006a0 2023-10-26T06:46:47.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.196+0000 7fb89a7fc700 1 -- 172.21.15.89:0/2398269996 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0018680 con 0x7fb8a41006a0 2023-10-26T06:46:47.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.197+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a41a5a30 con 0x7fb8a41006a0 2023-10-26T06:46:47.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.197+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a4108c30 con 0x7fb8a41006a0 2023-10-26T06:46:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.199+0000 7fb89a7fc700 1 -- 172.21.15.89:0/2398269996 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb8a00105a0 con 0x7fb8a41006a0 2023-10-26T06:46:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.201+0000 7fb89a7fc700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb8900a14d0 0x7fb8900a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.201+0000 7fb89a7fc700 1 -- 172.21.15.89:0/2398269996 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb8a0014070 con 0x7fb8a41006a0 2023-10-26T06:46:47.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.201+0000 7fb8a8b9a700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb8900a14d0 0x7fb8900a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.201+0000 7fb8a8b9a700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb8900a14d0 0x7fb8900a3990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4074220 tx=0x7fb894005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:47.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.204+0000 7fb89a7fc700 1 -- 172.21.15.89:0/2398269996 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb8a0052100 con 0x7fb8a41006a0 2023-10-26T06:46:47.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:47 smithi089 ceph-mon[95031]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:46:47.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:47 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2799124327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:47.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:47 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2977270177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.454+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8a4101de0 con 0x7fb8a41006a0 2023-10-26T06:46:47.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.454+0000 7fb89a7fc700 1 -- 172.21.15.89:0/2398269996 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fb8a0096510 con 0x7fb8a41006a0 2023-10-26T06:46:47.457 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:47.457 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:46:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb8900a14d0 msgr2=0x7fb8900a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb8900a14d0 0x7fb8900a3990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4074220 tx=0x7fb894005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 msgr2=0x7fb8a4072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4072af0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000e9a0 tx=0x7fb8a000ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 shutdown_connections 2023-10-26T06:46:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb8900a14d0 0x7fb8900a3990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb8a41006a0 0x7fb8a4072af0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8a41066b0 0x7fb8a4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 --2- 172.21.15.89:0/2398269996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8a4101060 0x7fb8a4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 >> 172.21.15.89:0/2398269996 conn(0x7fb8a40fc1d0 msgr2=0x7fb8a40fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 shutdown_connections 2023-10-26T06:46:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.457+0000 7fb8ab5ff700 1 -- 172.21.15.89:0/2398269996 wait complete. 2023-10-26T06:46:47.461 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:47 smithi135 ceph-mon[131403]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:46:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:47 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2799124327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:47.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:47 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2977270177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:47.478 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-26T06:46:47.478 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-26T06:46:47.479 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T06:46:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:47 smithi192 ceph-mon[131661]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:46:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:47 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2799124327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:47.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:47 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2977270177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 -- 172.21.15.89:0/3778783490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841006a0 msgr2=0x7faf84100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 --2- 172.21.15.89:0/3778783490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841006a0 0x7faf84100b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf740098f0 tx=0x7faf74009c00 comp rx=0 tx=0).stop 2023-10-26T06:46:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 -- 172.21.15.89:0/3778783490 shutdown_connections 2023-10-26T06:46:47.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 --2- 172.21.15.89:0/3778783490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 0x7faf8410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 --2- 172.21.15.89:0/3778783490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf841066b0 0x7faf84106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 --2- 172.21.15.89:0/3778783490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841006a0 0x7faf84100b20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 -- 172.21.15.89:0/3778783490 >> 172.21.15.89:0/3778783490 conn(0x7faf840fc170 msgr2=0x7faf840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 -- 172.21.15.89:0/3778783490 shutdown_connections 2023-10-26T06:46:47.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.675+0000 7faf8bdff700 1 -- 172.21.15.89:0/3778783490 wait complete. 2023-10-26T06:46:47.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.676+0000 7faf8bdff700 1 Processor -- start 2023-10-26T06:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8bdff700 1 -- start start 2023-10-26T06:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8bdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf841006a0 0x7faf84072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8bdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 0x7faf84073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8bdff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841066b0 0x7faf84073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8bdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf841a4ef0 con 0x7faf84101060 2023-10-26T06:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8bdff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf841a5060 con 0x7faf841066b0 2023-10-26T06:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8bdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf841a5220 con 0x7faf841006a0 2023-10-26T06:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8939a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 0x7faf84073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8939a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 0x7faf84073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54108/0 (socket says 172.21.15.89:54108) 2023-10-26T06:46:47.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8a39c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841066b0 0x7faf84073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf8939a700 1 -- 172.21.15.89:0/2528629504 learned_addr learned my addr 172.21.15.89:0/2528629504 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:47.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.677+0000 7faf89b9b700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf841006a0 0x7faf84072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8939a700 1 -- 172.21.15.89:0/2528629504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf841006a0 msgr2=0x7faf84072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8939a700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf841006a0 0x7faf84072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8939a700 1 -- 172.21.15.89:0/2528629504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841066b0 msgr2=0x7faf84073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8939a700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841066b0 0x7faf84073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8939a700 1 -- 172.21.15.89:0/2528629504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf74009520 con 0x7faf84101060 2023-10-26T06:46:47.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8939a700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 0x7faf84073030 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faf74005950 tx=0x7faf7400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf7affd700 1 -- 172.21.15.89:0/2528629504 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7401c070 con 0x7faf84101060 2023-10-26T06:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf7affd700 1 -- 172.21.15.89:0/2528629504 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf7400baa0 con 0x7faf84101060 2023-10-26T06:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf841a54a0 con 0x7faf84101060 2023-10-26T06:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf7affd700 1 -- 172.21.15.89:0/2528629504 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf740038e0 con 0x7faf84101060 2023-10-26T06:46:47.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.678+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf841a5980 con 0x7faf84101060 2023-10-26T06:46:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.679+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf84108ca0 con 0x7faf84101060 2023-10-26T06:46:47.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.681+0000 7faf7affd700 1 -- 172.21.15.89:0/2528629504 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faf74020030 con 0x7faf84101060 2023-10-26T06:46:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.682+0000 7faf7affd700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf700a14d0 0x7faf700a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.683+0000 7faf7affd700 1 -- 172.21.15.89:0/2528629504 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7faf740cdba0 con 0x7faf84101060 2023-10-26T06:46:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.683+0000 7faf89b9b700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf700a14d0 0x7faf700a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:47.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.683+0000 7faf89b9b700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf700a14d0 0x7faf700a3990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf80009be0 tx=0x7faf80009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:47.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.686+0000 7faf7affd700 1 -- 172.21.15.89:0/2528629504 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faf74097150 con 0x7faf84101060 2023-10-26T06:46:47.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.887+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf84073fd0 con 0x7faf700a14d0 2023-10-26T06:46:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.899+0000 7faf7affd700 1 -- 172.21.15.89:0/2528629504 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344469 (secure 0 0 0) 0x7faf84073fd0 con 0x7faf700a14d0 2023-10-26T06:46:47.903 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:47.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.903+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf700a14d0 msgr2=0x7faf700a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.903+0000 7faf8bdff700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf700a14d0 0x7faf700a3990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf80009be0 tx=0x7faf80009260 comp rx=0 tx=0).stop 2023-10-26T06:46:47.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.903+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 msgr2=0x7faf84073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:47.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.903+0000 7faf8bdff700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 0x7faf84073030 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faf74005950 tx=0x7faf7400fa00 comp rx=0 tx=0).stop 2023-10-26T06:46:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 shutdown_connections 2023-10-26T06:46:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf700a14d0 0x7faf700a3990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf841066b0 0x7faf84073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf841006a0 0x7faf84072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 --2- 172.21.15.89:0/2528629504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf84101060 0x7faf84073030 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:47.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 >> 172.21.15.89:0/2528629504 conn(0x7faf840fc170 msgr2=0x7faf840fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:47.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 shutdown_connections 2023-10-26T06:46:47.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:47.904+0000 7faf8bdff700 1 -- 172.21.15.89:0/2528629504 wait complete. 2023-10-26T06:46:47.913 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-26T06:46:47.926 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-26T06:46:47.738531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655931,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":350,"num_read_kb":291,"num_write":260,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":706,"ondisk_log_size":706,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337208,"kb_used_data":13016,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604616,"statfs":{"total":1151940427776,"available":1151595126784,"internally_reserved":0,"allocated":13328384,"data_stored":7510603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002855"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782197+0000","last_change":"2023-10-26T06:45:05.903170+0000","last_active":"2023-10-26T06:46:31.782197+0000","last_peered":"2023-10-26T06:46:31.782197+0000","last_clean":"2023-10-26T06:46:31.782197+0000","last_became_active":"2023-10-26T06:45:05.902574+0000","last_became_peered":"2023-10-26T06:45:05.902574+0000","last_unstale":"2023-10-26T06:46:31.782197+0000","last_undegraded":"2023-10-26T06:46:31.782197+0000","last_fullsized":"2023-10-26T06:46:31.782197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:33.065828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778600+0000","last_change":"2023-10-26T06:45:05.911249+0000","last_active":"2023-10-26T06:46:31.778600+0000","last_peered":"2023-10-26T06:46:31.778600+0000","last_clean":"2023-10-26T06:46:31.778600+0000","last_became_active":"2023-10-26T06:45:05.910859+0000","last_became_peered":"2023-10-26T06:45:05.910859+0000","last_unstale":"2023-10-26T06:46:31.778600+0000","last_undegraded":"2023-10-26T06:46:31.778600+0000","last_fullsized":"2023-10-26T06:46:31.778600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:07:13.362460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780029+0000","last_change":"2023-10-26T06:45:05.909219+0000","last_active":"2023-10-26T06:46:31.780029+0000","last_peered":"2023-10-26T06:46:31.780029+0000","last_clean":"2023-10-26T06:46:31.780029+0000","last_became_active":"2023-10-26T06:45:05.906721+0000","last_became_peered":"2023-10-26T06:45:05.906721+0000","last_unstale":"2023-10-26T06:46:31.780029+0000","last_undegraded":"2023-10-26T06:46:31.780029+0000","last_fullsized":"2023-10-26T06:46:31.780029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:41.461357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.762023+0000","last_change":"2023-10-26T06:45:05.897514+0000","last_active":"2023-10-26T06:46:31.762023+0000","last_peered":"2023-10-26T06:46:31.762023+0000","last_clean":"2023-10-26T06:46:31.762023+0000","last_became_active":"2023-10-26T06:45:05.897201+0000","last_became_peered":"2023-10-26T06:45:05.897201+0000","last_unstale":"2023-10-26T06:46:31.762023+0000","last_undegraded":"2023-10-26T06:46:31.762023+0000","last_fullsized":"2023-10-26T06:46:31.762023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:39.508654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780489+0000","last_change":"2023-10-26T06:45:05.900475+0000","last_active":"2023-10-26T06:46:31.780489+0000","last_peered":"2023-10-26T06:46:31.780489+0000","last_clean":"2023-10-26T06:46:31.780489+0000","last_became_active":"2023-10-26T06:45:05.900313+0000","last_became_peered":"2023-10-26T06:45:05.900313+0000","last_unstale":"2023-10-26T06:46:31.780489+0000","last_undegraded":"2023-10-26T06:46:31.780489+0000","last_fullsized":"2023-10-26T06:46:31.780489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:51.438741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764631+0000","last_change":"2023-10-26T06:45:05.909040+0000","last_active":"2023-10-26T06:46:31.764631+0000","last_peered":"2023-10-26T06:46:31.764631+0000","last_clean":"2023-10-26T06:46:31.764631+0000","last_became_active":"2023-10-26T06:45:05.908696+0000","last_became_peered":"2023-10-26T06:45:05.908696+0000","last_unstale":"2023-10-26T06:46:31.764631+0000","last_undegraded":"2023-10-26T06:46:31.764631+0000","last_fullsized":"2023-10-26T06:46:31.764631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:00.147712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764627+0000","last_change":"2023-10-26T06:45:05.898356+0000","last_active":"2023-10-26T06:46:31.764627+0000","last_peered":"2023-10-26T06:46:31.764627+0000","last_clean":"2023-10-26T06:46:31.764627+0000","last_became_active":"2023-10-26T06:45:05.898008+0000","last_became_peered":"2023-10-26T06:45:05.898008+0000","last_unstale":"2023-10-26T06:46:31.764627+0000","last_undegraded":"2023-10-26T06:46:31.764627+0000","last_fullsized":"2023-10-26T06:46:31.764627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:23.209987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781661+0000","last_change":"2023-10-26T06:45:05.911301+0000","last_active":"2023-10-26T06:46:31.781661+0000","last_peered":"2023-10-26T06:46:31.781661+0000","last_clean":"2023-10-26T06:46:31.781661+0000","last_became_active":"2023-10-26T06:45:05.910667+0000","last_became_peered":"2023-10-26T06:45:05.910667+0000","last_unstale":"2023-10-26T06:46:31.781661+0000","last_undegraded":"2023-10-26T06:46:31.781661+0000","last_fullsized":"2023-10-26T06:46:31.781661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:03:36.856248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.777770+0000","last_change":"2023-10-26T06:45:05.904456+0000","last_active":"2023-10-26T06:46:31.777770+0000","last_peered":"2023-10-26T06:46:31.777770+0000","last_clean":"2023-10-26T06:46:31.777770+0000","last_became_active":"2023-10-26T06:45:05.904165+0000","last_became_peered":"2023-10-26T06:45:05.904165+0000","last_unstale":"2023-10-26T06:46:31.777770+0000","last_undegraded":"2023-10-26T06:46:31.777770+0000","last_fullsized":"2023-10-26T06:46:31.777770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:51:01.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.768918+0000","last_change":"2023-10-26T06:46:03.232421+0000","last_active":"2023-10-26T06:46:31.768918+0000","last_peered":"2023-10-26T06:46:31.768918+0000","last_clean":"2023-10-26T06:46:31.768918+0000","last_became_active":"2023-10-26T06:46:03.232019+0000","last_became_peered":"2023-10-26T06:46:03.232019+0000","last_unstale":"2023-10-26T06:46:31.768918+0000","last_undegraded":"2023-10-26T06:46:31.768918+0000","last_fullsized":"2023-10-26T06:46:31.768918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:56.506294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779141+0000","last_change":"2023-10-26T06:45:05.906549+0000","last_active":"2023-10-26T06:46:31.779141+0000","last_peered":"2023-10-26T06:46:31.779141+0000","last_clean":"2023-10-26T06:46:31.779141+0000","last_became_active":"2023-10-26T06:45:05.905883+0000","last_became_peered":"2023-10-26T06:45:05.905883+0000","last_unstale":"2023-10-26T06:46:31.779141+0000","last_undegraded":"2023-10-26T06:46:31.779141+0000","last_fullsized":"2023-10-26T06:46:31.779141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:54.771031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763970+0000","last_change":"2023-10-26T06:46:03.230365+0000","last_active":"2023-10-26T06:46:31.763970+0000","last_peered":"2023-10-26T06:46:31.763970+0000","last_clean":"2023-10-26T06:46:31.763970+0000","last_became_active":"2023-10-26T06:46:03.230101+0000","last_became_peered":"2023-10-26T06:46:03.230101+0000","last_unstale":"2023-10-26T06:46:31.763970+0000","last_undegraded":"2023-10-26T06:46:31.763970+0000","last_fullsized":"2023-10-26T06:46:31.763970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:13.903128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779665+0000","last_change":"2023-10-26T06:45:05.908305+0000","last_active":"2023-10-26T06:46:31.779665+0000","last_peered":"2023-10-26T06:46:31.779665+0000","last_clean":"2023-10-26T06:46:31.779665+0000","last_became_active":"2023-10-26T06:45:05.907931+0000","last_became_peered":"2023-10-26T06:45:05.907931+0000","last_unstale":"2023-10-26T06:46:31.779665+0000","last_undegraded":"2023-10-26T06:46:31.779665+0000","last_fullsized":"2023-10-26T06:46:31.779665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:00:43.605324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.769635+0000","last_change":"2023-10-26T06:45:05.909737+0000","last_active":"2023-10-26T06:46:31.769635+0000","last_peered":"2023-10-26T06:46:31.769635+0000","last_clean":"2023-10-26T06:46:31.769635+0000","last_became_active":"2023-10-26T06:45:05.909445+0000","last_became_peered":"2023-10-26T06:45:05.909445+0000","last_unstale":"2023-10-26T06:46:31.769635+0000","last_undegraded":"2023-10-26T06:46:31.769635+0000","last_fullsized":"2023-10-26T06:46:31.769635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:38.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778290+0000","last_change":"2023-10-26T06:45:05.907722+0000","last_active":"2023-10-26T06:46:31.778290+0000","last_peered":"2023-10-26T06:46:31.778290+0000","last_clean":"2023-10-26T06:46:31.778290+0000","last_became_active":"2023-10-26T06:45:05.905622+0000","last_became_peered":"2023-10-26T06:45:05.905622+0000","last_unstale":"2023-10-26T06:46:31.778290+0000","last_undegraded":"2023-10-26T06:46:31.778290+0000","last_fullsized":"2023-10-26T06:46:31.778290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:30:01.048032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779200+0000","last_change":"2023-10-26T06:45:05.907801+0000","last_active":"2023-10-26T06:46:31.779200+0000","last_peered":"2023-10-26T06:46:31.779200+0000","last_clean":"2023-10-26T06:46:31.779200+0000","last_became_active":"2023-10-26T06:45:05.907485+0000","last_became_peered":"2023-10-26T06:45:05.907485+0000","last_unstale":"2023-10-26T06:46:31.779200+0000","last_undegraded":"2023-10-26T06:46:31.779200+0000","last_fullsized":"2023-10-26T06:46:31.779200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:16.850992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782324+0000","last_change":"2023-10-26T06:45:05.902859+0000","last_active":"2023-10-26T06:46:31.782324+0000","last_peered":"2023-10-26T06:46:31.782324+0000","last_clean":"2023-10-26T06:46:31.782324+0000","last_became_active":"2023-10-26T06:45:05.902218+0000","last_became_peered":"2023-10-26T06:45:05.902218+0000","last_unstale":"2023-10-26T06:46:31.782324+0000","last_undegraded":"2023-10-26T06:46:31.782324+0000","last_fullsized":"2023-10-26T06:46:31.782324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:47.717227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763315+0000","last_change":"2023-10-26T06:45:05.910445+0000","last_active":"2023-10-26T06:46:31.763315+0000","last_peered":"2023-10-26T06:46:31.763315+0000","last_clean":"2023-10-26T06:46:31.763315+0000","last_became_active":"2023-10-26T06:45:05.910043+0000","last_became_peered":"2023-10-26T06:45:05.910043+0000","last_unstale":"2023-10-26T06:46:31.763315+0000","last_undegraded":"2023-10-26T06:46:31.763315+0000","last_fullsized":"2023-10-26T06:46:31.763315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:33.751134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775030+0000","last_change":"2023-10-26T06:45:05.905340+0000","last_active":"2023-10-26T06:46:31.775030+0000","last_peered":"2023-10-26T06:46:31.775030+0000","last_clean":"2023-10-26T06:46:31.775030+0000","last_became_active":"2023-10-26T06:45:05.905030+0000","last_became_peered":"2023-10-26T06:45:05.905030+0000","last_unstale":"2023-10-26T06:46:31.775030+0000","last_undegraded":"2023-10-26T06:46:31.775030+0000","last_fullsized":"2023-10-26T06:46:31.775030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.303210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226621+0000","last_change":"2023-10-26T06:45:04.919128+0000","last_active":"2023-10-26T06:46:02.226621+0000","last_peered":"2023-10-26T06:46:02.226621+0000","last_clean":"2023-10-26T06:46:02.226621+0000","last_became_active":"2023-10-26T06:45:04.913034+0000","last_became_peered":"2023-10-26T06:45:04.913034+0000","last_unstale":"2023-10-26T06:46:02.226621+0000","last_undegraded":"2023-10-26T06:46:02.226621+0000","last_fullsized":"2023-10-26T06:46:02.226621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:50.835037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.771222+0000","last_change":"2023-10-26T06:45:05.905315+0000","last_active":"2023-10-26T06:46:31.771222+0000","last_peered":"2023-10-26T06:46:31.771222+0000","last_clean":"2023-10-26T06:46:31.771222+0000","last_became_active":"2023-10-26T06:45:05.904260+0000","last_became_peered":"2023-10-26T06:45:05.904260+0000","last_unstale":"2023-10-26T06:46:31.771222+0000","last_undegraded":"2023-10-26T06:46:31.771222+0000","last_fullsized":"2023-10-26T06:46:31.771222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:02.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230870+0000","last_change":"2023-10-26T06:45:04.920440+0000","last_active":"2023-10-26T06:46:02.230870+0000","last_peered":"2023-10-26T06:46:02.230870+0000","last_clean":"2023-10-26T06:46:02.230870+0000","last_became_active":"2023-10-26T06:45:04.920129+0000","last_became_peered":"2023-10-26T06:45:04.920129+0000","last_unstale":"2023-10-26T06:46:02.230870+0000","last_undegraded":"2023-10-26T06:46:02.230870+0000","last_fullsized":"2023-10-26T06:46:02.230870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:43.176830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780674+0000","last_change":"2023-10-26T06:45:05.902898+0000","last_active":"2023-10-26T06:46:31.780674+0000","last_peered":"2023-10-26T06:46:31.780674+0000","last_clean":"2023-10-26T06:46:31.780674+0000","last_became_active":"2023-10-26T06:45:05.902161+0000","last_became_peered":"2023-10-26T06:45:05.902161+0000","last_unstale":"2023-10-26T06:46:31.780674+0000","last_undegraded":"2023-10-26T06:46:31.780674+0000","last_fullsized":"2023-10-26T06:46:31.780674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:27.872225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227991+0000","last_change":"2023-10-26T06:45:04.911835+0000","last_active":"2023-10-26T06:46:02.227991+0000","last_peered":"2023-10-26T06:46:02.227991+0000","last_clean":"2023-10-26T06:46:02.227991+0000","last_became_active":"2023-10-26T06:45:04.911505+0000","last_became_peered":"2023-10-26T06:45:04.911505+0000","last_unstale":"2023-10-26T06:46:02.227991+0000","last_undegraded":"2023-10-26T06:46:02.227991+0000","last_fullsized":"2023-10-26T06:46:02.227991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:01:18.971381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779287+0000","last_change":"2023-10-26T06:45:05.909088+0000","last_active":"2023-10-26T06:46:31.779287+0000","last_peered":"2023-10-26T06:46:31.779287+0000","last_clean":"2023-10-26T06:46:31.779287+0000","last_became_active":"2023-10-26T06:45:05.908528+0000","last_became_peered":"2023-10-26T06:45:05.908528+0000","last_unstale":"2023-10-26T06:46:31.779287+0000","last_undegraded":"2023-10-26T06:46:31.779287+0000","last_fullsized":"2023-10-26T06:46:31.779287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:07.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.237699+0000","last_change":"2023-10-26T06:46:03.236510+0000","last_active":"2023-10-26T06:46:03.237699+0000","last_peered":"2023-10-26T06:46:03.237699+0000","last_clean":"2023-10-26T06:46:03.237699+0000","last_became_active":"2023-10-26T06:46:03.233056+0000","last_became_peered":"2023-10-26T06:46:03.233056+0000","last_unstale":"2023-10-26T06:46:03.237699+0000","last_undegraded":"2023-10-26T06:46:03.237699+0000","last_fullsized":"2023-10-26T06:46:03.237699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:33.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.777342+0000","last_change":"2023-10-26T06:45:05.898795+0000","last_active":"2023-10-26T06:46:31.777342+0000","last_peered":"2023-10-26T06:46:31.777342+0000","last_clean":"2023-10-26T06:46:31.777342+0000","last_became_active":"2023-10-26T06:45:05.898489+0000","last_became_peered":"2023-10-26T06:45:05.898489+0000","last_unstale":"2023-10-26T06:46:31.777342+0000","last_undegraded":"2023-10-26T06:46:31.777342+0000","last_fullsized":"2023-10-26T06:46:31.777342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:55.343736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230147+0000","last_change":"2023-10-26T06:45:04.908881+0000","last_active":"2023-10-26T06:46:02.230147+0000","last_peered":"2023-10-26T06:46:02.230147+0000","last_clean":"2023-10-26T06:46:02.230147+0000","last_became_active":"2023-10-26T06:45:04.908515+0000","last_became_peered":"2023-10-26T06:45:04.908515+0000","last_unstale":"2023-10-26T06:46:02.230147+0000","last_undegraded":"2023-10-26T06:46:02.230147+0000","last_fullsized":"2023-10-26T06:46:02.230147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:39.200294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773736+0000","last_change":"2023-10-26T06:46:03.237686+0000","last_active":"2023-10-26T06:46:31.773736+0000","last_peered":"2023-10-26T06:46:31.773736+0000","last_clean":"2023-10-26T06:46:31.773736+0000","last_became_active":"2023-10-26T06:46:03.237275+0000","last_became_peered":"2023-10-26T06:46:03.237275+0000","last_unstale":"2023-10-26T06:46:31.773736+0000","last_undegraded":"2023-10-26T06:46:31.773736+0000","last_fullsized":"2023-10-26T06:46:31.773736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:56:22.281364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.234834+0000","last_change":"2023-10-26T06:46:03.234834+0000","last_active":"2023-10-26T06:46:03.234834+0000","last_peered":"2023-10-26T06:46:03.234834+0000","last_clean":"2023-10-26T06:46:03.234834+0000","last_became_active":"2023-10-26T06:46:03.234289+0000","last_became_peered":"2023-10-26T06:46:03.234289+0000","last_unstale":"2023-10-26T06:46:03.234834+0000","last_undegraded":"2023-10-26T06:46:03.234834+0000","last_fullsized":"2023-10-26T06:46:03.234834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:50:53.362840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773846+0000","last_change":"2023-10-26T06:45:05.901825+0000","last_active":"2023-10-26T06:46:31.773846+0000","last_peered":"2023-10-26T06:46:31.773846+0000","last_clean":"2023-10-26T06:46:31.773846+0000","last_became_active":"2023-10-26T06:45:05.901509+0000","last_became_peered":"2023-10-26T06:45:05.901509+0000","last_unstale":"2023-10-26T06:46:31.773846+0000","last_undegraded":"2023-10-26T06:46:31.773846+0000","last_fullsized":"2023-10-26T06:46:31.773846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:16.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227045+0000","last_change":"2023-10-26T06:45:04.919813+0000","last_active":"2023-10-26T06:46:02.227045+0000","last_peered":"2023-10-26T06:46:02.227045+0000","last_clean":"2023-10-26T06:46:02.227045+0000","last_became_active":"2023-10-26T06:45:04.918016+0000","last_became_peered":"2023-10-26T06:45:04.918016+0000","last_unstale":"2023-10-26T06:46:02.227045+0000","last_undegraded":"2023-10-26T06:46:02.227045+0000","last_fullsized":"2023-10-26T06:46:02.227045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:32.336126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779735+0000","last_change":"2023-10-26T06:45:05.902182+0000","last_active":"2023-10-26T06:46:31.779735+0000","last_peered":"2023-10-26T06:46:31.779735+0000","last_clean":"2023-10-26T06:46:31.779735+0000","last_became_active":"2023-10-26T06:45:05.901428+0000","last_became_peered":"2023-10-26T06:45:05.901428+0000","last_unstale":"2023-10-26T06:46:31.779735+0000","last_undegraded":"2023-10-26T06:46:31.779735+0000","last_fullsized":"2023-10-26T06:46:31.779735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:12.722532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.229834+0000","last_change":"2023-10-26T06:45:04.909713+0000","last_active":"2023-10-26T06:46:02.229834+0000","last_peered":"2023-10-26T06:46:02.229834+0000","last_clean":"2023-10-26T06:46:02.229834+0000","last_became_active":"2023-10-26T06:45:04.909412+0000","last_became_peered":"2023-10-26T06:45:04.909412+0000","last_unstale":"2023-10-26T06:46:02.229834+0000","last_undegraded":"2023-10-26T06:46:02.229834+0000","last_fullsized":"2023-10-26T06:46:02.229834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:23.901745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773996+0000","last_change":"2023-10-26T06:45:05.908348+0000","last_active":"2023-10-26T06:46:31.773996+0000","last_peered":"2023-10-26T06:46:31.773996+0000","last_clean":"2023-10-26T06:46:31.773996+0000","last_became_active":"2023-10-26T06:45:05.908022+0000","last_became_peered":"2023-10-26T06:45:05.908022+0000","last_unstale":"2023-10-26T06:46:31.773996+0000","last_undegraded":"2023-10-26T06:46:31.773996+0000","last_fullsized":"2023-10-26T06:46:31.773996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:22:16.027732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.223807+0000","last_change":"2023-10-26T06:45:04.915491+0000","last_active":"2023-10-26T06:46:02.223807+0000","last_peered":"2023-10-26T06:46:02.223807+0000","last_clean":"2023-10-26T06:46:02.223807+0000","last_became_active":"2023-10-26T06:45:04.915191+0000","last_became_peered":"2023-10-26T06:45:04.915191+0000","last_unstale":"2023-10-26T06:46:02.223807+0000","last_undegraded":"2023-10-26T06:46:02.223807+0000","last_fullsized":"2023-10-26T06:46:02.223807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:57.758903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779813+0000","last_change":"2023-10-26T06:45:05.912676+0000","last_active":"2023-10-26T06:46:31.779813+0000","last_peered":"2023-10-26T06:46:31.779813+0000","last_clean":"2023-10-26T06:46:31.779813+0000","last_became_active":"2023-10-26T06:45:05.912372+0000","last_became_peered":"2023-10-26T06:45:05.912372+0000","last_unstale":"2023-10-26T06:46:31.779813+0000","last_undegraded":"2023-10-26T06:46:31.779813+0000","last_fullsized":"2023-10-26T06:46:31.779813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:33.299166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.474091+0000","last_change":"2023-10-26T06:45:04.912123+0000","last_active":"2023-10-26T06:46:02.474091+0000","last_peered":"2023-10-26T06:46:02.474091+0000","last_clean":"2023-10-26T06:46:02.474091+0000","last_became_active":"2023-10-26T06:45:04.911893+0000","last_became_peered":"2023-10-26T06:45:04.911893+0000","last_unstale":"2023-10-26T06:46:02.474091+0000","last_undegraded":"2023-10-26T06:46:02.474091+0000","last_fullsized":"2023-10-26T06:46:02.474091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:32.907496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780208+0000","last_change":"2023-10-26T06:45:05.893434+0000","last_active":"2023-10-26T06:46:31.780208+0000","last_peered":"2023-10-26T06:46:31.780208+0000","last_clean":"2023-10-26T06:46:31.780208+0000","last_became_active":"2023-10-26T06:45:05.893100+0000","last_became_peered":"2023-10-26T06:45:05.893100+0000","last_unstale":"2023-10-26T06:46:31.780208+0000","last_undegraded":"2023-10-26T06:46:31.780208+0000","last_fullsized":"2023-10-26T06:46:31.780208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:22:48.317428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230570+0000","last_change":"2023-10-26T06:45:04.908910+0000","last_active":"2023-10-26T06:46:02.230570+0000","last_peered":"2023-10-26T06:46:02.230570+0000","last_clean":"2023-10-26T06:46:02.230570+0000","last_became_active":"2023-10-26T06:45:04.908592+0000","last_became_peered":"2023-10-26T06:45:04.908592+0000","last_unstale":"2023-10-26T06:46:02.230570+0000","last_undegraded":"2023-10-26T06:46:02.230570+0000","last_fullsized":"2023-10-26T06:46:02.230570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:40.266013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.776098+0000","last_change":"2023-10-26T06:45:05.908887+0000","last_active":"2023-10-26T06:46:31.776098+0000","last_peered":"2023-10-26T06:46:31.776098+0000","last_clean":"2023-10-26T06:46:31.776098+0000","last_became_active":"2023-10-26T06:45:05.908163+0000","last_became_peered":"2023-10-26T06:45:05.908163+0000","last_unstale":"2023-10-26T06:46:31.776098+0000","last_undegraded":"2023-10-26T06:46:31.776098+0000","last_fullsized":"2023-10-26T06:46:31.776098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:09.339374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":253,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:47.249475+0000","last_change":"2023-10-26T06:45:04.905691+0000","last_active":"2023-10-26T06:46:47.249475+0000","last_peered":"2023-10-26T06:46:47.249475+0000","last_clean":"2023-10-26T06:46:47.249475+0000","last_became_active":"2023-10-26T06:45:04.905378+0000","last_became_peered":"2023-10-26T06:45:04.905378+0000","last_unstale":"2023-10-26T06:46:47.249475+0000","last_undegraded":"2023-10-26T06:46:47.249475+0000","last_fullsized":"2023-10-26T06:46:47.249475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:24:21.498141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64519,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":208,"num_read_kb":166,"num_write":30,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.765219+0000","last_change":"2023-10-26T06:45:05.897366+0000","last_active":"2023-10-26T06:46:31.765219+0000","last_peered":"2023-10-26T06:46:31.765219+0000","last_clean":"2023-10-26T06:46:31.765219+0000","last_became_active":"2023-10-26T06:45:05.897091+0000","last_became_peered":"2023-10-26T06:45:05.897091+0000","last_unstale":"2023-10-26T06:46:31.765219+0000","last_undegraded":"2023-10-26T06:46:31.765219+0000","last_fullsized":"2023-10-26T06:46:31.765219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:11.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226619+0000","last_change":"2023-10-26T06:45:04.914949+0000","last_active":"2023-10-26T06:46:02.226619+0000","last_peered":"2023-10-26T06:46:02.226619+0000","last_clean":"2023-10-26T06:46:02.226619+0000","last_became_active":"2023-10-26T06:45:04.914610+0000","last_became_peered":"2023-10-26T06:45:04.914610+0000","last_unstale":"2023-10-26T06:46:02.226619+0000","last_undegraded":"2023-10-26T06:46:02.226619+0000","last_fullsized":"2023-10-26T06:46:02.226619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:16.683352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774454+0000","last_change":"2023-10-26T06:45:05.911067+0000","last_active":"2023-10-26T06:46:31.774454+0000","last_peered":"2023-10-26T06:46:31.774454+0000","last_clean":"2023-10-26T06:46:31.774454+0000","last_became_active":"2023-10-26T06:45:05.908460+0000","last_became_peered":"2023-10-26T06:45:05.908460+0000","last_unstale":"2023-10-26T06:46:31.774454+0000","last_undegraded":"2023-10-26T06:46:31.774454+0000","last_fullsized":"2023-10-26T06:46:31.774454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:17.461781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230798+0000","last_change":"2023-10-26T06:45:04.919947+0000","last_active":"2023-10-26T06:46:02.230798+0000","last_peered":"2023-10-26T06:46:02.230798+0000","last_clean":"2023-10-26T06:46:02.230798+0000","last_became_active":"2023-10-26T06:45:04.919535+0000","last_became_peered":"2023-10-26T06:45:04.919535+0000","last_unstale":"2023-10-26T06:46:02.230798+0000","last_undegraded":"2023-10-26T06:46:02.230798+0000","last_fullsized":"2023-10-26T06:46:02.230798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:46:09.201139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782275+0000","last_change":"2023-10-26T06:45:05.906381+0000","last_active":"2023-10-26T06:46:31.782275+0000","last_peered":"2023-10-26T06:46:31.782275+0000","last_clean":"2023-10-26T06:46:31.782275+0000","last_became_active":"2023-10-26T06:45:05.904487+0000","last_became_peered":"2023-10-26T06:45:05.904487+0000","last_unstale":"2023-10-26T06:46:31.782275+0000","last_undegraded":"2023-10-26T06:46:31.782275+0000","last_fullsized":"2023-10-26T06:46:31.782275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:00.164936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227232+0000","last_change":"2023-10-26T06:45:04.908779+0000","last_active":"2023-10-26T06:46:02.227232+0000","last_peered":"2023-10-26T06:46:02.227232+0000","last_clean":"2023-10-26T06:46:02.227232+0000","last_became_active":"2023-10-26T06:45:04.908490+0000","last_became_peered":"2023-10-26T06:45:04.908490+0000","last_unstale":"2023-10-26T06:46:02.227232+0000","last_undegraded":"2023-10-26T06:46:02.227232+0000","last_fullsized":"2023-10-26T06:46:02.227232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:12:13.199863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764235+0000","last_change":"2023-10-26T06:45:05.909408+0000","last_active":"2023-10-26T06:46:31.764235+0000","last_peered":"2023-10-26T06:46:31.764235+0000","last_clean":"2023-10-26T06:46:31.764235+0000","last_became_active":"2023-10-26T06:45:05.909107+0000","last_became_peered":"2023-10-26T06:45:05.909107+0000","last_unstale":"2023-10-26T06:46:31.764235+0000","last_undegraded":"2023-10-26T06:46:31.764235+0000","last_fullsized":"2023-10-26T06:46:31.764235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:05.887472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.424977+0000","last_change":"2023-10-26T06:46:03.424977+0000","last_active":"2023-10-26T06:46:03.424977+0000","last_peered":"2023-10-26T06:46:03.424977+0000","last_clean":"2023-10-26T06:46:03.424977+0000","last_became_active":"2023-10-26T06:46:03.417899+0000","last_became_peered":"2023-10-26T06:46:03.417899+0000","last_unstale":"2023-10-26T06:46:03.424977+0000","last_undegraded":"2023-10-26T06:46:03.424977+0000","last_fullsized":"2023-10-26T06:46:03.424977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:51.092657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775545+0000","last_change":"2023-10-26T06:45:05.902156+0000","last_active":"2023-10-26T06:46:31.775545+0000","last_peered":"2023-10-26T06:46:31.775545+0000","last_clean":"2023-10-26T06:46:31.775545+0000","last_became_active":"2023-10-26T06:45:05.901865+0000","last_became_peered":"2023-10-26T06:45:05.901865+0000","last_unstale":"2023-10-26T06:46:31.775545+0000","last_undegraded":"2023-10-26T06:46:31.775545+0000","last_fullsized":"2023-10-26T06:46:31.775545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:06:08.167280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.228254+0000","last_change":"2023-10-26T06:45:04.913071+0000","last_active":"2023-10-26T06:46:02.228254+0000","last_peered":"2023-10-26T06:46:02.228254+0000","last_clean":"2023-10-26T06:46:02.228254+0000","last_became_active":"2023-10-26T06:45:04.912736+0000","last_became_peered":"2023-10-26T06:45:04.912736+0000","last_unstale":"2023-10-26T06:46:02.228254+0000","last_undegraded":"2023-10-26T06:46:02.228254+0000","last_fullsized":"2023-10-26T06:46:02.228254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:59.690355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.757581+0000","last_change":"2023-10-26T06:46:03.233858+0000","last_active":"2023-10-26T06:46:31.757581+0000","last_peered":"2023-10-26T06:46:31.757581+0000","last_clean":"2023-10-26T06:46:31.757581+0000","last_became_active":"2023-10-26T06:46:03.233429+0000","last_became_peered":"2023-10-26T06:46:03.233429+0000","last_unstale":"2023-10-26T06:46:31.757581+0000","last_undegraded":"2023-10-26T06:46:31.757581+0000","last_fullsized":"2023-10-26T06:46:31.757581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:41:39.237862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.237541+0000","last_change":"2023-10-26T06:46:03.237541+0000","last_active":"2023-10-26T06:46:03.237541+0000","last_peered":"2023-10-26T06:46:03.237541+0000","last_clean":"2023-10-26T06:46:03.237541+0000","last_became_active":"2023-10-26T06:46:03.237019+0000","last_became_peered":"2023-10-26T06:46:03.237019+0000","last_unstale":"2023-10-26T06:46:03.237541+0000","last_undegraded":"2023-10-26T06:46:03.237541+0000","last_fullsized":"2023-10-26T06:46:03.237541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:59:15.383799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775563+0000","last_change":"2023-10-26T06:45:05.904832+0000","last_active":"2023-10-26T06:46:31.775563+0000","last_peered":"2023-10-26T06:46:31.775563+0000","last_clean":"2023-10-26T06:46:31.775563+0000","last_became_active":"2023-10-26T06:45:05.904527+0000","last_became_peered":"2023-10-26T06:45:05.904527+0000","last_unstale":"2023-10-26T06:46:31.775563+0000","last_undegraded":"2023-10-26T06:46:31.775563+0000","last_fullsized":"2023-10-26T06:46:31.775563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:30.546198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473828+0000","last_change":"2023-10-26T06:45:04.922832+0000","last_active":"2023-10-26T06:46:02.473828+0000","last_peered":"2023-10-26T06:46:02.473828+0000","last_clean":"2023-10-26T06:46:02.473828+0000","last_became_active":"2023-10-26T06:45:04.922349+0000","last_became_peered":"2023-10-26T06:45:04.922349+0000","last_unstale":"2023-10-26T06:46:02.473828+0000","last_undegraded":"2023-10-26T06:46:02.473828+0000","last_fullsized":"2023-10-26T06:46:02.473828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:37.839093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.770899+0000","last_change":"2023-10-26T06:45:05.909946+0000","last_active":"2023-10-26T06:46:31.770899+0000","last_peered":"2023-10-26T06:46:31.770899+0000","last_clean":"2023-10-26T06:46:31.770899+0000","last_became_active":"2023-10-26T06:45:05.909650+0000","last_became_peered":"2023-10-26T06:45:05.909650+0000","last_unstale":"2023-10-26T06:46:31.770899+0000","last_undegraded":"2023-10-26T06:46:31.770899+0000","last_fullsized":"2023-10-26T06:46:31.770899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:41.813928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763208+0000","last_change":"2023-10-26T06:45:05.903400+0000","last_active":"2023-10-26T06:46:31.763208+0000","last_peered":"2023-10-26T06:46:31.763208+0000","last_clean":"2023-10-26T06:46:31.763208+0000","last_became_active":"2023-10-26T06:45:05.903101+0000","last_became_peered":"2023-10-26T06:45:05.903101+0000","last_unstale":"2023-10-26T06:46:31.763208+0000","last_undegraded":"2023-10-26T06:46:31.763208+0000","last_fullsized":"2023-10-26T06:46:31.763208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227030+0000","last_change":"2023-10-26T06:45:04.923960+0000","last_active":"2023-10-26T06:46:02.227030+0000","last_peered":"2023-10-26T06:46:02.227030+0000","last_clean":"2023-10-26T06:46:02.227030+0000","last_became_active":"2023-10-26T06:45:04.923613+0000","last_became_peered":"2023-10-26T06:45:04.923613+0000","last_unstale":"2023-10-26T06:46:02.227030+0000","last_undegraded":"2023-10-26T06:46:02.227030+0000","last_fullsized":"2023-10-26T06:46:02.227030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:53.342463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778716+0000","last_change":"2023-10-26T06:45:05.903966+0000","last_active":"2023-10-26T06:46:31.778716+0000","last_peered":"2023-10-26T06:46:31.778716+0000","last_clean":"2023-10-26T06:46:31.778716+0000","last_became_active":"2023-10-26T06:45:05.903583+0000","last_became_peered":"2023-10-26T06:45:05.903583+0000","last_unstale":"2023-10-26T06:46:31.778716+0000","last_undegraded":"2023-10-26T06:46:31.778716+0000","last_fullsized":"2023-10-26T06:46:31.778716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:18.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226986+0000","last_change":"2023-10-26T06:45:04.907838+0000","last_active":"2023-10-26T06:46:02.226986+0000","last_peered":"2023-10-26T06:46:02.226986+0000","last_clean":"2023-10-26T06:46:02.226986+0000","last_became_active":"2023-10-26T06:45:04.907508+0000","last_became_peered":"2023-10-26T06:45:04.907508+0000","last_unstale":"2023-10-26T06:46:02.226986+0000","last_undegraded":"2023-10-26T06:46:02.226986+0000","last_fullsized":"2023-10-26T06:46:02.226986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:34.852243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":321,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473730+0000","last_change":"2023-10-26T06:43:59.765466+0000","last_active":"2023-10-26T06:46:02.473730+0000","last_peered":"2023-10-26T06:46:02.473730+0000","last_clean":"2023-10-26T06:46:02.473730+0000","last_became_active":"2023-10-26T06:43:59.765005+0000","last_became_peered":"2023-10-26T06:43:59.765005+0000","last_unstale":"2023-10-26T06:46:02.473730+0000","last_undegraded":"2023-10-26T06:46:02.473730+0000","last_fullsized":"2023-10-26T06:46:02.473730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_clean_scrub_stamp":"2023-10-26T06:40:50.556448+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:18.688941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774103+0000","last_change":"2023-10-26T06:45:05.906909+0000","last_active":"2023-10-26T06:46:31.774103+0000","last_peered":"2023-10-26T06:46:31.774103+0000","last_clean":"2023-10-26T06:46:31.774103+0000","last_became_active":"2023-10-26T06:45:05.906531+0000","last_became_peered":"2023-10-26T06:45:05.906531+0000","last_unstale":"2023-10-26T06:46:31.774103+0000","last_undegraded":"2023-10-26T06:46:31.774103+0000","last_fullsized":"2023-10-26T06:46:31.774103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473805+0000","last_change":"2023-10-26T06:45:04.911635+0000","last_active":"2023-10-26T06:46:02.473805+0000","last_peered":"2023-10-26T06:46:02.473805+0000","last_clean":"2023-10-26T06:46:02.473805+0000","last_became_active":"2023-10-26T06:45:04.911289+0000","last_became_peered":"2023-10-26T06:45:04.911289+0000","last_unstale":"2023-10-26T06:46:02.473805+0000","last_undegraded":"2023-10-26T06:46:02.473805+0000","last_fullsized":"2023-10-26T06:46:02.473805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.115211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227809+0000","last_change":"2023-10-26T06:45:04.919853+0000","last_active":"2023-10-26T06:46:02.227809+0000","last_peered":"2023-10-26T06:46:02.227809+0000","last_clean":"2023-10-26T06:46:02.227809+0000","last_became_active":"2023-10-26T06:45:04.919488+0000","last_became_peered":"2023-10-26T06:45:04.919488+0000","last_unstale":"2023-10-26T06:46:02.227809+0000","last_undegraded":"2023-10-26T06:46:02.227809+0000","last_fullsized":"2023-10-26T06:46:02.227809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:06.555851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.776607+0000","last_change":"2023-10-26T06:45:05.903975+0000","last_active":"2023-10-26T06:46:31.776607+0000","last_peered":"2023-10-26T06:46:31.776607+0000","last_clean":"2023-10-26T06:46:31.776607+0000","last_became_active":"2023-10-26T06:45:05.903539+0000","last_became_peered":"2023-10-26T06:45:05.903539+0000","last_unstale":"2023-10-26T06:46:31.776607+0000","last_undegraded":"2023-10-26T06:46:31.776607+0000","last_fullsized":"2023-10-26T06:46:31.776607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:30.648947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:33.946402+0000","last_change":"2023-10-26T06:45:04.919815+0000","last_active":"2023-10-26T06:46:33.946402+0000","last_peered":"2023-10-26T06:46:33.946402+0000","last_clean":"2023-10-26T06:46:33.946402+0000","last_became_active":"2023-10-26T06:45:04.918141+0000","last_became_peered":"2023-10-26T06:45:04.918141+0000","last_unstale":"2023-10-26T06:46:33.946402+0000","last_undegraded":"2023-10-26T06:46:33.946402+0000","last_fullsized":"2023-10-26T06:46:33.946402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:53.024534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774509+0000","last_change":"2023-10-26T06:45:05.898180+0000","last_active":"2023-10-26T06:46:31.774509+0000","last_peered":"2023-10-26T06:46:31.774509+0000","last_clean":"2023-10-26T06:46:31.774509+0000","last_became_active":"2023-10-26T06:45:05.896937+0000","last_became_peered":"2023-10-26T06:45:05.896937+0000","last_unstale":"2023-10-26T06:46:31.774509+0000","last_undegraded":"2023-10-26T06:46:31.774509+0000","last_fullsized":"2023-10-26T06:46:31.774509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:48.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227006+0000","last_change":"2023-10-26T06:45:04.922456+0000","last_active":"2023-10-26T06:46:02.227006+0000","last_peered":"2023-10-26T06:46:02.227006+0000","last_clean":"2023-10-26T06:46:02.227006+0000","last_became_active":"2023-10-26T06:45:04.922112+0000","last_became_peered":"2023-10-26T06:45:04.922112+0000","last_unstale":"2023-10-26T06:46:02.227006+0000","last_undegraded":"2023-10-26T06:46:02.227006+0000","last_fullsized":"2023-10-26T06:46:02.227006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:44.055569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781350+0000","last_change":"2023-10-26T06:45:05.895440+0000","last_active":"2023-10-26T06:46:31.781350+0000","last_peered":"2023-10-26T06:46:31.781350+0000","last_clean":"2023-10-26T06:46:31.781350+0000","last_became_active":"2023-10-26T06:45:05.892605+0000","last_became_peered":"2023-10-26T06:45:05.892605+0000","last_unstale":"2023-10-26T06:46:31.781350+0000","last_undegraded":"2023-10-26T06:46:31.781350+0000","last_fullsized":"2023-10-26T06:46:31.781350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:09:27.392772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.225650+0000","last_change":"2023-10-26T06:45:04.916042+0000","last_active":"2023-10-26T06:46:02.225650+0000","last_peered":"2023-10-26T06:46:02.225650+0000","last_clean":"2023-10-26T06:46:02.225650+0000","last_became_active":"2023-10-26T06:45:04.915649+0000","last_became_peered":"2023-10-26T06:45:04.915649+0000","last_unstale":"2023-10-26T06:46:02.225650+0000","last_undegraded":"2023-10-26T06:46:02.225650+0000","last_fullsized":"2023-10-26T06:46:02.225650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:30.060074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.735910+0000","last_change":"2023-10-26T06:45:05.908179+0000","last_active":"2023-10-26T06:46:31.735910+0000","last_peered":"2023-10-26T06:46:31.735910+0000","last_clean":"2023-10-26T06:46:31.735910+0000","last_became_active":"2023-10-26T06:45:05.907857+0000","last_became_peered":"2023-10-26T06:45:05.907857+0000","last_unstale":"2023-10-26T06:46:31.735910+0000","last_undegraded":"2023-10-26T06:46:31.735910+0000","last_fullsized":"2023-10-26T06:46:31.735910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:20.017194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230341+0000","last_change":"2023-10-26T06:45:04.921016+0000","last_active":"2023-10-26T06:46:02.230341+0000","last_peered":"2023-10-26T06:46:02.230341+0000","last_clean":"2023-10-26T06:46:02.230341+0000","last_became_active":"2023-10-26T06:45:04.920741+0000","last_became_peered":"2023-10-26T06:45:04.920741+0000","last_unstale":"2023-10-26T06:46:02.230341+0000","last_undegraded":"2023-10-26T06:46:02.230341+0000","last_fullsized":"2023-10-26T06:46:02.230341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:05:06.218205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778934+0000","last_change":"2023-10-26T06:45:05.893562+0000","last_active":"2023-10-26T06:46:31.778934+0000","last_peered":"2023-10-26T06:46:31.778934+0000","last_clean":"2023-10-26T06:46:31.778934+0000","last_became_active":"2023-10-26T06:45:05.893111+0000","last_became_peered":"2023-10-26T06:45:05.893111+0000","last_unstale":"2023-10-26T06:46:31.778934+0000","last_undegraded":"2023-10-26T06:46:31.778934+0000","last_fullsized":"2023-10-26T06:46:31.778934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.305664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226475+0000","last_change":"2023-10-26T06:45:04.914936+0000","last_active":"2023-10-26T06:46:02.226475+0000","last_peered":"2023-10-26T06:46:02.226475+0000","last_clean":"2023-10-26T06:46:02.226475+0000","last_became_active":"2023-10-26T06:45:04.914617+0000","last_became_peered":"2023-10-26T06:45:04.914617+0000","last_unstale":"2023-10-26T06:46:02.226475+0000","last_undegraded":"2023-10-26T06:46:02.226475+0000","last_fullsized":"2023-10-26T06:46:02.226475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:15.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778284+0000","last_change":"2023-10-26T06:45:05.909101+0000","last_active":"2023-10-26T06:46:31.778284+0000","last_peered":"2023-10-26T06:46:31.778284+0000","last_clean":"2023-10-26T06:46:31.778284+0000","last_became_active":"2023-10-26T06:45:05.908800+0000","last_became_peered":"2023-10-26T06:45:05.908800+0000","last_unstale":"2023-10-26T06:46:31.778284+0000","last_undegraded":"2023-10-26T06:46:31.778284+0000","last_fullsized":"2023-10-26T06:46:31.778284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:37:18.605102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227537+0000","last_change":"2023-10-26T06:45:04.911714+0000","last_active":"2023-10-26T06:46:02.227537+0000","last_peered":"2023-10-26T06:46:02.227537+0000","last_clean":"2023-10-26T06:46:02.227537+0000","last_became_active":"2023-10-26T06:45:04.911343+0000","last_became_peered":"2023-10-26T06:45:04.911343+0000","last_unstale":"2023-10-26T06:46:02.227537+0000","last_undegraded":"2023-10-26T06:46:02.227537+0000","last_fullsized":"2023-10-26T06:46:02.227537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:30:35.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773867+0000","last_change":"2023-10-26T06:45:05.906804+0000","last_active":"2023-10-26T06:46:31.773867+0000","last_peered":"2023-10-26T06:46:31.773867+0000","last_clean":"2023-10-26T06:46:31.773867+0000","last_became_active":"2023-10-26T06:45:05.906522+0000","last_became_peered":"2023-10-26T06:45:05.906522+0000","last_unstale":"2023-10-26T06:46:31.773867+0000","last_undegraded":"2023-10-26T06:46:31.773867+0000","last_fullsized":"2023-10-26T06:46:31.773867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:32.707204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226957+0000","last_change":"2023-10-26T06:45:04.906345+0000","last_active":"2023-10-26T06:46:02.226957+0000","last_peered":"2023-10-26T06:46:02.226957+0000","last_clean":"2023-10-26T06:46:02.226957+0000","last_became_active":"2023-10-26T06:45:04.906034+0000","last_became_peered":"2023-10-26T06:45:04.906034+0000","last_unstale":"2023-10-26T06:46:02.226957+0000","last_undegraded":"2023-10-26T06:46:02.226957+0000","last_fullsized":"2023-10-26T06:46:02.226957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:53.690788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779248+0000","last_change":"2023-10-26T06:45:05.906368+0000","last_active":"2023-10-26T06:46:31.779248+0000","last_peered":"2023-10-26T06:46:31.779248+0000","last_clean":"2023-10-26T06:46:31.779248+0000","last_became_active":"2023-10-26T06:45:05.905471+0000","last_became_peered":"2023-10-26T06:45:05.905471+0000","last_unstale":"2023-10-26T06:46:31.779248+0000","last_undegraded":"2023-10-26T06:46:31.779248+0000","last_fullsized":"2023-10-26T06:46:31.779248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:45.493704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230243+0000","last_change":"2023-10-26T06:45:04.917530+0000","last_active":"2023-10-26T06:46:02.230243+0000","last_peered":"2023-10-26T06:46:02.230243+0000","last_clean":"2023-10-26T06:46:02.230243+0000","last_became_active":"2023-10-26T06:45:04.917193+0000","last_became_peered":"2023-10-26T06:45:04.917193+0000","last_unstale":"2023-10-26T06:46:02.230243+0000","last_undegraded":"2023-10-26T06:46:02.230243+0000","last_fullsized":"2023-10-26T06:46:02.230243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.965048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.765239+0000","last_change":"2023-10-26T06:45:05.899429+0000","last_active":"2023-10-26T06:46:31.765239+0000","last_peered":"2023-10-26T06:46:31.765239+0000","last_clean":"2023-10-26T06:46:31.765239+0000","last_became_active":"2023-10-26T06:45:05.899081+0000","last_became_peered":"2023-10-26T06:45:05.899081+0000","last_unstale":"2023-10-26T06:46:31.765239+0000","last_undegraded":"2023-10-26T06:46:31.765239+0000","last_fullsized":"2023-10-26T06:46:31.765239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:41.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227783+0000","last_change":"2023-10-26T06:45:04.920295+0000","last_active":"2023-10-26T06:46:02.227783+0000","last_peered":"2023-10-26T06:46:02.227783+0000","last_clean":"2023-10-26T06:46:02.227783+0000","last_became_active":"2023-10-26T06:45:04.919920+0000","last_became_peered":"2023-10-26T06:45:04.919920+0000","last_unstale":"2023-10-26T06:46:02.227783+0000","last_undegraded":"2023-10-26T06:46:02.227783+0000","last_fullsized":"2023-10-26T06:46:02.227783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:59.260841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764977+0000","last_change":"2023-10-26T06:45:05.899610+0000","last_active":"2023-10-26T06:46:31.764977+0000","last_peered":"2023-10-26T06:46:31.764977+0000","last_clean":"2023-10-26T06:46:31.764977+0000","last_became_active":"2023-10-26T06:45:05.899283+0000","last_became_peered":"2023-10-26T06:45:05.899283+0000","last_unstale":"2023-10-26T06:46:31.764977+0000","last_undegraded":"2023-10-26T06:46:31.764977+0000","last_fullsized":"2023-10-26T06:46:31.764977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:48.730703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.759894+0000","last_change":"2023-10-26T06:45:05.892012+0000","last_active":"2023-10-26T06:46:31.759894+0000","last_peered":"2023-10-26T06:46:31.759894+0000","last_clean":"2023-10-26T06:46:31.759894+0000","last_became_active":"2023-10-26T06:45:05.891744+0000","last_became_peered":"2023-10-26T06:45:05.891744+0000","last_unstale":"2023-10-26T06:46:31.759894+0000","last_undegraded":"2023-10-26T06:46:31.759894+0000","last_fullsized":"2023-10-26T06:46:31.759894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:29.599184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763039+0000","last_change":"2023-10-26T06:45:05.904348+0000","last_active":"2023-10-26T06:46:31.763039+0000","last_peered":"2023-10-26T06:46:31.763039+0000","last_clean":"2023-10-26T06:46:31.763039+0000","last_became_active":"2023-10-26T06:45:05.903949+0000","last_became_peered":"2023-10-26T06:45:05.903949+0000","last_unstale":"2023-10-26T06:46:31.763039+0000","last_undegraded":"2023-10-26T06:46:31.763039+0000","last_fullsized":"2023-10-26T06:46:31.763039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:29.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.771138+0000","last_change":"2023-10-26T06:46:03.237595+0000","last_active":"2023-10-26T06:46:31.771138+0000","last_peered":"2023-10-26T06:46:31.771138+0000","last_clean":"2023-10-26T06:46:31.771138+0000","last_became_active":"2023-10-26T06:46:03.237136+0000","last_became_peered":"2023-10-26T06:46:03.237136+0000","last_unstale":"2023-10-26T06:46:31.771138+0000","last_undegraded":"2023-10-26T06:46:31.771138+0000","last_fullsized":"2023-10-26T06:46:31.771138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:03.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780714+0000","last_change":"2023-10-26T06:45:05.901225+0000","last_active":"2023-10-26T06:46:31.780714+0000","last_peered":"2023-10-26T06:46:31.780714+0000","last_clean":"2023-10-26T06:46:31.780714+0000","last_became_active":"2023-10-26T06:45:05.900743+0000","last_became_peered":"2023-10-26T06:45:05.900743+0000","last_unstale":"2023-10-26T06:46:31.780714+0000","last_undegraded":"2023-10-26T06:46:31.780714+0000","last_fullsized":"2023-10-26T06:46:31.780714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:02.359654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781128+0000","last_change":"2023-10-26T06:45:05.906805+0000","last_active":"2023-10-26T06:46:31.781128+0000","last_peered":"2023-10-26T06:46:31.781128+0000","last_clean":"2023-10-26T06:46:31.781128+0000","last_became_active":"2023-10-26T06:45:05.906464+0000","last_became_peered":"2023-10-26T06:45:05.906464+0000","last_unstale":"2023-10-26T06:46:31.781128+0000","last_undegraded":"2023-10-26T06:46:31.781128+0000","last_fullsized":"2023-10-26T06:46:31.781128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:24.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.759205+0000","last_change":"2023-10-26T06:45:05.898568+0000","last_active":"2023-10-26T06:46:31.759205+0000","last_peered":"2023-10-26T06:46:31.759205+0000","last_clean":"2023-10-26T06:46:31.759205+0000","last_became_active":"2023-10-26T06:45:05.897886+0000","last_became_peered":"2023-10-26T06:45:05.897886+0000","last_unstale":"2023-10-26T06:46:31.759205+0000","last_undegraded":"2023-10-26T06:46:31.759205+0000","last_fullsized":"2023-10-26T06:46:31.759205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:14.998233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779508+0000","last_change":"2023-10-26T06:45:05.899326+0000","last_active":"2023-10-26T06:46:31.779508+0000","last_peered":"2023-10-26T06:46:31.779508+0000","last_clean":"2023-10-26T06:46:31.779508+0000","last_became_active":"2023-10-26T06:45:05.899040+0000","last_became_peered":"2023-10-26T06:45:05.899040+0000","last_unstale":"2023-10-26T06:46:31.779508+0000","last_undegraded":"2023-10-26T06:46:31.779508+0000","last_fullsized":"2023-10-26T06:46:31.779508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:55:35.666122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773908+0000","last_change":"2023-10-26T06:45:05.904155+0000","last_active":"2023-10-26T06:46:31.773908+0000","last_peered":"2023-10-26T06:46:31.773908+0000","last_clean":"2023-10-26T06:46:31.773908+0000","last_became_active":"2023-10-26T06:45:05.903734+0000","last_became_peered":"2023-10-26T06:45:05.903734+0000","last_unstale":"2023-10-26T06:46:31.773908+0000","last_undegraded":"2023-10-26T06:46:31.773908+0000","last_fullsized":"2023-10-26T06:46:31.773908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:43.252789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781034+0000","last_change":"2023-10-26T06:45:05.897174+0000","last_active":"2023-10-26T06:46:31.781034+0000","last_peered":"2023-10-26T06:46:31.781034+0000","last_clean":"2023-10-26T06:46:31.781034+0000","last_became_active":"2023-10-26T06:45:05.896893+0000","last_became_peered":"2023-10-26T06:45:05.896893+0000","last_unstale":"2023-10-26T06:46:31.781034+0000","last_undegraded":"2023-10-26T06:46:31.781034+0000","last_fullsized":"2023-10-26T06:46:31.781034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:14.291682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773128+0000","last_change":"2023-10-26T06:45:05.905987+0000","last_active":"2023-10-26T06:46:31.773128+0000","last_peered":"2023-10-26T06:46:31.773128+0000","last_clean":"2023-10-26T06:46:31.773128+0000","last_became_active":"2023-10-26T06:45:05.904933+0000","last_became_peered":"2023-10-26T06:45:05.904933+0000","last_unstale":"2023-10-26T06:46:31.773128+0000","last_undegraded":"2023-10-26T06:46:31.773128+0000","last_fullsized":"2023-10-26T06:46:31.773128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:19.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782354+0000","last_change":"2023-10-26T06:45:05.912781+0000","last_active":"2023-10-26T06:46:31.782354+0000","last_peered":"2023-10-26T06:46:31.782354+0000","last_clean":"2023-10-26T06:46:31.782354+0000","last_became_active":"2023-10-26T06:45:05.912415+0000","last_became_peered":"2023-10-26T06:45:05.912415+0000","last_unstale":"2023-10-26T06:46:31.782354+0000","last_undegraded":"2023-10-26T06:46:31.782354+0000","last_fullsized":"2023-10-26T06:46:31.782354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:25.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779371+0000","last_change":"2023-10-26T06:45:05.909106+0000","last_active":"2023-10-26T06:46:31.779371+0000","last_peered":"2023-10-26T06:46:31.779371+0000","last_clean":"2023-10-26T06:46:31.779371+0000","last_became_active":"2023-10-26T06:45:05.908751+0000","last_became_peered":"2023-10-26T06:45:05.908751+0000","last_unstale":"2023-10-26T06:46:31.779371+0000","last_undegraded":"2023-10-26T06:46:31.779371+0000","last_fullsized":"2023-10-26T06:46:31.779371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:17.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.761209+0000","last_change":"2023-10-26T06:45:05.898262+0000","last_active":"2023-10-26T06:46:31.761209+0000","last_peered":"2023-10-26T06:46:31.761209+0000","last_clean":"2023-10-26T06:46:31.761209+0000","last_became_active":"2023-10-26T06:45:05.897947+0000","last_became_peered":"2023-10-26T06:45:05.897947+0000","last_unstale":"2023-10-26T06:46:31.761209+0000","last_undegraded":"2023-10-26T06:46:31.761209+0000","last_fullsized":"2023-10-26T06:46:31.761209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:39:45.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65317,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":214,"num_read_kb":172,"num_write":77,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":195951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":462460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.624},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48599999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.80700000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.432}]},{"osd":2,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.372},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88100000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":1.52,"15min":1.52},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.012,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95199999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93700000000000006,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47599999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1040000000000001,"5min":1.47,"15min":1.47},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0549999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.74299999999999999}]},{"osd":10,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96199999999999997,"5min":1.327,"15min":1.327},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.036,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.57799999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1117664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44500000000000001,"15min":0.44400000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.498},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82499999999999996,"5min":1.004,"15min":1.004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79800000000000004,"5min":0.874,"15min":0.874},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.627}]},{"osd":5,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41699999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.752},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95599999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.046,"5min":1.133,"15min":1.133},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Oct 26 06:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.73399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0089999999999999,"5min":1.04,"15min":1.04},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.60599999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.097,"15min":1.097},"last":0.748}]}]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":463042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.51400000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":0.69499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.1099999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.6319999999999997,"5min":4.6319999999999997,"15min":4.6319999999999997},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.84499999999999997,"15min":0.86499999999999999},"min":{"1min":0.55200000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93100000000000005,"5min":1.173,"15min":1.173},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.87},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97199999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.877},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.496},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.877,"15min":0.90700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.64600000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93700000000000006,"15min":0.96499999999999997},"min":{"1min":0.59799999999999998,"5min":0.503,"15min":0.503},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.94099999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.762,"5min":1.885,"15min":1.885},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.70099999999999996,"15min":0.77100000000000002},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80000000000000004,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.505}]},{"osd":1,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.628,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.50700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44600000000000001},"min":{"1min":0.34200000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.48799999999999999,"15min":0.48499999999999999},"min":{"1min":0.40000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.68500000000000005,"15min":0.65900000000000003},"min":{"1min":0.49199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85699999999999998,"5min":0.88,"15min":0.88},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0580000000000001,"5min":1.095,"15min":1.095},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.81799999999999995,"15min":0.88200000000000001},"min":{"1min":0.56499999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.038,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.84699999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.63800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.74299999999999999,"15min":0.71899999999999997},"min":{"1min":0.68400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.022,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72899999999999998,"15min":0.69699999999999995},"min":{"1min":0.63400000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Oct 26 06:46:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.86199999999999999}]},{"osd":11,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.94699999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1053533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41699999999999998,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60299999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78500000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.748,"15min":0.748},"last":0.503},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80100000000000005}]},{"osd":8,"last update":"Thu Oct 26 06:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.877},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70799999999999996,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83999999999999997}]},{"osd":9,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.753},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.88200000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.82099999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.747,"15min":0.76100000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.90800000000000003,"5min":1.048,"15min":1.048},"last":0.77500000000000002}]},{"osd":11,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.94699999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.44400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70699999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79100000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.68600000000000005,"15min":0.64300000000000002},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68600000000000005,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":5.5549999999999997,"5min":5.5549999999999997,"15min":5.5549999999999997},"last":0.91500000000000004}]},{"osd":3,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71499999999999997,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.70799999999999996,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.84199999999999997,"5min":1.208,"15min":1.208},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94999999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.385,"15min":1.385},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.749,"15min":0.747},"min":{"1min":0.57499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90600000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.82399999999999995,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":5.532,"5min":5.532,"15min":5.532},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":0.79900000000000004,"15min":0.73299999999999998},"min":{"1min":0.56499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":5.5810000000000004,"5min":5.5810000000000004,"15min":5.5810000000000004},"last":0.876}]},{"osd":8,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.59299999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95299999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Thu Oct 26 06:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92200000000000004,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.82099999999999995,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84899999999999998,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96799999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.93400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":521600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.54100000000000004},"min":{"1min":0.47299999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.92300000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44700000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0209999999999999,"5min":1.256,"15min":1.256},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"min":{"1min":0.498,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.627},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.56899999999999995,"15min":0.52500000000000002},"min":{"1min":0.52600000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.49299999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61099999999999999,"15min":0.55700000000000005},"min":{"1min":0.47499999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.45500000000000002,"15min":0.442},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.373},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.42799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39100000000000001}]},{"osd":5,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.58399999999999996},"min":{"1min":0.496,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.56299999999999994,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.53700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.52400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96199999999999997,"5min":1.03,"15min":1.03},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.497,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.72999999999999998,"15min":0.70399999999999996},"min":{"1min":0.58499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.996,"5min":1.298,"15min":1.298},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.64800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.70199999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.751},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.52300000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.82799999999999996}]}]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 06:46:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65900000000000003,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.61499999999999999},"min":{"1min":0.47399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5820000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.68200000000000005,"15min":0.626},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Oct 26 06:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4450000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.68400000000000005,"15min":0.63100000000000001},"min":{"1min":0.53000000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.749,"15min":0.69399999999999995},"min":{"1min":0.53300000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":0.89900000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:46:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68700000000000006,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.4300000000000002,"5min":3.4300000000000002,"15min":3.4300000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.73599999999999999,"15min":0.68200000000000005},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0099999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Oct 26 06:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.61599999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.054,"5min":0.93700000000000006,"15min":0.92900000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.92300000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.437,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99199999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.626},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.87,"15min":0.89000000000000001},"min":{"1min":0.47299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.544,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.60599999999999998}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.64000000000000001,"5min":0.625,"15min":0.625},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.089,"5min":0.92700000000000005,"15min":0.90000000000000002},"min":{"1min":0.64500000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.85999999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":526344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58099999999999996,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.56499999999999995,"15min":0.52300000000000002},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Oct 26 06:46:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.625},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89700000000000002,"5min":2.887,"15min":2.887},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.501,"15min":0.46899999999999997},"min":{"1min":0.40500000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.56399999999999995,"15min":0.51400000000000001},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.786,"5min":3.786,"15min":3.786},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.623},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79000000000000004,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93400000000000005,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":0.88}]},{"osd":7,"last update":"Thu Oct 26 06:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.78700000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.997,"15min":0.997},"last":1.3480000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89300000000000002,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.68100000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86499999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.70199999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78500000000000003,"15min":0.77100000000000002},"min":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.86199999999999999,"5min":3.661,"15min":3.661},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.58099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":3.431,"15min":3.431},"last":0.95099999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.746,"15min":0.72099999999999997},"min":{"1min":0.51700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4649999999999999,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.77100000000000002,"15min":0.748},"min":{"1min":0.626,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.7040000000000002,"5min":3.7040000000000002,"15min":3.7040000000000002},"last":0.80300000000000005}]},{"osd":11,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78100000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":462479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44400000000000001,"15min":0.40899999999999997},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48399999999999999,"15min":0.45200000000000001},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.722,"15min":1.722},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.60599999999999998},"min":{"1min":0.48799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.875,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94599999999999995,"5min":1.427,"15min":1.427},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85599999999999998,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Oct 26 06:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":1.667,"15min":1.667},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.304,"15min":1.304},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.55600000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96699999999999997,"5min":1.456,"15min":1.456},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75,"15min":0.70199999999999996},"min":{"1min":0.623,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.115,"5min":1.637,"15min":1.637},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90800000000000003,"5min":1.589,"15min":1.589},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64700000000000002,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.516,"15min":1.516},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Oct 26 06:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96999999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.69899999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.85299999999999998,"15min":0.86099999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.3200000000000001,"5min":1.395,"15min":1.395},"last":0.875},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.92400000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0780000000000001,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.85399999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.434}]},{"osd":1,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.35299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.38},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.57899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66400000000000003,"5min":1.222,"15min":1.222},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:46:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.48199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.81899999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86199999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.28,"15min":1.28},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84599999999999997,"5min":1.403,"15min":1.403},"last":0.57499999999999996}]},{"osd":8,"last update":"Thu Oct 26 06:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.63100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88200000000000001,"5min":0.998,"15min":0.998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.623,"5min":0.433,"15min":0.433},"max":{"1min":0.82099999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.72699999999999998,"15min":0.71299999999999997},"min":{"1min":0.56999999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.748},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.024,"5min":1.175,"15min":1.175},"last":0.77000000000000002}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94099999999999995,"5min":1.141,"15min":1.141},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91700000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Oct 26 06:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78900000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1053133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":0.871,"15min":0.871},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59699999999999998,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.85799999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.313,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73599999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.46200000000000002,"15min":0.439},"min":{"1min":0.37,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.496}]},{"osd":3,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.748,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.48599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75}]},{"osd":4,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63500000000000001,"15min":0.60999999999999999},"min":{"1min":0.54400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68100000000000005,"15min":0.64900000000000002},"min":{"1min":0.59699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70199999999999996,"15min":0.66400000000000003},"min":{"1min":0.63100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84399999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.56200000000000006,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.874}]},{"osd":8,"last update":"Thu Oct 26 06:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66500000000000004,"15min":0.63900000000000001},"min":{"1min":0.42099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64500000000000002}]},{"osd":9,"last update":"Thu Oct 26 06:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.66000000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.83999999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.66500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-26T06:46:47.936 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T06:46:48.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 -- 172.21.15.89:0/2566828387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1066b0 msgr2=0x7fe66c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 --2- 172.21.15.89:0/2566828387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1066b0 0x7fe66c106a90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe668009990 tx=0x7fe668009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 -- 172.21.15.89:0/2566828387 shutdown_connections 2023-10-26T06:46:48.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 --2- 172.21.15.89:0/2566828387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1066b0 0x7fe66c106a90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 --2- 172.21.15.89:0/2566828387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c101060 0x7fe66c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 --2- 172.21.15.89:0/2566828387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c1006a0 0x7fe66c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 -- 172.21.15.89:0/2566828387 >> 172.21.15.89:0/2566828387 conn(0x7fe66c0fc1d0 msgr2=0x7fe66c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 -- 172.21.15.89:0/2566828387 shutdown_connections 2023-10-26T06:46:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.165+0000 7fe6745ff700 1 -- 172.21.15.89:0/2566828387 wait complete. 2023-10-26T06:46:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 Processor -- start 2023-10-26T06:46:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 -- start start 2023-10-26T06:46:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1006a0 0x7fe66c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c101060 0x7fe66c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c1066b0 0x7fe66c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19cb60 con 0x7fe66c1006a0 2023-10-26T06:46:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19ccd0 con 0x7fe66c101060 2023-10-26T06:46:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe6745ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19ce90 con 0x7fe66c1066b0 2023-10-26T06:46:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe67239b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1006a0 0x7fe66c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe67239b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1006a0 0x7fe66c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54120/0 (socket says 172.21.15.89:54120) 2023-10-26T06:46:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.166+0000 7fe67239b700 1 -- 172.21.15.89:0/417434436 learned_addr learned my addr 172.21.15.89:0/417434436 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe671b9a700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c101060 0x7fe66c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe672b9c700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c1066b0 0x7fe66c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe67239b700 1 -- 172.21.15.89:0/417434436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c1066b0 msgr2=0x7fe66c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe67239b700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c1066b0 0x7fe66c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe67239b700 1 -- 172.21.15.89:0/417434436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c101060 msgr2=0x7fe66c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe67239b700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c101060 0x7fe66c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe67239b700 1 -- 172.21.15.89:0/417434436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6680095f0 con 0x7fe66c1006a0 2023-10-26T06:46:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe67239b700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1006a0 0x7fe66c198d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe668000c00 tx=0x7fe668005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe6637fe700 1 -- 172.21.15.89:0/417434436 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66801d070 con 0x7fe66c1006a0 2023-10-26T06:46:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe6637fe700 1 -- 172.21.15.89:0/417434436 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe668022470 con 0x7fe66c1006a0 2023-10-26T06:46:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c19d110 con 0x7fe66c1006a0 2023-10-26T06:46:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.167+0000 7fe6637fe700 1 -- 172.21.15.89:0/417434436 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668017650 con 0x7fe66c1006a0 2023-10-26T06:46:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.168+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c1a5ad0 con 0x7fe66c1006a0 2023-10-26T06:46:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.169+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c101de0 con 0x7fe66c1006a0 2023-10-26T06:46:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.170+0000 7fe6637fe700 1 -- 172.21.15.89:0/417434436 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fe6680225e0 con 0x7fe66c1006a0 2023-10-26T06:46:48.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.172+0000 7fe6637fe700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6580a1550 0x7fe6580a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.172+0000 7fe6637fe700 1 -- 172.21.15.89:0/417434436 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fe6680ce0e0 con 0x7fe66c1006a0 2023-10-26T06:46:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.172+0000 7fe671b9a700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6580a1550 0x7fe6580a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.172+0000 7fe671b9a700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6580a1550 0x7fe6580a3a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe66c19a240 tx=0x7fe65c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.176+0000 7fe6637fe700 1 -- 172.21.15.89:0/417434436 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe6680976f0 con 0x7fe66c1006a0 2023-10-26T06:46:48.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:48 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2398269996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:48.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.372+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe66c110620 con 0x7fe6580a1550 2023-10-26T06:46:48.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.383+0000 7fe6637fe700 1 -- 172.21.15.89:0/417434436 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344469 (secure 0 0 0) 0x7fe66c110620 con 0x7fe6580a1550 2023-10-26T06:46:48.389 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6580a1550 msgr2=0x7fe6580a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6580a1550 0x7fe6580a3a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe66c19a240 tx=0x7fe65c009260 comp rx=0 tx=0).stop 2023-10-26T06:46:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1006a0 msgr2=0x7fe66c198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1006a0 0x7fe66c198d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe668000c00 tx=0x7fe668005740 comp rx=0 tx=0).stop 2023-10-26T06:46:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 shutdown_connections 2023-10-26T06:46:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe6580a1550 0x7fe6580a3a10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe66c1006a0 0x7fe66c198d30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c1066b0 0x7fe66c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 --2- 172.21.15.89:0/417434436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe66c101060 0x7fe66c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 >> 172.21.15.89:0/417434436 conn(0x7fe66c0fc1d0 msgr2=0x7fe66c10be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 shutdown_connections 2023-10-26T06:46:48.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.387+0000 7fe6745ff700 1 -- 172.21.15.89:0/417434436 wait complete. 2023-10-26T06:46:48.396 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-26T06:46:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-26T06:46:47.738531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655931,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":350,"num_read_kb":291,"num_write":260,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":706,"ondisk_log_size":706,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337208,"kb_used_data":13016,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604616,"statfs":{"total":1151940427776,"available":1151595126784,"internally_reserved":0,"allocated":13328384,"data_stored":7510603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002855"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782197+0000","last_change":"2023-10-26T06:45:05.903170+0000","last_active":"2023-10-26T06:46:31.782197+0000","last_peered":"2023-10-26T06:46:31.782197+0000","last_clean":"2023-10-26T06:46:31.782197+0000","last_became_active":"2023-10-26T06:45:05.902574+0000","last_became_peered":"2023-10-26T06:45:05.902574+0000","last_unstale":"2023-10-26T06:46:31.782197+0000","last_undegraded":"2023-10-26T06:46:31.782197+0000","last_fullsized":"2023-10-26T06:46:31.782197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:33.065828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778600+0000","last_change":"2023-10-26T06:45:05.911249+0000","last_active":"2023-10-26T06:46:31.778600+0000","last_peered":"2023-10-26T06:46:31.778600+0000","last_clean":"2023-10-26T06:46:31.778600+0000","last_became_active":"2023-10-26T06:45:05.910859+0000","last_became_peered":"2023-10-26T06:45:05.910859+0000","last_unstale":"2023-10-26T06:46:31.778600+0000","last_undegraded":"2023-10-26T06:46:31.778600+0000","last_fullsized":"2023-10-26T06:46:31.778600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:07:13.362460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780029+0000","last_change":"2023-10-26T06:45:05.909219+0000","last_active":"2023-10-26T06:46:31.780029+0000","last_peered":"2023-10-26T06:46:31.780029+0000","last_clean":"2023-10-26T06:46:31.780029+0000","last_became_active":"2023-10-26T06:45:05.906721+0000","last_became_peered":"2023-10-26T06:45:05.906721+0000","last_unstale":"2023-10-26T06:46:31.780029+0000","last_undegraded":"2023-10-26T06:46:31.780029+0000","last_fullsized":"2023-10-26T06:46:31.780029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:41.461357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.762023+0000","last_change":"2023-10-26T06:45:05.897514+0000","last_active":"2023-10-26T06:46:31.762023+0000","last_peered":"2023-10-26T06:46:31.762023+0000","last_clean":"2023-10-26T06:46:31.762023+0000","last_became_active":"2023-10-26T06:45:05.897201+0000","last_became_peered":"2023-10-26T06:45:05.897201+0000","last_unstale":"2023-10-26T06:46:31.762023+0000","last_undegraded":"2023-10-26T06:46:31.762023+0000","last_fullsized":"2023-10-26T06:46:31.762023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:39.508654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780489+0000","last_change":"2023-10-26T06:45:05.900475+0000","last_active":"2023-10-26T06:46:31.780489+0000","last_peered":"2023-10-26T06:46:31.780489+0000","last_clean":"2023-10-26T06:46:31.780489+0000","last_became_active":"2023-10-26T06:45:05.900313+0000","last_became_peered":"2023-10-26T06:45:05.900313+0000","last_unstale":"2023-10-26T06:46:31.780489+0000","last_undegraded":"2023-10-26T06:46:31.780489+0000","last_fullsized":"2023-10-26T06:46:31.780489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:51.438741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764631+0000","last_change":"2023-10-26T06:45:05.909040+0000","last_active":"2023-10-26T06:46:31.764631+0000","last_peered":"2023-10-26T06:46:31.764631+0000","last_clean":"2023-10-26T06:46:31.764631+0000","last_became_active":"2023-10-26T06:45:05.908696+0000","last_became_peered":"2023-10-26T06:45:05.908696+0000","last_unstale":"2023-10-26T06:46:31.764631+0000","last_undegraded":"2023-10-26T06:46:31.764631+0000","last_fullsized":"2023-10-26T06:46:31.764631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:00.147712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764627+0000","last_change":"2023-10-26T06:45:05.898356+0000","last_active":"2023-10-26T06:46:31.764627+0000","last_peered":"2023-10-26T06:46:31.764627+0000","last_clean":"2023-10-26T06:46:31.764627+0000","last_became_active":"2023-10-26T06:45:05.898008+0000","last_became_peered":"2023-10-26T06:45:05.898008+0000","last_unstale":"2023-10-26T06:46:31.764627+0000","last_undegraded":"2023-10-26T06:46:31.764627+0000","last_fullsized":"2023-10-26T06:46:31.764627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:23.209987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781661+0000","last_change":"2023-10-26T06:45:05.911301+0000","last_active":"2023-10-26T06:46:31.781661+0000","last_peered":"2023-10-26T06:46:31.781661+0000","last_clean":"2023-10-26T06:46:31.781661+0000","last_became_active":"2023-10-26T06:45:05.910667+0000","last_became_peered":"2023-10-26T06:45:05.910667+0000","last_unstale":"2023-10-26T06:46:31.781661+0000","last_undegraded":"2023-10-26T06:46:31.781661+0000","last_fullsized":"2023-10-26T06:46:31.781661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:03:36.856248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.777770+0000","last_change":"2023-10-26T06:45:05.904456+0000","last_active":"2023-10-26T06:46:31.777770+0000","last_peered":"2023-10-26T06:46:31.777770+0000","last_clean":"2023-10-26T06:46:31.777770+0000","last_became_active":"2023-10-26T06:45:05.904165+0000","last_became_peered":"2023-10-26T06:45:05.904165+0000","last_unstale":"2023-10-26T06:46:31.777770+0000","last_undegraded":"2023-10-26T06:46:31.777770+0000","last_fullsized":"2023-10-26T06:46:31.777770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:51:01.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.768918+0000","last_change":"2023-10-26T06:46:03.232421+0000","last_active":"2023-10-26T06:46:31.768918+0000","last_peered":"2023-10-26T06:46:31.768918+0000","last_clean":"2023-10-26T06:46:31.768918+0000","last_became_active":"2023-10-26T06:46:03.232019+0000","last_became_peered":"2023-10-26T06:46:03.232019+0000","last_unstale":"2023-10-26T06:46:31.768918+0000","last_undegraded":"2023-10-26T06:46:31.768918+0000","last_fullsized":"2023-10-26T06:46:31.768918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:56.506294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779141+0000","last_change":"2023-10-26T06:45:05.906549+0000","last_active":"2023-10-26T06:46:31.779141+0000","last_peered":"2023-10-26T06:46:31.779141+0000","last_clean":"2023-10-26T06:46:31.779141+0000","last_became_active":"2023-10-26T06:45:05.905883+0000","last_became_peered":"2023-10-26T06:45:05.905883+0000","last_unstale":"2023-10-26T06:46:31.779141+0000","last_undegraded":"2023-10-26T06:46:31.779141+0000","last_fullsized":"2023-10-26T06:46:31.779141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:54.771031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763970+0000","last_change":"2023-10-26T06:46:03.230365+0000","last_active":"2023-10-26T06:46:31.763970+0000","last_peered":"2023-10-26T06:46:31.763970+0000","last_clean":"2023-10-26T06:46:31.763970+0000","last_became_active":"2023-10-26T06:46:03.230101+0000","last_became_peered":"2023-10-26T06:46:03.230101+0000","last_unstale":"2023-10-26T06:46:31.763970+0000","last_undegraded":"2023-10-26T06:46:31.763970+0000","last_fullsized":"2023-10-26T06:46:31.763970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:13.903128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779665+0000","last_change":"2023-10-26T06:45:05.908305+0000","last_active":"2023-10-26T06:46:31.779665+0000","last_peered":"2023-10-26T06:46:31.779665+0000","last_clean":"2023-10-26T06:46:31.779665+0000","last_became_active":"2023-10-26T06:45:05.907931+0000","last_became_peered":"2023-10-26T06:45:05.907931+0000","last_unstale":"2023-10-26T06:46:31.779665+0000","last_undegraded":"2023-10-26T06:46:31.779665+0000","last_fullsized":"2023-10-26T06:46:31.779665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:00:43.605324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.769635+0000","last_change":"2023-10-26T06:45:05.909737+0000","last_active":"2023-10-26T06:46:31.769635+0000","last_peered":"2023-10-26T06:46:31.769635+0000","last_clean":"2023-10-26T06:46:31.769635+0000","last_became_active":"2023-10-26T06:45:05.909445+0000","last_became_peered":"2023-10-26T06:45:05.909445+0000","last_unstale":"2023-10-26T06:46:31.769635+0000","last_undegraded":"2023-10-26T06:46:31.769635+0000","last_fullsized":"2023-10-26T06:46:31.769635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:38.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778290+0000","last_change":"2023-10-26T06:45:05.907722+0000","last_active":"2023-10-26T06:46:31.778290+0000","last_peered":"2023-10-26T06:46:31.778290+0000","last_clean":"2023-10-26T06:46:31.778290+0000","last_became_active":"2023-10-26T06:45:05.905622+0000","last_became_peered":"2023-10-26T06:45:05.905622+0000","last_unstale":"2023-10-26T06:46:31.778290+0000","last_undegraded":"2023-10-26T06:46:31.778290+0000","last_fullsized":"2023-10-26T06:46:31.778290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:30:01.048032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779200+0000","last_change":"2023-10-26T06:45:05.907801+0000","last_active":"2023-10-26T06:46:31.779200+0000","last_peered":"2023-10-26T06:46:31.779200+0000","last_clean":"2023-10-26T06:46:31.779200+0000","last_became_active":"2023-10-26T06:45:05.907485+0000","last_became_peered":"2023-10-26T06:45:05.907485+0000","last_unstale":"2023-10-26T06:46:31.779200+0000","last_undegraded":"2023-10-26T06:46:31.779200+0000","last_fullsized":"2023-10-26T06:46:31.779200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:16.850992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782324+0000","last_change":"2023-10-26T06:45:05.902859+0000","last_active":"2023-10-26T06:46:31.782324+0000","last_peered":"2023-10-26T06:46:31.782324+0000","last_clean":"2023-10-26T06:46:31.782324+0000","last_became_active":"2023-10-26T06:45:05.902218+0000","last_became_peered":"2023-10-26T06:45:05.902218+0000","last_unstale":"2023-10-26T06:46:31.782324+0000","last_undegraded":"2023-10-26T06:46:31.782324+0000","last_fullsized":"2023-10-26T06:46:31.782324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:47.717227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763315+0000","last_change":"2023-10-26T06:45:05.910445+0000","last_active":"2023-10-26T06:46:31.763315+0000","last_peered":"2023-10-26T06:46:31.763315+0000","last_clean":"2023-10-26T06:46:31.763315+0000","last_became_active":"2023-10-26T06:45:05.910043+0000","last_became_peered":"2023-10-26T06:45:05.910043+0000","last_unstale":"2023-10-26T06:46:31.763315+0000","last_undegraded":"2023-10-26T06:46:31.763315+0000","last_fullsized":"2023-10-26T06:46:31.763315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:33.751134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775030+0000","last_change":"2023-10-26T06:45:05.905340+0000","last_active":"2023-10-26T06:46:31.775030+0000","last_peered":"2023-10-26T06:46:31.775030+0000","last_clean":"2023-10-26T06:46:31.775030+0000","last_became_active":"2023-10-26T06:45:05.905030+0000","last_became_peered":"2023-10-26T06:45:05.905030+0000","last_unstale":"2023-10-26T06:46:31.775030+0000","last_undegraded":"2023-10-26T06:46:31.775030+0000","last_fullsized":"2023-10-26T06:46:31.775030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.303210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226621+0000","last_change":"2023-10-26T06:45:04.919128+0000","last_active":"2023-10-26T06:46:02.226621+0000","last_peered":"2023-10-26T06:46:02.226621+0000","last_clean":"2023-10-26T06:46:02.226621+0000","last_became_active":"2023-10-26T06:45:04.913034+0000","last_became_peered":"2023-10-26T06:45:04.913034+0000","last_unstale":"2023-10-26T06:46:02.226621+0000","last_undegraded":"2023-10-26T06:46:02.226621+0000","last_fullsized":"2023-10-26T06:46:02.226621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:50.835037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.771222+0000","last_change":"2023-10-26T06:45:05.905315+0000","last_active":"2023-10-26T06:46:31.771222+0000","last_peered":"2023-10-26T06:46:31.771222+0000","last_clean":"2023-10-26T06:46:31.771222+0000","last_became_active":"2023-10-26T06:45:05.904260+0000","last_became_peered":"2023-10-26T06:45:05.904260+0000","last_unstale":"2023-10-26T06:46:31.771222+0000","last_undegraded":"2023-10-26T06:46:31.771222+0000","last_fullsized":"2023-10-26T06:46:31.771222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:02.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230870+0000","last_change":"2023-10-26T06:45:04.920440+0000","last_active":"2023-10-26T06:46:02.230870+0000","last_peered":"2023-10-26T06:46:02.230870+0000","last_clean":"2023-10-26T06:46:02.230870+0000","last_became_active":"2023-10-26T06:45:04.920129+0000","last_became_peered":"2023-10-26T06:45:04.920129+0000","last_unstale":"2023-10-26T06:46:02.230870+0000","last_undegraded":"2023-10-26T06:46:02.230870+0000","last_fullsized":"2023-10-26T06:46:02.230870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:43.176830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780674+0000","last_change":"2023-10-26T06:45:05.902898+0000","last_active":"2023-10-26T06:46:31.780674+0000","last_peered":"2023-10-26T06:46:31.780674+0000","last_clean":"2023-10-26T06:46:31.780674+0000","last_became_active":"2023-10-26T06:45:05.902161+0000","last_became_peered":"2023-10-26T06:45:05.902161+0000","last_unstale":"2023-10-26T06:46:31.780674+0000","last_undegraded":"2023-10-26T06:46:31.780674+0000","last_fullsized":"2023-10-26T06:46:31.780674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:27.872225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227991+0000","last_change":"2023-10-26T06:45:04.911835+0000","last_active":"2023-10-26T06:46:02.227991+0000","last_peered":"2023-10-26T06:46:02.227991+0000","last_clean":"2023-10-26T06:46:02.227991+0000","last_became_active":"2023-10-26T06:45:04.911505+0000","last_became_peered":"2023-10-26T06:45:04.911505+0000","last_unstale":"2023-10-26T06:46:02.227991+0000","last_undegraded":"2023-10-26T06:46:02.227991+0000","last_fullsized":"2023-10-26T06:46:02.227991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:01:18.971381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779287+0000","last_change":"2023-10-26T06:45:05.909088+0000","last_active":"2023-10-26T06:46:31.779287+0000","last_peered":"2023-10-26T06:46:31.779287+0000","last_clean":"2023-10-26T06:46:31.779287+0000","last_became_active":"2023-10-26T06:45:05.908528+0000","last_became_peered":"2023-10-26T06:45:05.908528+0000","last_unstale":"2023-10-26T06:46:31.779287+0000","last_undegraded":"2023-10-26T06:46:31.779287+0000","last_fullsized":"2023-10-26T06:46:31.779287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:07.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.237699+0000","last_change":"2023-10-26T06:46:03.236510+0000","last_active":"2023-10-26T06:46:03.237699+0000","last_peered":"2023-10-26T06:46:03.237699+0000","last_clean":"2023-10-26T06:46:03.237699+0000","last_became_active":"2023-10-26T06:46:03.233056+0000","last_became_peered":"2023-10-26T06:46:03.233056+0000","last_unstale":"2023-10-26T06:46:03.237699+0000","last_undegraded":"2023-10-26T06:46:03.237699+0000","last_fullsized":"2023-10-26T06:46:03.237699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:33.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.777342+0000","last_change":"2023-10-26T06:45:05.898795+0000","last_active":"2023-10-26T06:46:31.777342+0000","last_peered":"2023-10-26T06:46:31.777342+0000","last_clean":"2023-10-26T06:46:31.777342+0000","last_became_active":"2023-10-26T06:45:05.898489+0000","last_became_peered":"2023-10-26T06:45:05.898489+0000","last_unstale":"2023-10-26T06:46:31.777342+0000","last_undegraded":"2023-10-26T06:46:31.777342+0000","last_fullsized":"2023-10-26T06:46:31.777342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:55.343736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230147+0000","last_change":"2023-10-26T06:45:04.908881+0000","last_active":"2023-10-26T06:46:02.230147+0000","last_peered":"2023-10-26T06:46:02.230147+0000","last_clean":"2023-10-26T06:46:02.230147+0000","last_became_active":"2023-10-26T06:45:04.908515+0000","last_became_peered":"2023-10-26T06:45:04.908515+0000","last_unstale":"2023-10-26T06:46:02.230147+0000","last_undegraded":"2023-10-26T06:46:02.230147+0000","last_fullsized":"2023-10-26T06:46:02.230147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:39.200294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773736+0000","last_change":"2023-10-26T06:46:03.237686+0000","last_active":"2023-10-26T06:46:31.773736+0000","last_peered":"2023-10-26T06:46:31.773736+0000","last_clean":"2023-10-26T06:46:31.773736+0000","last_became_active":"2023-10-26T06:46:03.237275+0000","last_became_peered":"2023-10-26T06:46:03.237275+0000","last_unstale":"2023-10-26T06:46:31.773736+0000","last_undegraded":"2023-10-26T06:46:31.773736+0000","last_fullsized":"2023-10-26T06:46:31.773736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:56:22.281364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.234834+0000","last_change":"2023-10-26T06:46:03.234834+0000","last_active":"2023-10-26T06:46:03.234834+0000","last_peered":"2023-10-26T06:46:03.234834+0000","last_clean":"2023-10-26T06:46:03.234834+0000","last_became_active":"2023-10-26T06:46:03.234289+0000","last_became_peered":"2023-10-26T06:46:03.234289+0000","last_unstale":"2023-10-26T06:46:03.234834+0000","last_undegraded":"2023-10-26T06:46:03.234834+0000","last_fullsized":"2023-10-26T06:46:03.234834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:50:53.362840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773846+0000","last_change":"2023-10-26T06:45:05.901825+0000","last_active":"2023-10-26T06:46:31.773846+0000","last_peered":"2023-10-26T06:46:31.773846+0000","last_clean":"2023-10-26T06:46:31.773846+0000","last_became_active":"2023-10-26T06:45:05.901509+0000","last_became_peered":"2023-10-26T06:45:05.901509+0000","last_unstale":"2023-10-26T06:46:31.773846+0000","last_undegraded":"2023-10-26T06:46:31.773846+0000","last_fullsized":"2023-10-26T06:46:31.773846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:16.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227045+0000","last_change":"2023-10-26T06:45:04.919813+0000","last_active":"2023-10-26T06:46:02.227045+0000","last_peered":"2023-10-26T06:46:02.227045+0000","last_clean":"2023-10-26T06:46:02.227045+0000","last_became_active":"2023-10-26T06:45:04.918016+0000","last_became_peered":"2023-10-26T06:45:04.918016+0000","last_unstale":"2023-10-26T06:46:02.227045+0000","last_undegraded":"2023-10-26T06:46:02.227045+0000","last_fullsized":"2023-10-26T06:46:02.227045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:32.336126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779735+0000","last_change":"2023-10-26T06:45:05.902182+0000","last_active":"2023-10-26T06:46:31.779735+0000","last_peered":"2023-10-26T06:46:31.779735+0000","last_clean":"2023-10-26T06:46:31.779735+0000","last_became_active":"2023-10-26T06:45:05.901428+0000","last_became_peered":"2023-10-26T06:45:05.901428+0000","last_unstale":"2023-10-26T06:46:31.779735+0000","last_undegraded":"2023-10-26T06:46:31.779735+0000","last_fullsized":"2023-10-26T06:46:31.779735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:12.722532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.229834+0000","last_change":"2023-10-26T06:45:04.909713+0000","last_active":"2023-10-26T06:46:02.229834+0000","last_peered":"2023-10-26T06:46:02.229834+0000","last_clean":"2023-10-26T06:46:02.229834+0000","last_became_active":"2023-10-26T06:45:04.909412+0000","last_became_peered":"2023-10-26T06:45:04.909412+0000","last_unstale":"2023-10-26T06:46:02.229834+0000","last_undegraded":"2023-10-26T06:46:02.229834+0000","last_fullsized":"2023-10-26T06:46:02.229834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:23.901745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773996+0000","last_change":"2023-10-26T06:45:05.908348+0000","last_active":"2023-10-26T06:46:31.773996+0000","last_peered":"2023-10-26T06:46:31.773996+0000","last_clean":"2023-10-26T06:46:31.773996+0000","last_became_active":"2023-10-26T06:45:05.908022+0000","last_became_peered":"2023-10-26T06:45:05.908022+0000","last_unstale":"2023-10-26T06:46:31.773996+0000","last_undegraded":"2023-10-26T06:46:31.773996+0000","last_fullsized":"2023-10-26T06:46:31.773996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:22:16.027732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.223807+0000","last_change":"2023-10-26T06:45:04.915491+0000","last_active":"2023-10-26T06:46:02.223807+0000","last_peered":"2023-10-26T06:46:02.223807+0000","last_clean":"2023-10-26T06:46:02.223807+0000","last_became_active":"2023-10-26T06:45:04.915191+0000","last_became_peered":"2023-10-26T06:45:04.915191+0000","last_unstale":"2023-10-26T06:46:02.223807+0000","last_undegraded":"2023-10-26T06:46:02.223807+0000","last_fullsized":"2023-10-26T06:46:02.223807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:57.758903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779813+0000","last_change":"2023-10-26T06:45:05.912676+0000","last_active":"2023-10-26T06:46:31.779813+0000","last_peered":"2023-10-26T06:46:31.779813+0000","last_clean":"2023-10-26T06:46:31.779813+0000","last_became_active":"2023-10-26T06:45:05.912372+0000","last_became_peered":"2023-10-26T06:45:05.912372+0000","last_unstale":"2023-10-26T06:46:31.779813+0000","last_undegraded":"2023-10-26T06:46:31.779813+0000","last_fullsized":"2023-10-26T06:46:31.779813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:33.299166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.474091+0000","last_change":"2023-10-26T06:45:04.912123+0000","last_active":"2023-10-26T06:46:02.474091+0000","last_peered":"2023-10-26T06:46:02.474091+0000","last_clean":"2023-10-26T06:46:02.474091+0000","last_became_active":"2023-10-26T06:45:04.911893+0000","last_became_peered":"2023-10-26T06:45:04.911893+0000","last_unstale":"2023-10-26T06:46:02.474091+0000","last_undegraded":"2023-10-26T06:46:02.474091+0000","last_fullsized":"2023-10-26T06:46:02.474091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:32.907496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780208+0000","last_change":"2023-10-26T06:45:05.893434+0000","last_active":"2023-10-26T06:46:31.780208+0000","last_peered":"2023-10-26T06:46:31.780208+0000","last_clean":"2023-10-26T06:46:31.780208+0000","last_became_active":"2023-10-26T06:45:05.893100+0000","last_became_peered":"2023-10-26T06:45:05.893100+0000","last_unstale":"2023-10-26T06:46:31.780208+0000","last_undegraded":"2023-10-26T06:46:31.780208+0000","last_fullsized":"2023-10-26T06:46:31.780208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:22:48.317428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230570+0000","last_change":"2023-10-26T06:45:04.908910+0000","last_active":"2023-10-26T06:46:02.230570+0000","last_peered":"2023-10-26T06:46:02.230570+0000","last_clean":"2023-10-26T06:46:02.230570+0000","last_became_active":"2023-10-26T06:45:04.908592+0000","last_became_peered":"2023-10-26T06:45:04.908592+0000","last_unstale":"2023-10-26T06:46:02.230570+0000","last_undegraded":"2023-10-26T06:46:02.230570+0000","last_fullsized":"2023-10-26T06:46:02.230570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:40.266013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.776098+0000","last_change":"2023-10-26T06:45:05.908887+0000","last_active":"2023-10-26T06:46:31.776098+0000","last_peered":"2023-10-26T06:46:31.776098+0000","last_clean":"2023-10-26T06:46:31.776098+0000","last_became_active":"2023-10-26T06:45:05.908163+0000","last_became_peered":"2023-10-26T06:45:05.908163+0000","last_unstale":"2023-10-26T06:46:31.776098+0000","last_undegraded":"2023-10-26T06:46:31.776098+0000","last_fullsized":"2023-10-26T06:46:31.776098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:09.339374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":253,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:47.249475+0000","last_change":"2023-10-26T06:45:04.905691+0000","last_active":"2023-10-26T06:46:47.249475+0000","last_peered":"2023-10-26T06:46:47.249475+0000","last_clean":"2023-10-26T06:46:47.249475+0000","last_became_active":"2023-10-26T06:45:04.905378+0000","last_became_peered":"2023-10-26T06:45:04.905378+0000","last_unstale":"2023-10-26T06:46:47.249475+0000","last_undegraded":"2023-10-26T06:46:47.249475+0000","last_fullsized":"2023-10-26T06:46:47.249475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:24:21.498141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64519,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":208,"num_read_kb":166,"num_write":30,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.765219+0000","last_change":"2023-10-26T06:45:05.897366+0000","last_active":"2023-10-26T06:46:31.765219+0000","last_peered":"2023-10-26T06:46:31.765219+0000","last_clean":"2023-10-26T06:46:31.765219+0000","last_became_active":"2023-10-26T06:45:05.897091+0000","last_became_peered":"2023-10-26T06:45:05.897091+0000","last_unstale":"2023-10-26T06:46:31.765219+0000","last_undegraded":"2023-10-26T06:46:31.765219+0000","last_fullsized":"2023-10-26T06:46:31.765219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:11.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226619+0000","last_change":"2023-10-26T06:45:04.914949+0000","last_active":"2023-10-26T06:46:02.226619+0000","last_peered":"2023-10-26T06:46:02.226619+0000","last_clean":"2023-10-26T06:46:02.226619+0000","last_became_active":"2023-10-26T06:45:04.914610+0000","last_became_peered":"2023-10-26T06:45:04.914610+0000","last_unstale":"2023-10-26T06:46:02.226619+0000","last_undegraded":"2023-10-26T06:46:02.226619+0000","last_fullsized":"2023-10-26T06:46:02.226619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:16.683352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774454+0000","last_change":"2023-10-26T06:45:05.911067+0000","last_active":"2023-10-26T06:46:31.774454+0000","last_peered":"2023-10-26T06:46:31.774454+0000","last_clean":"2023-10-26T06:46:31.774454+0000","last_became_active":"2023-10-26T06:45:05.908460+0000","last_became_peered":"2023-10-26T06:45:05.908460+0000","last_unstale":"2023-10-26T06:46:31.774454+0000","last_undegraded":"2023-10-26T06:46:31.774454+0000","last_fullsized":"2023-10-26T06:46:31.774454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:17.461781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230798+0000","last_change":"2023-10-26T06:45:04.919947+0000","last_active":"2023-10-26T06:46:02.230798+0000","last_peered":"2023-10-26T06:46:02.230798+0000","last_clean":"2023-10-26T06:46:02.230798+0000","last_became_active":"2023-10-26T06:45:04.919535+0000","last_became_peered":"2023-10-26T06:45:04.919535+0000","last_unstale":"2023-10-26T06:46:02.230798+0000","last_undegraded":"2023-10-26T06:46:02.230798+0000","last_fullsized":"2023-10-26T06:46:02.230798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:46:09.201139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782275+0000","last_change":"2023-10-26T06:45:05.906381+0000","last_active":"2023-10-26T06:46:31.782275+0000","last_peered":"2023-10-26T06:46:31.782275+0000","last_clean":"2023-10-26T06:46:31.782275+0000","last_became_active":"2023-10-26T06:45:05.904487+0000","last_became_peered":"2023-10-26T06:45:05.904487+0000","last_unstale":"2023-10-26T06:46:31.782275+0000","last_undegraded":"2023-10-26T06:46:31.782275+0000","last_fullsized":"2023-10-26T06:46:31.782275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:00.164936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227232+0000","last_change":"2023-10-26T06:45:04.908779+0000","last_active":"2023-10-26T06:46:02.227232+0000","last_peered":"2023-10-26T06:46:02.227232+0000","last_clean":"2023-10-26T06:46:02.227232+0000","last_became_active":"2023-10-26T06:45:04.908490+0000","last_became_peered":"2023-10-26T06:45:04.908490+0000","last_unstale":"2023-10-26T06:46:02.227232+0000","last_undegraded":"2023-10-26T06:46:02.227232+0000","last_fullsized":"2023-10-26T06:46:02.227232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:12:13.199863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764235+0000","last_change":"2023-10-26T06:45:05.909408+0000","last_active":"2023-10-26T06:46:31.764235+0000","last_peered":"2023-10-26T06:46:31.764235+0000","last_clean":"2023-10-26T06:46:31.764235+0000","last_became_active":"2023-10-26T06:45:05.909107+0000","last_became_peered":"2023-10-26T06:45:05.909107+0000","last_unstale":"2023-10-26T06:46:31.764235+0000","last_undegraded":"2023-10-26T06:46:31.764235+0000","last_fullsized":"2023-10-26T06:46:31.764235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:05.887472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.424977+0000","last_change":"2023-10-26T06:46:03.424977+0000","last_active":"2023-10-26T06:46:03.424977+0000","last_peered":"2023-10-26T06:46:03.424977+0000","last_clean":"2023-10-26T06:46:03.424977+0000","last_became_active":"2023-10-26T06:46:03.417899+0000","last_became_peered":"2023-10-26T06:46:03.417899+0000","last_unstale":"2023-10-26T06:46:03.424977+0000","last_undegraded":"2023-10-26T06:46:03.424977+0000","last_fullsized":"2023-10-26T06:46:03.424977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:51.092657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775545+0000","last_change":"2023-10-26T06:45:05.902156+0000","last_active":"2023-10-26T06:46:31.775545+0000","last_peered":"2023-10-26T06:46:31.775545+0000","last_clean":"2023-10-26T06:46:31.775545+0000","last_became_active":"2023-10-26T06:45:05.901865+0000","last_became_peered":"2023-10-26T06:45:05.901865+0000","last_unstale":"2023-10-26T06:46:31.775545+0000","last_undegraded":"2023-10-26T06:46:31.775545+0000","last_fullsized":"2023-10-26T06:46:31.775545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:06:08.167280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.228254+0000","last_change":"2023-10-26T06:45:04.913071+0000","last_active":"2023-10-26T06:46:02.228254+0000","last_peered":"2023-10-26T06:46:02.228254+0000","last_clean":"2023-10-26T06:46:02.228254+0000","last_became_active":"2023-10-26T06:45:04.912736+0000","last_became_peered":"2023-10-26T06:45:04.912736+0000","last_unstale":"2023-10-26T06:46:02.228254+0000","last_undegraded":"2023-10-26T06:46:02.228254+0000","last_fullsized":"2023-10-26T06:46:02.228254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:59.690355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.757581+0000","last_change":"2023-10-26T06:46:03.233858+0000","last_active":"2023-10-26T06:46:31.757581+0000","last_peered":"2023-10-26T06:46:31.757581+0000","last_clean":"2023-10-26T06:46:31.757581+0000","last_became_active":"2023-10-26T06:46:03.233429+0000","last_became_peered":"2023-10-26T06:46:03.233429+0000","last_unstale":"2023-10-26T06:46:31.757581+0000","last_undegraded":"2023-10-26T06:46:31.757581+0000","last_fullsized":"2023-10-26T06:46:31.757581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:41:39.237862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.237541+0000","last_change":"2023-10-26T06:46:03.237541+0000","last_active":"2023-10-26T06:46:03.237541+0000","last_peered":"2023-10-26T06:46:03.237541+0000","last_clean":"2023-10-26T06:46:03.237541+0000","last_became_active":"2023-10-26T06:46:03.237019+0000","last_became_peered":"2023-10-26T06:46:03.237019+0000","last_unstale":"2023-10-26T06:46:03.237541+0000","last_undegraded":"2023-10-26T06:46:03.237541+0000","last_fullsized":"2023-10-26T06:46:03.237541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:59:15.383799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775563+0000","last_change":"2023-10-26T06:45:05.904832+0000","last_active":"2023-10-26T06:46:31.775563+0000","last_peered":"2023-10-26T06:46:31.775563+0000","last_clean":"2023-10-26T06:46:31.775563+0000","last_became_active":"2023-10-26T06:45:05.904527+0000","last_became_peered":"2023-10-26T06:45:05.904527+0000","last_unstale":"2023-10-26T06:46:31.775563+0000","last_undegraded":"2023-10-26T06:46:31.775563+0000","last_fullsized":"2023-10-26T06:46:31.775563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:30.546198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473828+0000","last_change":"2023-10-26T06:45:04.922832+0000","last_active":"2023-10-26T06:46:02.473828+0000","last_peered":"2023-10-26T06:46:02.473828+0000","last_clean":"2023-10-26T06:46:02.473828+0000","last_became_active":"2023-10-26T06:45:04.922349+0000","last_became_peered":"2023-10-26T06:45:04.922349+0000","last_unstale":"2023-10-26T06:46:02.473828+0000","last_undegraded":"2023-10-26T06:46:02.473828+0000","last_fullsized":"2023-10-26T06:46:02.473828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:37.839093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.770899+0000","last_change":"2023-10-26T06:45:05.909946+0000","last_active":"2023-10-26T06:46:31.770899+0000","last_peered":"2023-10-26T06:46:31.770899+0000","last_clean":"2023-10-26T06:46:31.770899+0000","last_became_active":"2023-10-26T06:45:05.909650+0000","last_became_peered":"2023-10-26T06:45:05.909650+0000","last_unstale":"2023-10-26T06:46:31.770899+0000","last_undegraded":"2023-10-26T06:46:31.770899+0000","last_fullsized":"2023-10-26T06:46:31.770899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:41.813928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763208+0000","last_change":"2023-10-26T06:45:05.903400+0000","last_active":"2023-10-26T06:46:31.763208+0000","last_peered":"2023-10-26T06:46:31.763208+0000","last_clean":"2023-10-26T06:46:31.763208+0000","last_became_active":"2023-10-26T06:45:05.903101+0000","last_became_peered":"2023-10-26T06:45:05.903101+0000","last_unstale":"2023-10-26T06:46:31.763208+0000","last_undegraded":"2023-10-26T06:46:31.763208+0000","last_fullsized":"2023-10-26T06:46:31.763208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227030+0000","last_change":"2023-10-26T06:45:04.923960+0000","last_active":"2023-10-26T06:46:02.227030+0000","last_peered":"2023-10-26T06:46:02.227030+0000","last_clean":"2023-10-26T06:46:02.227030+0000","last_became_active":"2023-10-26T06:45:04.923613+0000","last_became_peered":"2023-10-26T06:45:04.923613+0000","last_unstale":"2023-10-26T06:46:02.227030+0000","last_undegraded":"2023-10-26T06:46:02.227030+0000","last_fullsized":"2023-10-26T06:46:02.227030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:53.342463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778716+0000","last_change":"2023-10-26T06:45:05.903966+0000","last_active":"2023-10-26T06:46:31.778716+0000","last_peered":"2023-10-26T06:46:31.778716+0000","last_clean":"2023-10-26T06:46:31.778716+0000","last_became_active":"2023-10-26T06:45:05.903583+0000","last_became_peered":"2023-10-26T06:45:05.903583+0000","last_unstale":"2023-10-26T06:46:31.778716+0000","last_undegraded":"2023-10-26T06:46:31.778716+0000","last_fullsized":"2023-10-26T06:46:31.778716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:18.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226986+0000","last_change":"2023-10-26T06:45:04.907838+0000","last_active":"2023-10-26T06:46:02.226986+0000","last_peered":"2023-10-26T06:46:02.226986+0000","last_clean":"2023-10-26T06:46:02.226986+0000","last_became_active":"2023-10-26T06:45:04.907508+0000","last_became_peered":"2023-10-26T06:45:04.907508+0000","last_unstale":"2023-10-26T06:46:02.226986+0000","last_undegraded":"2023-10-26T06:46:02.226986+0000","last_fullsized":"2023-10-26T06:46:02.226986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:34.852243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":321,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473730+0000","last_change":"2023-10-26T06:43:59.765466+0000","last_active":"2023-10-26T06:46:02.473730+0000","last_peered":"2023-10-26T06:46:02.473730+0000","last_clean":"2023-10-26T06:46:02.473730+0000","last_became_active":"2023-10-26T06:43:59.765005+0000","last_became_peered":"2023-10-26T06:43:59.765005+0000","last_unstale":"2023-10-26T06:46:02.473730+0000","last_undegraded":"2023-10-26T06:46:02.473730+0000","last_fullsized":"2023-10-26T06:46:02.473730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_clean_scrub_stamp":"2023-10-26T06:40:50.556448+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:18.688941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774103+0000","last_change":"2023-10-26T06:45:05.906909+0000","last_active":"2023-10-26T06:46:31.774103+0000","last_peered":"2023-10-26T06:46:31.774103+0000","last_clean":"2023-10-26T06:46:31.774103+0000","last_became_active":"2023-10-26T06:45:05.906531+0000","last_became_peered":"2023-10-26T06:45:05.906531+0000","last_unstale":"2023-10-26T06:46:31.774103+0000","last_undegraded":"2023-10-26T06:46:31.774103+0000","last_fullsized":"2023-10-26T06:46:31.774103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473805+0000","last_change":"2023-10-26T06:45:04.911635+0000","last_active":"2023-10-26T06:46:02.473805+0000","last_peered":"2023-10-26T06:46:02.473805+0000","last_clean":"2023-10-26T06:46:02.473805+0000","last_became_active":"2023-10-26T06:45:04.911289+0000","last_became_peered":"2023-10-26T06:45:04.911289+0000","last_unstale":"2023-10-26T06:46:02.473805+0000","last_undegraded":"2023-10-26T06:46:02.473805+0000","last_fullsized":"2023-10-26T06:46:02.473805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.115211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227809+0000","last_change":"2023-10-26T06:45:04.919853+0000","last_active":"2023-10-26T06:46:02.227809+0000","last_peered":"2023-10-26T06:46:02.227809+0000","last_clean":"2023-10-26T06:46:02.227809+0000","last_became_active":"2023-10-26T06:45:04.919488+0000","last_became_peered":"2023-10-26T06:45:04.919488+0000","last_unstale":"2023-10-26T06:46:02.227809+0000","last_undegraded":"2023-10-26T06:46:02.227809+0000","last_fullsized":"2023-10-26T06:46:02.227809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:06.555851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.776607+0000","last_change":"2023-10-26T06:45:05.903975+0000","last_active":"2023-10-26T06:46:31.776607+0000","last_peered":"2023-10-26T06:46:31.776607+0000","last_clean":"2023-10-26T06:46:31.776607+0000","last_became_active":"2023-10-26T06:45:05.903539+0000","last_became_peered":"2023-10-26T06:45:05.903539+0000","last_unstale":"2023-10-26T06:46:31.776607+0000","last_undegraded":"2023-10-26T06:46:31.776607+0000","last_fullsized":"2023-10-26T06:46:31.776607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:30.648947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:33.946402+0000","last_change":"2023-10-26T06:45:04.919815+0000","last_active":"2023-10-26T06:46:33.946402+0000","last_peered":"2023-10-26T06:46:33.946402+0000","last_clean":"2023-10-26T06:46:33.946402+0000","last_became_active":"2023-10-26T06:45:04.918141+0000","last_became_peered":"2023-10-26T06:45:04.918141+0000","last_unstale":"2023-10-26T06:46:33.946402+0000","last_undegraded":"2023-10-26T06:46:33.946402+0000","last_fullsized":"2023-10-26T06:46:33.946402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:53.024534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774509+0000","last_change":"2023-10-26T06:45:05.898180+0000","last_active":"2023-10-26T06:46:31.774509+0000","last_peered":"2023-10-26T06:46:31.774509+0000","last_clean":"2023-10-26T06:46:31.774509+0000","last_became_active":"2023-10-26T06:45:05.896937+0000","last_became_peered":"2023-10-26T06:45:05.896937+0000","last_unstale":"2023-10-26T06:46:31.774509+0000","last_undegraded":"2023-10-26T06:46:31.774509+0000","last_fullsized":"2023-10-26T06:46:31.774509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:48.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227006+0000","last_change":"2023-10-26T06:45:04.922456+0000","last_active":"2023-10-26T06:46:02.227006+0000","last_peered":"2023-10-26T06:46:02.227006+0000","last_clean":"2023-10-26T06:46:02.227006+0000","last_became_active":"2023-10-26T06:45:04.922112+0000","last_became_peered":"2023-10-26T06:45:04.922112+0000","last_unstale":"2023-10-26T06:46:02.227006+0000","last_undegraded":"2023-10-26T06:46:02.227006+0000","last_fullsized":"2023-10-26T06:46:02.227006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:44.055569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781350+0000","last_change":"2023-10-26T06:45:05.895440+0000","last_active":"2023-10-26T06:46:31.781350+0000","last_peered":"2023-10-26T06:46:31.781350+0000","last_clean":"2023-10-26T06:46:31.781350+0000","last_became_active":"2023-10-26T06:45:05.892605+0000","last_became_peered":"2023-10-26T06:45:05.892605+0000","last_unstale":"2023-10-26T06:46:31.781350+0000","last_undegraded":"2023-10-26T06:46:31.781350+0000","last_fullsized":"2023-10-26T06:46:31.781350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:09:27.392772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.225650+0000","last_change":"2023-10-26T06:45:04.916042+0000","last_active":"2023-10-26T06:46:02.225650+0000","last_peered":"2023-10-26T06:46:02.225650+0000","last_clean":"2023-10-26T06:46:02.225650+0000","last_became_active":"2023-10-26T06:45:04.915649+0000","last_became_peered":"2023-10-26T06:45:04.915649+0000","last_unstale":"2023-10-26T06:46:02.225650+0000","last_undegraded":"2023-10-26T06:46:02.225650+0000","last_fullsized":"2023-10-26T06:46:02.225650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:30.060074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.735910+0000","last_change":"2023-10-26T06:45:05.908179+0000","last_active":"2023-10-26T06:46:31.735910+0000","last_peered":"2023-10-26T06:46:31.735910+0000","last_clean":"2023-10-26T06:46:31.735910+0000","last_became_active":"2023-10-26T06:45:05.907857+0000","last_became_peered":"2023-10-26T06:45:05.907857+0000","last_unstale":"2023-10-26T06:46:31.735910+0000","last_undegraded":"2023-10-26T06:46:31.735910+0000","last_fullsized":"2023-10-26T06:46:31.735910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:20.017194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230341+0000","last_change":"2023-10-26T06:45:04.921016+0000","last_active":"2023-10-26T06:46:02.230341+0000","last_peered":"2023-10-26T06:46:02.230341+0000","last_clean":"2023-10-26T06:46:02.230341+0000","last_became_active":"2023-10-26T06:45:04.920741+0000","last_became_peered":"2023-10-26T06:45:04.920741+0000","last_unstale":"2023-10-26T06:46:02.230341+0000","last_undegraded":"2023-10-26T06:46:02.230341+0000","last_fullsized":"2023-10-26T06:46:02.230341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:05:06.218205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778934+0000","last_change":"2023-10-26T06:45:05.893562+0000","last_active":"2023-10-26T06:46:31.778934+0000","last_peered":"2023-10-26T06:46:31.778934+0000","last_clean":"2023-10-26T06:46:31.778934+0000","last_became_active":"2023-10-26T06:45:05.893111+0000","last_became_peered":"2023-10-26T06:45:05.893111+0000","last_unstale":"2023-10-26T06:46:31.778934+0000","last_undegraded":"2023-10-26T06:46:31.778934+0000","last_fullsized":"2023-10-26T06:46:31.778934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.305664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226475+0000","last_change":"2023-10-26T06:45:04.914936+0000","last_active":"2023-10-26T06:46:02.226475+0000","last_peered":"2023-10-26T06:46:02.226475+0000","last_clean":"2023-10-26T06:46:02.226475+0000","last_became_active":"2023-10-26T06:45:04.914617+0000","last_became_peered":"2023-10-26T06:45:04.914617+0000","last_unstale":"2023-10-26T06:46:02.226475+0000","last_undegraded":"2023-10-26T06:46:02.226475+0000","last_fullsized":"2023-10-26T06:46:02.226475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:15.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778284+0000","last_change":"2023-10-26T06:45:05.909101+0000","last_active":"2023-10-26T06:46:31.778284+0000","last_peered":"2023-10-26T06:46:31.778284+0000","last_clean":"2023-10-26T06:46:31.778284+0000","last_became_active":"2023-10-26T06:45:05.908800+0000","last_became_peered":"2023-10-26T06:45:05.908800+0000","last_unstale":"2023-10-26T06:46:31.778284+0000","last_undegraded":"2023-10-26T06:46:31.778284+0000","last_fullsized":"2023-10-26T06:46:31.778284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:37:18.605102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227537+0000","last_change":"2023-10-26T06:45:04.911714+0000","last_active":"2023-10-26T06:46:02.227537+0000","last_peered":"2023-10-26T06:46:02.227537+0000","last_clean":"2023-10-26T06:46:02.227537+0000","last_became_active":"2023-10-26T06:45:04.911343+0000","last_became_peered":"2023-10-26T06:45:04.911343+0000","last_unstale":"2023-10-26T06:46:02.227537+0000","last_undegraded":"2023-10-26T06:46:02.227537+0000","last_fullsized":"2023-10-26T06:46:02.227537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:30:35.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773867+0000","last_change":"2023-10-26T06:45:05.906804+0000","last_active":"2023-10-26T06:46:31.773867+0000","last_peered":"2023-10-26T06:46:31.773867+0000","last_clean":"2023-10-26T06:46:31.773867+0000","last_became_active":"2023-10-26T06:45:05.906522+0000","last_became_peered":"2023-10-26T06:45:05.906522+0000","last_unstale":"2023-10-26T06:46:31.773867+0000","last_undegraded":"2023-10-26T06:46:31.773867+0000","last_fullsized":"2023-10-26T06:46:31.773867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:32.707204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226957+0000","last_change":"2023-10-26T06:45:04.906345+0000","last_active":"2023-10-26T06:46:02.226957+0000","last_peered":"2023-10-26T06:46:02.226957+0000","last_clean":"2023-10-26T06:46:02.226957+0000","last_became_active":"2023-10-26T06:45:04.906034+0000","last_became_peered":"2023-10-26T06:45:04.906034+0000","last_unstale":"2023-10-26T06:46:02.226957+0000","last_undegraded":"2023-10-26T06:46:02.226957+0000","last_fullsized":"2023-10-26T06:46:02.226957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:53.690788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779248+0000","last_change":"2023-10-26T06:45:05.906368+0000","last_active":"2023-10-26T06:46:31.779248+0000","last_peered":"2023-10-26T06:46:31.779248+0000","last_clean":"2023-10-26T06:46:31.779248+0000","last_became_active":"2023-10-26T06:45:05.905471+0000","last_became_peered":"2023-10-26T06:45:05.905471+0000","last_unstale":"2023-10-26T06:46:31.779248+0000","last_undegraded":"2023-10-26T06:46:31.779248+0000","last_fullsized":"2023-10-26T06:46:31.779248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:45.493704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230243+0000","last_change":"2023-10-26T06:45:04.917530+0000","last_active":"2023-10-26T06:46:02.230243+0000","last_peered":"2023-10-26T06:46:02.230243+0000","last_clean":"2023-10-26T06:46:02.230243+0000","last_became_active":"2023-10-26T06:45:04.917193+0000","last_became_peered":"2023-10-26T06:45:04.917193+0000","last_unstale":"2023-10-26T06:46:02.230243+0000","last_undegraded":"2023-10-26T06:46:02.230243+0000","last_fullsized":"2023-10-26T06:46:02.230243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.965048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.765239+0000","last_change":"2023-10-26T06:45:05.899429+0000","last_active":"2023-10-26T06:46:31.765239+0000","last_peered":"2023-10-26T06:46:31.765239+0000","last_clean":"2023-10-26T06:46:31.765239+0000","last_became_active":"2023-10-26T06:45:05.899081+0000","last_became_peered":"2023-10-26T06:45:05.899081+0000","last_unstale":"2023-10-26T06:46:31.765239+0000","last_undegraded":"2023-10-26T06:46:31.765239+0000","last_fullsized":"2023-10-26T06:46:31.765239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:41.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227783+0000","last_change":"2023-10-26T06:45:04.920295+0000","last_active":"2023-10-26T06:46:02.227783+0000","last_peered":"2023-10-26T06:46:02.227783+0000","last_clean":"2023-10-26T06:46:02.227783+0000","last_became_active":"2023-10-26T06:45:04.919920+0000","last_became_peered":"2023-10-26T06:45:04.919920+0000","last_unstale":"2023-10-26T06:46:02.227783+0000","last_undegraded":"2023-10-26T06:46:02.227783+0000","last_fullsized":"2023-10-26T06:46:02.227783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:59.260841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764977+0000","last_change":"2023-10-26T06:45:05.899610+0000","last_active":"2023-10-26T06:46:31.764977+0000","last_peered":"2023-10-26T06:46:31.764977+0000","last_clean":"2023-10-26T06:46:31.764977+0000","last_became_active":"2023-10-26T06:45:05.899283+0000","last_became_peered":"2023-10-26T06:45:05.899283+0000","last_unstale":"2023-10-26T06:46:31.764977+0000","last_undegraded":"2023-10-26T06:46:31.764977+0000","last_fullsized":"2023-10-26T06:46:31.764977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:48.730703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.759894+0000","last_change":"2023-10-26T06:45:05.892012+0000","last_active":"2023-10-26T06:46:31.759894+0000","last_peered":"2023-10-26T06:46:31.759894+0000","last_clean":"2023-10-26T06:46:31.759894+0000","last_became_active":"2023-10-26T06:45:05.891744+0000","last_became_peered":"2023-10-26T06:45:05.891744+0000","last_unstale":"2023-10-26T06:46:31.759894+0000","last_undegraded":"2023-10-26T06:46:31.759894+0000","last_fullsized":"2023-10-26T06:46:31.759894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:29.599184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763039+0000","last_change":"2023-10-26T06:45:05.904348+0000","last_active":"2023-10-26T06:46:31.763039+0000","last_peered":"2023-10-26T06:46:31.763039+0000","last_clean":"2023-10-26T06:46:31.763039+0000","last_became_active":"2023-10-26T06:45:05.903949+0000","last_became_peered":"2023-10-26T06:45:05.903949+0000","last_unstale":"2023-10-26T06:46:31.763039+0000","last_undegraded":"2023-10-26T06:46:31.763039+0000","last_fullsized":"2023-10-26T06:46:31.763039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:29.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.771138+0000","last_change":"2023-10-26T06:46:03.237595+0000","last_active":"2023-10-26T06:46:31.771138+0000","last_peered":"2023-10-26T06:46:31.771138+0000","last_clean":"2023-10-26T06:46:31.771138+0000","last_became_active":"2023-10-26T06:46:03.237136+0000","last_became_peered":"2023-10-26T06:46:03.237136+0000","last_unstale":"2023-10-26T06:46:31.771138+0000","last_undegraded":"2023-10-26T06:46:31.771138+0000","last_fullsized":"2023-10-26T06:46:31.771138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:03.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780714+0000","last_change":"2023-10-26T06:45:05.901225+0000","last_active":"2023-10-26T06:46:31.780714+0000","last_peered":"2023-10-26T06:46:31.780714+0000","last_clean":"2023-10-26T06:46:31.780714+0000","last_became_active":"2023-10-26T06:45:05.900743+0000","last_became_peered":"2023-10-26T06:45:05.900743+0000","last_unstale":"2023-10-26T06:46:31.780714+0000","last_undegraded":"2023-10-26T06:46:31.780714+0000","last_fullsized":"2023-10-26T06:46:31.780714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:02.359654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781128+0000","last_change":"2023-10-26T06:45:05.906805+0000","last_active":"2023-10-26T06:46:31.781128+0000","last_peered":"2023-10-26T06:46:31.781128+0000","last_clean":"2023-10-26T06:46:31.781128+0000","last_became_active":"2023-10-26T06:45:05.906464+0000","last_became_peered":"2023-10-26T06:45:05.906464+0000","last_unstale":"2023-10-26T06:46:31.781128+0000","last_undegraded":"2023-10-26T06:46:31.781128+0000","last_fullsized":"2023-10-26T06:46:31.781128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:24.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.759205+0000","last_change":"2023-10-26T06:45:05.898568+0000","last_active":"2023-10-26T06:46:31.759205+0000","last_peered":"2023-10-26T06:46:31.759205+0000","last_clean":"2023-10-26T06:46:31.759205+0000","last_became_active":"2023-10-26T06:45:05.897886+0000","last_became_peered":"2023-10-26T06:45:05.897886+0000","last_unstale":"2023-10-26T06:46:31.759205+0000","last_undegraded":"2023-10-26T06:46:31.759205+0000","last_fullsized":"2023-10-26T06:46:31.759205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:14.998233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779508+0000","last_change":"2023-10-26T06:45:05.899326+0000","last_active":"2023-10-26T06:46:31.779508+0000","last_peered":"2023-10-26T06:46:31.779508+0000","last_clean":"2023-10-26T06:46:31.779508+0000","last_became_active":"2023-10-26T06:45:05.899040+0000","last_became_peered":"2023-10-26T06:45:05.899040+0000","last_unstale":"2023-10-26T06:46:31.779508+0000","last_undegraded":"2023-10-26T06:46:31.779508+0000","last_fullsized":"2023-10-26T06:46:31.779508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:55:35.666122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773908+0000","last_change":"2023-10-26T06:45:05.904155+0000","last_active":"2023-10-26T06:46:31.773908+0000","last_peered":"2023-10-26T06:46:31.773908+0000","last_clean":"2023-10-26T06:46:31.773908+0000","last_became_active":"2023-10-26T06:45:05.903734+0000","last_became_peered":"2023-10-26T06:45:05.903734+0000","last_unstale":"2023-10-26T06:46:31.773908+0000","last_undegraded":"2023-10-26T06:46:31.773908+0000","last_fullsized":"2023-10-26T06:46:31.773908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:43.252789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781034+0000","last_change":"2023-10-26T06:45:05.897174+0000","last_active":"2023-10-26T06:46:31.781034+0000","last_peered":"2023-10-26T06:46:31.781034+0000","last_clean":"2023-10-26T06:46:31.781034+0000","last_became_active":"2023-10-26T06:45:05.896893+0000","last_became_peered":"2023-10-26T06:45:05.896893+0000","last_unstale":"2023-10-26T06:46:31.781034+0000","last_undegraded":"2023-10-26T06:46:31.781034+0000","last_fullsized":"2023-10-26T06:46:31.781034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:14.291682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773128+0000","last_change":"2023-10-26T06:45:05.905987+0000","last_active":"2023-10-26T06:46:31.773128+0000","last_peered":"2023-10-26T06:46:31.773128+0000","last_clean":"2023-10-26T06:46:31.773128+0000","last_became_active":"2023-10-26T06:45:05.904933+0000","last_became_peered":"2023-10-26T06:45:05.904933+0000","last_unstale":"2023-10-26T06:46:31.773128+0000","last_undegraded":"2023-10-26T06:46:31.773128+0000","last_fullsized":"2023-10-26T06:46:31.773128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:19.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782354+0000","last_change":"2023-10-26T06:45:05.912781+0000","last_active":"2023-10-26T06:46:31.782354+0000","last_peered":"2023-10-26T06:46:31.782354+0000","last_clean":"2023-10-26T06:46:31.782354+0000","last_became_active":"2023-10-26T06:45:05.912415+0000","last_became_peered":"2023-10-26T06:45:05.912415+0000","last_unstale":"2023-10-26T06:46:31.782354+0000","last_undegraded":"2023-10-26T06:46:31.782354+0000","last_fullsized":"2023-10-26T06:46:31.782354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:25.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779371+0000","last_change":"2023-10-26T06:45:05.909106+0000","last_active":"2023-10-26T06:46:31.779371+0000","last_peered":"2023-10-26T06:46:31.779371+0000","last_clean":"2023-10-26T06:46:31.779371+0000","last_became_active":"2023-10-26T06:45:05.908751+0000","last_became_peered":"2023-10-26T06:45:05.908751+0000","last_unstale":"2023-10-26T06:46:31.779371+0000","last_undegraded":"2023-10-26T06:46:31.779371+0000","last_fullsized":"2023-10-26T06:46:31.779371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:17.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.761209+0000","last_change":"2023-10-26T06:45:05.898262+0000","last_active":"2023-10-26T06:46:31.761209+0000","last_peered":"2023-10-26T06:46:31.761209+0000","last_clean":"2023-10-26T06:46:31.761209+0000","last_became_active":"2023-10-26T06:45:05.897947+0000","last_became_peered":"2023-10-26T06:45:05.897947+0000","last_unstale":"2023-10-26T06:46:31.761209+0000","last_undegraded":"2023-10-26T06:46:31.761209+0000","last_fullsized":"2023-10-26T06:46:31.761209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:39:45.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65317,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":214,"num_read_kb":172,"num_write":77,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":195951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":462460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.624},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48599999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.80700000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.432}]},{"osd":2,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.372},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88100000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":1.52,"15min":1.52},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.012,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95199999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93700000000000006,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47599999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1040000000000001,"5min":1.47,"15min":1.47},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0549999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.74299999999999999}]},{"osd":10,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96199999999999997,"5min":1.327,"15min":1.327},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.036,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.57799999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1117664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44500000000000001,"15min":0.44400000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.498},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82499999999999996,"5min":1.004,"15min":1.004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79800000000000004,"5min":0.874,"15min":0.874},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.627}]},{"osd":5,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41699999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.752},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95599999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.046,"5min":1.133,"15min":1.133},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Oct 26 06:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.73399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0089999999999999,"5min":1.04,"15min":1.04},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.60599999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.097,"15min":1.097},"last":0.748}]}]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":463042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.51400000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":0.69499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.1099999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.6319999999999997,"5min":4.6319999999999997,"15min":4.6319999999999997},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.84499999999999997,"15min":0.86499999999999999},"min":{"1min":0.55200000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93100000000000005,"5min":1.173,"15min":1.173},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.87},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97199999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.877},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.496},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.877,"15min":0.90700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.64600000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93700000000000006,"15min":0.96499999999999997},"min":{"1min":0.59799999999999998,"5min":0.503,"15min":0.503},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.94099999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.762,"5min":1.885,"15min":1.885},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.70099999999999996,"15min":0.77100000000000002},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80000000000000004,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.505}]},{"osd":1,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.628,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.50700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44600000000000001},"min":{"1min":0.34200000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.48799999999999999,"15min":0.48499999999999999},"min":{"1min":0.40000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.68500000000000005,"15min":0.65900000000000003},"min":{"1min":0.49199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85699999999999998,"5min":0.88,"15min":0.88},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0580000000000001,"5min":1.095,"15min":1.095},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.81799999999999995,"15min":0.88200000000000001},"min":{"1min":0.56499999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.038,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.84699999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.63800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.74299999999999999,"15min":0.71899999999999997},"min":{"1min":0.68400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.022,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72899999999999998,"15min":0.69699999999999995},"min":{"1min":0.63400000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Oct 26 06:46:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.86199999999999999}]},{"osd":11,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.94699999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1053533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41699999999999998,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60299999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78500000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.748,"15min":0.748},"last":0.503},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80100000000000005}]},{"osd":8,"last update":"Thu Oct 26 06:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.877},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70799999999999996,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83999999999999997}]},{"osd":9,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.753},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.88200000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.82099999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.747,"15min":0.76100000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.90800000000000003,"5min":1.048,"15min":1.048},"last":0.77500000000000002}]},{"osd":11,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.94699999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.44400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70699999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79100000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.68600000000000005,"15min":0.64300000000000002},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68600000000000005,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":5.5549999999999997,"5min":5.5549999999999997,"15min":5.5549999999999997},"last":0.91500000000000004}]},{"osd":3,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71499999999999997,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.70799999999999996,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.84199999999999997,"5min":1.208,"15min":1.208},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94999999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.385,"15min":1.385},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.749,"15min":0.747},"min":{"1min":0.57499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90600000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.82399999999999995,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":5.532,"5min":5.532,"15min":5.532},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":0.79900000000000004,"15min":0.73299999999999998},"min":{"1min":0.56499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":5.5810000000000004,"5min":5.5810000000000004,"15min":5.5810000000000004},"last":0.876}]},{"osd":8,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.59299999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95299999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Thu Oct 26 06:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92200000000000004,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.82099999999999995,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84899999999999998,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96799999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.93400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":521600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.54100000000000004},"min":{"1min":0.47299999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.92300000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44700000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0209999999999999,"5min":1.256,"15min":1.256},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"min":{"1min":0.498,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.627},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.56899999999999995,"15min":0.52500000000000002},"min":{"1min":0.52600000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.49299999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61099999999999999,"15min":0.55700000000000005},"min":{"1min":0.47499999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.45500000000000002,"15min":0.442},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.373},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.42799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39100000000000001}]},{"osd":5,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.58399999999999996},"min":{"1min":0.496,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.56299999999999994,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.53700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.52400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96199999999999997,"5min":1.03,"15min":1.03},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.497,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.72999999999999998,"15min":0.70399999999999996},"min":{"1min":0.58499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.996,"5min":1.298,"15min":1.298},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.64800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.70199999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.751},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.52300000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.82799999999999996}]}]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 06:46:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65900000000000003,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.61499999999999999},"min":{"1min":0.47399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5820000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.68200000000000005,"15min":0.626},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Oct 26 06:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4450000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.68400000000000005,"15min":0.63100000000000001},"min":{"1min":0.53000000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.749,"15min":0.69399999999999995},"min":{"1min":0.53300000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":0.89900000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:46:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68700000000000006,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.4300000000000002,"5min":3.4300000000000002,"15min":3.4300000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.73599999999999999,"15min":0.68200000000000005},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0099999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Oct 26 06:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.61599999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.054,"5min":0.93700000000000006,"15min":0.92900000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.92300000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.437,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99199999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.626},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.87,"15min":0.89000000000000001},"min":{"1min":0.47299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.544,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.60599999999999998}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.64000000000000001,"5min":0.625,"15min":0.625},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.089,"5min":0.92700000000000005,"15min":0.90000000000000002},"min":{"1min":0.64500000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.85999999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":526344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58099999999999996,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.56499999999999995,"15min":0.52300000000000002},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Oct 26 06:46:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.625},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89700000000000002,"5min":2.887,"15min":2.887},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.501,"15min":0.46899999999999997},"min":{"1min":0.40500000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.56399999999999995,"15min":0.51400000000000001},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.786,"5min":3.786,"15min":3.786},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.623},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79000000000000004,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93400000000000005,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":0.88}]},{"osd":7,"last update":"Thu Oct 26 06:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.78700000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.997,"15min":0.997},"last":1.3480000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89300000000000002,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.68100000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86499999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.70199999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78500000000000003,"15min":0.77100000000000002},"min":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.86199999999999999,"5min":3.661,"15min":3.661},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.58099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":3.431,"15min":3.431},"last":0.95099999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.746,"15min":0.72099999999999997},"min":{"1min":0.51700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4649999999999999,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.77100000000000002,"15min":0.748},"min":{"1min":0.626,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.7040000000000002,"5min":3.7040000000000002,"15min":3.7040000000000002},"last":0.80300000000000005}]},{"osd":11,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78100000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":462479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44400000000000001,"15min":0.40899999999999997},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48399999999999999,"15min":0.45200000000000001},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.722,"15min":1.722},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.60599999999999998},"min":{"1min":0.48799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.875,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94599999999999995,"5min":1.427,"15min":1.427},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85599999999999998,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Oct 26 06:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":1.667,"15min":1.667},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.304,"15min":1.304},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.55600000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96699999999999997,"5min":1.456,"15min":1.456},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75,"15min":0.70199999999999996},"min":{"1min":0.623,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.115,"5min":1.637,"15min":1.637},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90800000000000003,"5min":1.589,"15min":1.589},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64700000000000002,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.516,"15min":1.516},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Oct 26 06:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96999999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.69899999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.85299999999999998,"15min":0.86099999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.3200000000000001,"5min":1.395,"15min":1.395},"last":0.875},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.92400000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0780000000000001,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.85399999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.434}]},{"osd":1,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.35299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.38},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.57899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66400000000000003,"5min":1.222,"15min":1.222},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:46:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.48199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.81899999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86199999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.28,"15min":1.28},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84599999999999997,"5min":1.403,"15min":1.403},"last":0.57499999999999996}]},{"osd":8,"last update":"Thu Oct 26 06:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.63100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88200000000000001,"5min":0.998,"15min":0.998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.623,"5min":0.433,"15min":0.433},"max":{"1min":0.82099999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.72699999999999998,"15min":0.71299999999999997},"min":{"1min":0.56999999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.748},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.024,"5min":1.175,"15min":1.175},"last":0.77000000000000002}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94099999999999995,"5min":1.141,"15min":1.141},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91700000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Oct 26 06:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78900000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1053133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":0.871,"15min":0.871},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59699999999999998,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.85799999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.313,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73599999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.46200000000000002,"15min":0.439},"min":{"1min":0.37,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.496}]},{"osd":3,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.748,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.48599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75}]},{"osd":4,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63500000000000001,"15min":0.60999999999999999},"min":{"1min":0.54400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68100000000000005,"15min":0.64900000000000002},"min":{"1min":0.59699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70199999999999996,"15min":0.66400000000000003},"min":{"1min":0.63100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84399999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.56200000000000006,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.874}]},{"osd":8,"last update":"Thu Oct 26 06:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66500000000000004,"15min":0.63900000000000001},"min":{"1min":0.42099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64500000000000002}]},{"osd":9,"last update":"Thu Oct 26 06:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.66000000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.83999999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.66500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-26T06:46:48.416 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-26T06:46:48.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T06:46:48.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:48 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2398269996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:48.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:48 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2398269996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 -- 172.21.15.89:0/1958972851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581006a0 msgr2=0x7fb258100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/1958972851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581006a0 0x7fb258100b20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009990 tx=0x7fb24c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 -- 172.21.15.89:0/1958972851 shutdown_connections 2023-10-26T06:46:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/1958972851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb258101060 0x7fb25810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/1958972851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2581066b0 0x7fb258106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/1958972851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581006a0 0x7fb258100b20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 -- 172.21.15.89:0/1958972851 >> 172.21.15.89:0/1958972851 conn(0x7fb2580fc170 msgr2=0x7fb2580fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 -- 172.21.15.89:0/1958972851 shutdown_connections 2023-10-26T06:46:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.646+0000 7fb25c9ff700 1 -- 172.21.15.89:0/1958972851 wait complete. 2023-10-26T06:46:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 Processor -- start 2023-10-26T06:46:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 -- start start 2023-10-26T06:46:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2581006a0 0x7fb25819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb258101060 0x7fb25819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581066b0 0x7fb258198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258199180 con 0x7fb2581066b0 2023-10-26T06:46:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581992f0 con 0x7fb2581006a0 2023-10-26T06:46:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb25c9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581994b0 con 0x7fb258101060 2023-10-26T06:46:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb256d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581066b0 0x7fb258198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb256d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581066b0 0x7fb258198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54150/0 (socket says 172.21.15.89:54150) 2023-10-26T06:46:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.647+0000 7fb256d9d700 1 -- 172.21.15.89:0/2017602281 learned_addr learned my addr 172.21.15.89:0/2017602281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb25659c700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2581006a0 0x7fb25819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb255d9b700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb258101060 0x7fb25819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb256d9d700 1 -- 172.21.15.89:0/2017602281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb258101060 msgr2=0x7fb25819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb256d9d700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb258101060 0x7fb25819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb256d9d700 1 -- 172.21.15.89:0/2017602281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2581006a0 msgr2=0x7fb25819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb256d9d700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2581006a0 0x7fb25819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb256d9d700 1 -- 172.21.15.89:0/2017602281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0095f0 con 0x7fb2581066b0 2023-10-26T06:46:48.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb256d9d700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581066b0 0x7fb258198b20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb250007ee0 tx=0x7fb25000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:48.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb2477fe700 1 -- 172.21.15.89:0/2017602281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250017070 con 0x7fb2581066b0 2023-10-26T06:46:48.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb2477fe700 1 -- 172.21.15.89:0/2017602281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25000f040 con 0x7fb2581066b0 2023-10-26T06:46:48.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb258199790 con 0x7fb2581066b0 2023-10-26T06:46:48.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb2477fe700 1 -- 172.21.15.89:0/2017602281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250005580 con 0x7fb2581066b0 2023-10-26T06:46:48.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.648+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2581a5a30 con 0x7fb2581066b0 2023-10-26T06:46:48.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.649+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb258101de0 con 0x7fb2581066b0 2023-10-26T06:46:48.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.651+0000 7fb2477fe700 1 -- 172.21.15.89:0/2017602281 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb25000db20 con 0x7fb2581066b0 2023-10-26T06:46:48.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.652+0000 7fb2477fe700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb2400a1550 0x7fb2400a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:48.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.653+0000 7fb2477fe700 1 -- 172.21.15.89:0/2017602281 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb2500cc550 con 0x7fb2581066b0 2023-10-26T06:46:48.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.653+0000 7fb25659c700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb2400a1550 0x7fb2400a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:48.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.653+0000 7fb25659c700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb2400a1550 0x7fb2400a3a10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb248009be0 tx=0x7fb248009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.656+0000 7fb2477fe700 1 -- 172.21.15.89:0/2017602281 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb250095a70 con 0x7fb2581066b0 2023-10-26T06:46:48.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.853+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb258110620 con 0x7fb2400a1550 2023-10-26T06:46:48.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.863+0000 7fb2477fe700 1 -- 172.21.15.89:0/2017602281 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344469 (secure 0 0 0) 0x7fb258110620 con 0x7fb2400a1550 2023-10-26T06:46:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:48.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb2400a1550 msgr2=0x7fb2400a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb2400a1550 0x7fb2400a3a10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb248009be0 tx=0x7fb248009260 comp rx=0 tx=0).stop 2023-10-26T06:46:48.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581066b0 msgr2=0x7fb258198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:48.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581066b0 0x7fb258198b20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb250007ee0 tx=0x7fb25000d3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 shutdown_connections 2023-10-26T06:46:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb2400a1550 0x7fb2400a3a10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2581066b0 0x7fb258198b20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2581006a0 0x7fb25819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 --2- 172.21.15.89:0/2017602281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb258101060 0x7fb25819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 >> 172.21.15.89:0/2017602281 conn(0x7fb2580fc170 msgr2=0x7fb25810bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:48.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 shutdown_connections 2023-10-26T06:46:48.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:48.868+0000 7fb25c9ff700 1 -- 172.21.15.89:0/2017602281 wait complete. 2023-10-26T06:46:48.877 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2023-10-26T06:46:48.889 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-26T06:46:47.738531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655931,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":350,"num_read_kb":291,"num_write":260,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":706,"ondisk_log_size":706,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337208,"kb_used_data":13016,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604616,"statfs":{"total":1151940427776,"available":1151595126784,"internally_reserved":0,"allocated":13328384,"data_stored":7510603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":22,"num_write":4,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002855"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782197+0000","last_change":"2023-10-26T06:45:05.903170+0000","last_active":"2023-10-26T06:46:31.782197+0000","last_peered":"2023-10-26T06:46:31.782197+0000","last_clean":"2023-10-26T06:46:31.782197+0000","last_became_active":"2023-10-26T06:45:05.902574+0000","last_became_peered":"2023-10-26T06:45:05.902574+0000","last_unstale":"2023-10-26T06:46:31.782197+0000","last_undegraded":"2023-10-26T06:46:31.782197+0000","last_fullsized":"2023-10-26T06:46:31.782197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:33.065828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778600+0000","last_change":"2023-10-26T06:45:05.911249+0000","last_active":"2023-10-26T06:46:31.778600+0000","last_peered":"2023-10-26T06:46:31.778600+0000","last_clean":"2023-10-26T06:46:31.778600+0000","last_became_active":"2023-10-26T06:45:05.910859+0000","last_became_peered":"2023-10-26T06:45:05.910859+0000","last_unstale":"2023-10-26T06:46:31.778600+0000","last_undegraded":"2023-10-26T06:46:31.778600+0000","last_fullsized":"2023-10-26T06:46:31.778600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:07:13.362460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780029+0000","last_change":"2023-10-26T06:45:05.909219+0000","last_active":"2023-10-26T06:46:31.780029+0000","last_peered":"2023-10-26T06:46:31.780029+0000","last_clean":"2023-10-26T06:46:31.780029+0000","last_became_active":"2023-10-26T06:45:05.906721+0000","last_became_peered":"2023-10-26T06:45:05.906721+0000","last_unstale":"2023-10-26T06:46:31.780029+0000","last_undegraded":"2023-10-26T06:46:31.780029+0000","last_fullsized":"2023-10-26T06:46:31.780029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:41.461357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.762023+0000","last_change":"2023-10-26T06:45:05.897514+0000","last_active":"2023-10-26T06:46:31.762023+0000","last_peered":"2023-10-26T06:46:31.762023+0000","last_clean":"2023-10-26T06:46:31.762023+0000","last_became_active":"2023-10-26T06:45:05.897201+0000","last_became_peered":"2023-10-26T06:45:05.897201+0000","last_unstale":"2023-10-26T06:46:31.762023+0000","last_undegraded":"2023-10-26T06:46:31.762023+0000","last_fullsized":"2023-10-26T06:46:31.762023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:39.508654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780489+0000","last_change":"2023-10-26T06:45:05.900475+0000","last_active":"2023-10-26T06:46:31.780489+0000","last_peered":"2023-10-26T06:46:31.780489+0000","last_clean":"2023-10-26T06:46:31.780489+0000","last_became_active":"2023-10-26T06:45:05.900313+0000","last_became_peered":"2023-10-26T06:45:05.900313+0000","last_unstale":"2023-10-26T06:46:31.780489+0000","last_undegraded":"2023-10-26T06:46:31.780489+0000","last_fullsized":"2023-10-26T06:46:31.780489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:51.438741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764631+0000","last_change":"2023-10-26T06:45:05.909040+0000","last_active":"2023-10-26T06:46:31.764631+0000","last_peered":"2023-10-26T06:46:31.764631+0000","last_clean":"2023-10-26T06:46:31.764631+0000","last_became_active":"2023-10-26T06:45:05.908696+0000","last_became_peered":"2023-10-26T06:45:05.908696+0000","last_unstale":"2023-10-26T06:46:31.764631+0000","last_undegraded":"2023-10-26T06:46:31.764631+0000","last_fullsized":"2023-10-26T06:46:31.764631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:00.147712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764627+0000","last_change":"2023-10-26T06:45:05.898356+0000","last_active":"2023-10-26T06:46:31.764627+0000","last_peered":"2023-10-26T06:46:31.764627+0000","last_clean":"2023-10-26T06:46:31.764627+0000","last_became_active":"2023-10-26T06:45:05.898008+0000","last_became_peered":"2023-10-26T06:45:05.898008+0000","last_unstale":"2023-10-26T06:46:31.764627+0000","last_undegraded":"2023-10-26T06:46:31.764627+0000","last_fullsized":"2023-10-26T06:46:31.764627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:23.209987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781661+0000","last_change":"2023-10-26T06:45:05.911301+0000","last_active":"2023-10-26T06:46:31.781661+0000","last_peered":"2023-10-26T06:46:31.781661+0000","last_clean":"2023-10-26T06:46:31.781661+0000","last_became_active":"2023-10-26T06:45:05.910667+0000","last_became_peered":"2023-10-26T06:45:05.910667+0000","last_unstale":"2023-10-26T06:46:31.781661+0000","last_undegraded":"2023-10-26T06:46:31.781661+0000","last_fullsized":"2023-10-26T06:46:31.781661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:03:36.856248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.777770+0000","last_change":"2023-10-26T06:45:05.904456+0000","last_active":"2023-10-26T06:46:31.777770+0000","last_peered":"2023-10-26T06:46:31.777770+0000","last_clean":"2023-10-26T06:46:31.777770+0000","last_became_active":"2023-10-26T06:45:05.904165+0000","last_became_peered":"2023-10-26T06:45:05.904165+0000","last_unstale":"2023-10-26T06:46:31.777770+0000","last_undegraded":"2023-10-26T06:46:31.777770+0000","last_fullsized":"2023-10-26T06:46:31.777770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:51:01.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.768918+0000","last_change":"2023-10-26T06:46:03.232421+0000","last_active":"2023-10-26T06:46:31.768918+0000","last_peered":"2023-10-26T06:46:31.768918+0000","last_clean":"2023-10-26T06:46:31.768918+0000","last_became_active":"2023-10-26T06:46:03.232019+0000","last_became_peered":"2023-10-26T06:46:03.232019+0000","last_unstale":"2023-10-26T06:46:31.768918+0000","last_undegraded":"2023-10-26T06:46:31.768918+0000","last_fullsized":"2023-10-26T06:46:31.768918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:56.506294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779141+0000","last_change":"2023-10-26T06:45:05.906549+0000","last_active":"2023-10-26T06:46:31.779141+0000","last_peered":"2023-10-26T06:46:31.779141+0000","last_clean":"2023-10-26T06:46:31.779141+0000","last_became_active":"2023-10-26T06:45:05.905883+0000","last_became_peered":"2023-10-26T06:45:05.905883+0000","last_unstale":"2023-10-26T06:46:31.779141+0000","last_undegraded":"2023-10-26T06:46:31.779141+0000","last_fullsized":"2023-10-26T06:46:31.779141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:54.771031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763970+0000","last_change":"2023-10-26T06:46:03.230365+0000","last_active":"2023-10-26T06:46:31.763970+0000","last_peered":"2023-10-26T06:46:31.763970+0000","last_clean":"2023-10-26T06:46:31.763970+0000","last_became_active":"2023-10-26T06:46:03.230101+0000","last_became_peered":"2023-10-26T06:46:03.230101+0000","last_unstale":"2023-10-26T06:46:31.763970+0000","last_undegraded":"2023-10-26T06:46:31.763970+0000","last_fullsized":"2023-10-26T06:46:31.763970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:13.903128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779665+0000","last_change":"2023-10-26T06:45:05.908305+0000","last_active":"2023-10-26T06:46:31.779665+0000","last_peered":"2023-10-26T06:46:31.779665+0000","last_clean":"2023-10-26T06:46:31.779665+0000","last_became_active":"2023-10-26T06:45:05.907931+0000","last_became_peered":"2023-10-26T06:45:05.907931+0000","last_unstale":"2023-10-26T06:46:31.779665+0000","last_undegraded":"2023-10-26T06:46:31.779665+0000","last_fullsized":"2023-10-26T06:46:31.779665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:00:43.605324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.769635+0000","last_change":"2023-10-26T06:45:05.909737+0000","last_active":"2023-10-26T06:46:31.769635+0000","last_peered":"2023-10-26T06:46:31.769635+0000","last_clean":"2023-10-26T06:46:31.769635+0000","last_became_active":"2023-10-26T06:45:05.909445+0000","last_became_peered":"2023-10-26T06:45:05.909445+0000","last_unstale":"2023-10-26T06:46:31.769635+0000","last_undegraded":"2023-10-26T06:46:31.769635+0000","last_fullsized":"2023-10-26T06:46:31.769635+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:38.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778290+0000","last_change":"2023-10-26T06:45:05.907722+0000","last_active":"2023-10-26T06:46:31.778290+0000","last_peered":"2023-10-26T06:46:31.778290+0000","last_clean":"2023-10-26T06:46:31.778290+0000","last_became_active":"2023-10-26T06:45:05.905622+0000","last_became_peered":"2023-10-26T06:45:05.905622+0000","last_unstale":"2023-10-26T06:46:31.778290+0000","last_undegraded":"2023-10-26T06:46:31.778290+0000","last_fullsized":"2023-10-26T06:46:31.778290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:30:01.048032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779200+0000","last_change":"2023-10-26T06:45:05.907801+0000","last_active":"2023-10-26T06:46:31.779200+0000","last_peered":"2023-10-26T06:46:31.779200+0000","last_clean":"2023-10-26T06:46:31.779200+0000","last_became_active":"2023-10-26T06:45:05.907485+0000","last_became_peered":"2023-10-26T06:45:05.907485+0000","last_unstale":"2023-10-26T06:46:31.779200+0000","last_undegraded":"2023-10-26T06:46:31.779200+0000","last_fullsized":"2023-10-26T06:46:31.779200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:16.850992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782324+0000","last_change":"2023-10-26T06:45:05.902859+0000","last_active":"2023-10-26T06:46:31.782324+0000","last_peered":"2023-10-26T06:46:31.782324+0000","last_clean":"2023-10-26T06:46:31.782324+0000","last_became_active":"2023-10-26T06:45:05.902218+0000","last_became_peered":"2023-10-26T06:45:05.902218+0000","last_unstale":"2023-10-26T06:46:31.782324+0000","last_undegraded":"2023-10-26T06:46:31.782324+0000","last_fullsized":"2023-10-26T06:46:31.782324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:47.717227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763315+0000","last_change":"2023-10-26T06:45:05.910445+0000","last_active":"2023-10-26T06:46:31.763315+0000","last_peered":"2023-10-26T06:46:31.763315+0000","last_clean":"2023-10-26T06:46:31.763315+0000","last_became_active":"2023-10-26T06:45:05.910043+0000","last_became_peered":"2023-10-26T06:45:05.910043+0000","last_unstale":"2023-10-26T06:46:31.763315+0000","last_undegraded":"2023-10-26T06:46:31.763315+0000","last_fullsized":"2023-10-26T06:46:31.763315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:33.751134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775030+0000","last_change":"2023-10-26T06:45:05.905340+0000","last_active":"2023-10-26T06:46:31.775030+0000","last_peered":"2023-10-26T06:46:31.775030+0000","last_clean":"2023-10-26T06:46:31.775030+0000","last_became_active":"2023-10-26T06:45:05.905030+0000","last_became_peered":"2023-10-26T06:45:05.905030+0000","last_unstale":"2023-10-26T06:46:31.775030+0000","last_undegraded":"2023-10-26T06:46:31.775030+0000","last_fullsized":"2023-10-26T06:46:31.775030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.303210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226621+0000","last_change":"2023-10-26T06:45:04.919128+0000","last_active":"2023-10-26T06:46:02.226621+0000","last_peered":"2023-10-26T06:46:02.226621+0000","last_clean":"2023-10-26T06:46:02.226621+0000","last_became_active":"2023-10-26T06:45:04.913034+0000","last_became_peered":"2023-10-26T06:45:04.913034+0000","last_unstale":"2023-10-26T06:46:02.226621+0000","last_undegraded":"2023-10-26T06:46:02.226621+0000","last_fullsized":"2023-10-26T06:46:02.226621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:50.835037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.771222+0000","last_change":"2023-10-26T06:45:05.905315+0000","last_active":"2023-10-26T06:46:31.771222+0000","last_peered":"2023-10-26T06:46:31.771222+0000","last_clean":"2023-10-26T06:46:31.771222+0000","last_became_active":"2023-10-26T06:45:05.904260+0000","last_became_peered":"2023-10-26T06:45:05.904260+0000","last_unstale":"2023-10-26T06:46:31.771222+0000","last_undegraded":"2023-10-26T06:46:31.771222+0000","last_fullsized":"2023-10-26T06:46:31.771222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:02.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230870+0000","last_change":"2023-10-26T06:45:04.920440+0000","last_active":"2023-10-26T06:46:02.230870+0000","last_peered":"2023-10-26T06:46:02.230870+0000","last_clean":"2023-10-26T06:46:02.230870+0000","last_became_active":"2023-10-26T06:45:04.920129+0000","last_became_peered":"2023-10-26T06:45:04.920129+0000","last_unstale":"2023-10-26T06:46:02.230870+0000","last_undegraded":"2023-10-26T06:46:02.230870+0000","last_fullsized":"2023-10-26T06:46:02.230870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:43.176830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780674+0000","last_change":"2023-10-26T06:45:05.902898+0000","last_active":"2023-10-26T06:46:31.780674+0000","last_peered":"2023-10-26T06:46:31.780674+0000","last_clean":"2023-10-26T06:46:31.780674+0000","last_became_active":"2023-10-26T06:45:05.902161+0000","last_became_peered":"2023-10-26T06:45:05.902161+0000","last_unstale":"2023-10-26T06:46:31.780674+0000","last_undegraded":"2023-10-26T06:46:31.780674+0000","last_fullsized":"2023-10-26T06:46:31.780674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:27.872225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227991+0000","last_change":"2023-10-26T06:45:04.911835+0000","last_active":"2023-10-26T06:46:02.227991+0000","last_peered":"2023-10-26T06:46:02.227991+0000","last_clean":"2023-10-26T06:46:02.227991+0000","last_became_active":"2023-10-26T06:45:04.911505+0000","last_became_peered":"2023-10-26T06:45:04.911505+0000","last_unstale":"2023-10-26T06:46:02.227991+0000","last_undegraded":"2023-10-26T06:46:02.227991+0000","last_fullsized":"2023-10-26T06:46:02.227991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:01:18.971381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779287+0000","last_change":"2023-10-26T06:45:05.909088+0000","last_active":"2023-10-26T06:46:31.779287+0000","last_peered":"2023-10-26T06:46:31.779287+0000","last_clean":"2023-10-26T06:46:31.779287+0000","last_became_active":"2023-10-26T06:45:05.908528+0000","last_became_peered":"2023-10-26T06:45:05.908528+0000","last_unstale":"2023-10-26T06:46:31.779287+0000","last_undegraded":"2023-10-26T06:46:31.779287+0000","last_fullsized":"2023-10-26T06:46:31.779287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:07.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.237699+0000","last_change":"2023-10-26T06:46:03.236510+0000","last_active":"2023-10-26T06:46:03.237699+0000","last_peered":"2023-10-26T06:46:03.237699+0000","last_clean":"2023-10-26T06:46:03.237699+0000","last_became_active":"2023-10-26T06:46:03.233056+0000","last_became_peered":"2023-10-26T06:46:03.233056+0000","last_unstale":"2023-10-26T06:46:03.237699+0000","last_undegraded":"2023-10-26T06:46:03.237699+0000","last_fullsized":"2023-10-26T06:46:03.237699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:33.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.777342+0000","last_change":"2023-10-26T06:45:05.898795+0000","last_active":"2023-10-26T06:46:31.777342+0000","last_peered":"2023-10-26T06:46:31.777342+0000","last_clean":"2023-10-26T06:46:31.777342+0000","last_became_active":"2023-10-26T06:45:05.898489+0000","last_became_peered":"2023-10-26T06:45:05.898489+0000","last_unstale":"2023-10-26T06:46:31.777342+0000","last_undegraded":"2023-10-26T06:46:31.777342+0000","last_fullsized":"2023-10-26T06:46:31.777342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:55.343736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230147+0000","last_change":"2023-10-26T06:45:04.908881+0000","last_active":"2023-10-26T06:46:02.230147+0000","last_peered":"2023-10-26T06:46:02.230147+0000","last_clean":"2023-10-26T06:46:02.230147+0000","last_became_active":"2023-10-26T06:45:04.908515+0000","last_became_peered":"2023-10-26T06:45:04.908515+0000","last_unstale":"2023-10-26T06:46:02.230147+0000","last_undegraded":"2023-10-26T06:46:02.230147+0000","last_fullsized":"2023-10-26T06:46:02.230147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:39.200294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773736+0000","last_change":"2023-10-26T06:46:03.237686+0000","last_active":"2023-10-26T06:46:31.773736+0000","last_peered":"2023-10-26T06:46:31.773736+0000","last_clean":"2023-10-26T06:46:31.773736+0000","last_became_active":"2023-10-26T06:46:03.237275+0000","last_became_peered":"2023-10-26T06:46:03.237275+0000","last_unstale":"2023-10-26T06:46:31.773736+0000","last_undegraded":"2023-10-26T06:46:31.773736+0000","last_fullsized":"2023-10-26T06:46:31.773736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:56:22.281364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.234834+0000","last_change":"2023-10-26T06:46:03.234834+0000","last_active":"2023-10-26T06:46:03.234834+0000","last_peered":"2023-10-26T06:46:03.234834+0000","last_clean":"2023-10-26T06:46:03.234834+0000","last_became_active":"2023-10-26T06:46:03.234289+0000","last_became_peered":"2023-10-26T06:46:03.234289+0000","last_unstale":"2023-10-26T06:46:03.234834+0000","last_undegraded":"2023-10-26T06:46:03.234834+0000","last_fullsized":"2023-10-26T06:46:03.234834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:50:53.362840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773846+0000","last_change":"2023-10-26T06:45:05.901825+0000","last_active":"2023-10-26T06:46:31.773846+0000","last_peered":"2023-10-26T06:46:31.773846+0000","last_clean":"2023-10-26T06:46:31.773846+0000","last_became_active":"2023-10-26T06:45:05.901509+0000","last_became_peered":"2023-10-26T06:45:05.901509+0000","last_unstale":"2023-10-26T06:46:31.773846+0000","last_undegraded":"2023-10-26T06:46:31.773846+0000","last_fullsized":"2023-10-26T06:46:31.773846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:16.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227045+0000","last_change":"2023-10-26T06:45:04.919813+0000","last_active":"2023-10-26T06:46:02.227045+0000","last_peered":"2023-10-26T06:46:02.227045+0000","last_clean":"2023-10-26T06:46:02.227045+0000","last_became_active":"2023-10-26T06:45:04.918016+0000","last_became_peered":"2023-10-26T06:45:04.918016+0000","last_unstale":"2023-10-26T06:46:02.227045+0000","last_undegraded":"2023-10-26T06:46:02.227045+0000","last_fullsized":"2023-10-26T06:46:02.227045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:32.336126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779735+0000","last_change":"2023-10-26T06:45:05.902182+0000","last_active":"2023-10-26T06:46:31.779735+0000","last_peered":"2023-10-26T06:46:31.779735+0000","last_clean":"2023-10-26T06:46:31.779735+0000","last_became_active":"2023-10-26T06:45:05.901428+0000","last_became_peered":"2023-10-26T06:45:05.901428+0000","last_unstale":"2023-10-26T06:46:31.779735+0000","last_undegraded":"2023-10-26T06:46:31.779735+0000","last_fullsized":"2023-10-26T06:46:31.779735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:12.722532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.229834+0000","last_change":"2023-10-26T06:45:04.909713+0000","last_active":"2023-10-26T06:46:02.229834+0000","last_peered":"2023-10-26T06:46:02.229834+0000","last_clean":"2023-10-26T06:46:02.229834+0000","last_became_active":"2023-10-26T06:45:04.909412+0000","last_became_peered":"2023-10-26T06:45:04.909412+0000","last_unstale":"2023-10-26T06:46:02.229834+0000","last_undegraded":"2023-10-26T06:46:02.229834+0000","last_fullsized":"2023-10-26T06:46:02.229834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:23.901745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773996+0000","last_change":"2023-10-26T06:45:05.908348+0000","last_active":"2023-10-26T06:46:31.773996+0000","last_peered":"2023-10-26T06:46:31.773996+0000","last_clean":"2023-10-26T06:46:31.773996+0000","last_became_active":"2023-10-26T06:45:05.908022+0000","last_became_peered":"2023-10-26T06:45:05.908022+0000","last_unstale":"2023-10-26T06:46:31.773996+0000","last_undegraded":"2023-10-26T06:46:31.773996+0000","last_fullsized":"2023-10-26T06:46:31.773996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:22:16.027732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.223807+0000","last_change":"2023-10-26T06:45:04.915491+0000","last_active":"2023-10-26T06:46:02.223807+0000","last_peered":"2023-10-26T06:46:02.223807+0000","last_clean":"2023-10-26T06:46:02.223807+0000","last_became_active":"2023-10-26T06:45:04.915191+0000","last_became_peered":"2023-10-26T06:45:04.915191+0000","last_unstale":"2023-10-26T06:46:02.223807+0000","last_undegraded":"2023-10-26T06:46:02.223807+0000","last_fullsized":"2023-10-26T06:46:02.223807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:57.758903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779813+0000","last_change":"2023-10-26T06:45:05.912676+0000","last_active":"2023-10-26T06:46:31.779813+0000","last_peered":"2023-10-26T06:46:31.779813+0000","last_clean":"2023-10-26T06:46:31.779813+0000","last_became_active":"2023-10-26T06:45:05.912372+0000","last_became_peered":"2023-10-26T06:45:05.912372+0000","last_unstale":"2023-10-26T06:46:31.779813+0000","last_undegraded":"2023-10-26T06:46:31.779813+0000","last_fullsized":"2023-10-26T06:46:31.779813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:33.299166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.474091+0000","last_change":"2023-10-26T06:45:04.912123+0000","last_active":"2023-10-26T06:46:02.474091+0000","last_peered":"2023-10-26T06:46:02.474091+0000","last_clean":"2023-10-26T06:46:02.474091+0000","last_became_active":"2023-10-26T06:45:04.911893+0000","last_became_peered":"2023-10-26T06:45:04.911893+0000","last_unstale":"2023-10-26T06:46:02.474091+0000","last_undegraded":"2023-10-26T06:46:02.474091+0000","last_fullsized":"2023-10-26T06:46:02.474091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:32.907496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780208+0000","last_change":"2023-10-26T06:45:05.893434+0000","last_active":"2023-10-26T06:46:31.780208+0000","last_peered":"2023-10-26T06:46:31.780208+0000","last_clean":"2023-10-26T06:46:31.780208+0000","last_became_active":"2023-10-26T06:45:05.893100+0000","last_became_peered":"2023-10-26T06:45:05.893100+0000","last_unstale":"2023-10-26T06:46:31.780208+0000","last_undegraded":"2023-10-26T06:46:31.780208+0000","last_fullsized":"2023-10-26T06:46:31.780208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:22:48.317428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230570+0000","last_change":"2023-10-26T06:45:04.908910+0000","last_active":"2023-10-26T06:46:02.230570+0000","last_peered":"2023-10-26T06:46:02.230570+0000","last_clean":"2023-10-26T06:46:02.230570+0000","last_became_active":"2023-10-26T06:45:04.908592+0000","last_became_peered":"2023-10-26T06:45:04.908592+0000","last_unstale":"2023-10-26T06:46:02.230570+0000","last_undegraded":"2023-10-26T06:46:02.230570+0000","last_fullsized":"2023-10-26T06:46:02.230570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:40.266013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.776098+0000","last_change":"2023-10-26T06:45:05.908887+0000","last_active":"2023-10-26T06:46:31.776098+0000","last_peered":"2023-10-26T06:46:31.776098+0000","last_clean":"2023-10-26T06:46:31.776098+0000","last_became_active":"2023-10-26T06:45:05.908163+0000","last_became_peered":"2023-10-26T06:45:05.908163+0000","last_unstale":"2023-10-26T06:46:31.776098+0000","last_undegraded":"2023-10-26T06:46:31.776098+0000","last_fullsized":"2023-10-26T06:46:31.776098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:09.339374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":253,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:47.249475+0000","last_change":"2023-10-26T06:45:04.905691+0000","last_active":"2023-10-26T06:46:47.249475+0000","last_peered":"2023-10-26T06:46:47.249475+0000","last_clean":"2023-10-26T06:46:47.249475+0000","last_became_active":"2023-10-26T06:45:04.905378+0000","last_became_peered":"2023-10-26T06:45:04.905378+0000","last_unstale":"2023-10-26T06:46:47.249475+0000","last_undegraded":"2023-10-26T06:46:47.249475+0000","last_fullsized":"2023-10-26T06:46:47.249475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T18:24:21.498141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64519,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":208,"num_read_kb":166,"num_write":30,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.765219+0000","last_change":"2023-10-26T06:45:05.897366+0000","last_active":"2023-10-26T06:46:31.765219+0000","last_peered":"2023-10-26T06:46:31.765219+0000","last_clean":"2023-10-26T06:46:31.765219+0000","last_became_active":"2023-10-26T06:45:05.897091+0000","last_became_peered":"2023-10-26T06:45:05.897091+0000","last_unstale":"2023-10-26T06:46:31.765219+0000","last_undegraded":"2023-10-26T06:46:31.765219+0000","last_fullsized":"2023-10-26T06:46:31.765219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:11.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226619+0000","last_change":"2023-10-26T06:45:04.914949+0000","last_active":"2023-10-26T06:46:02.226619+0000","last_peered":"2023-10-26T06:46:02.226619+0000","last_clean":"2023-10-26T06:46:02.226619+0000","last_became_active":"2023-10-26T06:45:04.914610+0000","last_became_peered":"2023-10-26T06:45:04.914610+0000","last_unstale":"2023-10-26T06:46:02.226619+0000","last_undegraded":"2023-10-26T06:46:02.226619+0000","last_fullsized":"2023-10-26T06:46:02.226619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:16.683352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774454+0000","last_change":"2023-10-26T06:45:05.911067+0000","last_active":"2023-10-26T06:46:31.774454+0000","last_peered":"2023-10-26T06:46:31.774454+0000","last_clean":"2023-10-26T06:46:31.774454+0000","last_became_active":"2023-10-26T06:45:05.908460+0000","last_became_peered":"2023-10-26T06:45:05.908460+0000","last_unstale":"2023-10-26T06:46:31.774454+0000","last_undegraded":"2023-10-26T06:46:31.774454+0000","last_fullsized":"2023-10-26T06:46:31.774454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:17.461781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230798+0000","last_change":"2023-10-26T06:45:04.919947+0000","last_active":"2023-10-26T06:46:02.230798+0000","last_peered":"2023-10-26T06:46:02.230798+0000","last_clean":"2023-10-26T06:46:02.230798+0000","last_became_active":"2023-10-26T06:45:04.919535+0000","last_became_peered":"2023-10-26T06:45:04.919535+0000","last_unstale":"2023-10-26T06:46:02.230798+0000","last_undegraded":"2023-10-26T06:46:02.230798+0000","last_fullsized":"2023-10-26T06:46:02.230798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:46:09.201139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782275+0000","last_change":"2023-10-26T06:45:05.906381+0000","last_active":"2023-10-26T06:46:31.782275+0000","last_peered":"2023-10-26T06:46:31.782275+0000","last_clean":"2023-10-26T06:46:31.782275+0000","last_became_active":"2023-10-26T06:45:05.904487+0000","last_became_peered":"2023-10-26T06:45:05.904487+0000","last_unstale":"2023-10-26T06:46:31.782275+0000","last_undegraded":"2023-10-26T06:46:31.782275+0000","last_fullsized":"2023-10-26T06:46:31.782275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:00.164936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227232+0000","last_change":"2023-10-26T06:45:04.908779+0000","last_active":"2023-10-26T06:46:02.227232+0000","last_peered":"2023-10-26T06:46:02.227232+0000","last_clean":"2023-10-26T06:46:02.227232+0000","last_became_active":"2023-10-26T06:45:04.908490+0000","last_became_peered":"2023-10-26T06:45:04.908490+0000","last_unstale":"2023-10-26T06:46:02.227232+0000","last_undegraded":"2023-10-26T06:46:02.227232+0000","last_fullsized":"2023-10-26T06:46:02.227232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:12:13.199863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764235+0000","last_change":"2023-10-26T06:45:05.909408+0000","last_active":"2023-10-26T06:46:31.764235+0000","last_peered":"2023-10-26T06:46:31.764235+0000","last_clean":"2023-10-26T06:46:31.764235+0000","last_became_active":"2023-10-26T06:45:05.909107+0000","last_became_peered":"2023-10-26T06:45:05.909107+0000","last_unstale":"2023-10-26T06:46:31.764235+0000","last_undegraded":"2023-10-26T06:46:31.764235+0000","last_fullsized":"2023-10-26T06:46:31.764235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:05.887472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.424977+0000","last_change":"2023-10-26T06:46:03.424977+0000","last_active":"2023-10-26T06:46:03.424977+0000","last_peered":"2023-10-26T06:46:03.424977+0000","last_clean":"2023-10-26T06:46:03.424977+0000","last_became_active":"2023-10-26T06:46:03.417899+0000","last_became_peered":"2023-10-26T06:46:03.417899+0000","last_unstale":"2023-10-26T06:46:03.424977+0000","last_undegraded":"2023-10-26T06:46:03.424977+0000","last_fullsized":"2023-10-26T06:46:03.424977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:51.092657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775545+0000","last_change":"2023-10-26T06:45:05.902156+0000","last_active":"2023-10-26T06:46:31.775545+0000","last_peered":"2023-10-26T06:46:31.775545+0000","last_clean":"2023-10-26T06:46:31.775545+0000","last_became_active":"2023-10-26T06:45:05.901865+0000","last_became_peered":"2023-10-26T06:45:05.901865+0000","last_unstale":"2023-10-26T06:46:31.775545+0000","last_undegraded":"2023-10-26T06:46:31.775545+0000","last_fullsized":"2023-10-26T06:46:31.775545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:06:08.167280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.228254+0000","last_change":"2023-10-26T06:45:04.913071+0000","last_active":"2023-10-26T06:46:02.228254+0000","last_peered":"2023-10-26T06:46:02.228254+0000","last_clean":"2023-10-26T06:46:02.228254+0000","last_became_active":"2023-10-26T06:45:04.912736+0000","last_became_peered":"2023-10-26T06:45:04.912736+0000","last_unstale":"2023-10-26T06:46:02.228254+0000","last_undegraded":"2023-10-26T06:46:02.228254+0000","last_fullsized":"2023-10-26T06:46:02.228254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:59.690355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.757581+0000","last_change":"2023-10-26T06:46:03.233858+0000","last_active":"2023-10-26T06:46:31.757581+0000","last_peered":"2023-10-26T06:46:31.757581+0000","last_clean":"2023-10-26T06:46:31.757581+0000","last_became_active":"2023-10-26T06:46:03.233429+0000","last_became_peered":"2023-10-26T06:46:03.233429+0000","last_unstale":"2023-10-26T06:46:31.757581+0000","last_undegraded":"2023-10-26T06:46:31.757581+0000","last_fullsized":"2023-10-26T06:46:31.757581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:41:39.237862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:03.237541+0000","last_change":"2023-10-26T06:46:03.237541+0000","last_active":"2023-10-26T06:46:03.237541+0000","last_peered":"2023-10-26T06:46:03.237541+0000","last_clean":"2023-10-26T06:46:03.237541+0000","last_became_active":"2023-10-26T06:46:03.237019+0000","last_became_peered":"2023-10-26T06:46:03.237019+0000","last_unstale":"2023-10-26T06:46:03.237541+0000","last_undegraded":"2023-10-26T06:46:03.237541+0000","last_fullsized":"2023-10-26T06:46:03.237541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:59:15.383799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.775563+0000","last_change":"2023-10-26T06:45:05.904832+0000","last_active":"2023-10-26T06:46:31.775563+0000","last_peered":"2023-10-26T06:46:31.775563+0000","last_clean":"2023-10-26T06:46:31.775563+0000","last_became_active":"2023-10-26T06:45:05.904527+0000","last_became_peered":"2023-10-26T06:45:05.904527+0000","last_unstale":"2023-10-26T06:46:31.775563+0000","last_undegraded":"2023-10-26T06:46:31.775563+0000","last_fullsized":"2023-10-26T06:46:31.775563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:30.546198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473828+0000","last_change":"2023-10-26T06:45:04.922832+0000","last_active":"2023-10-26T06:46:02.473828+0000","last_peered":"2023-10-26T06:46:02.473828+0000","last_clean":"2023-10-26T06:46:02.473828+0000","last_became_active":"2023-10-26T06:45:04.922349+0000","last_became_peered":"2023-10-26T06:45:04.922349+0000","last_unstale":"2023-10-26T06:46:02.473828+0000","last_undegraded":"2023-10-26T06:46:02.473828+0000","last_fullsized":"2023-10-26T06:46:02.473828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:37.839093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.770899+0000","last_change":"2023-10-26T06:45:05.909946+0000","last_active":"2023-10-26T06:46:31.770899+0000","last_peered":"2023-10-26T06:46:31.770899+0000","last_clean":"2023-10-26T06:46:31.770899+0000","last_became_active":"2023-10-26T06:45:05.909650+0000","last_became_peered":"2023-10-26T06:45:05.909650+0000","last_unstale":"2023-10-26T06:46:31.770899+0000","last_undegraded":"2023-10-26T06:46:31.770899+0000","last_fullsized":"2023-10-26T06:46:31.770899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:41.813928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763208+0000","last_change":"2023-10-26T06:45:05.903400+0000","last_active":"2023-10-26T06:46:31.763208+0000","last_peered":"2023-10-26T06:46:31.763208+0000","last_clean":"2023-10-26T06:46:31.763208+0000","last_became_active":"2023-10-26T06:45:05.903101+0000","last_became_peered":"2023-10-26T06:45:05.903101+0000","last_unstale":"2023-10-26T06:46:31.763208+0000","last_undegraded":"2023-10-26T06:46:31.763208+0000","last_fullsized":"2023-10-26T06:46:31.763208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227030+0000","last_change":"2023-10-26T06:45:04.923960+0000","last_active":"2023-10-26T06:46:02.227030+0000","last_peered":"2023-10-26T06:46:02.227030+0000","last_clean":"2023-10-26T06:46:02.227030+0000","last_became_active":"2023-10-26T06:45:04.923613+0000","last_became_peered":"2023-10-26T06:45:04.923613+0000","last_unstale":"2023-10-26T06:46:02.227030+0000","last_undegraded":"2023-10-26T06:46:02.227030+0000","last_fullsized":"2023-10-26T06:46:02.227030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:53.342463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778716+0000","last_change":"2023-10-26T06:45:05.903966+0000","last_active":"2023-10-26T06:46:31.778716+0000","last_peered":"2023-10-26T06:46:31.778716+0000","last_clean":"2023-10-26T06:46:31.778716+0000","last_became_active":"2023-10-26T06:45:05.903583+0000","last_became_peered":"2023-10-26T06:45:05.903583+0000","last_unstale":"2023-10-26T06:46:31.778716+0000","last_undegraded":"2023-10-26T06:46:31.778716+0000","last_fullsized":"2023-10-26T06:46:31.778716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:18.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226986+0000","last_change":"2023-10-26T06:45:04.907838+0000","last_active":"2023-10-26T06:46:02.226986+0000","last_peered":"2023-10-26T06:46:02.226986+0000","last_clean":"2023-10-26T06:46:02.226986+0000","last_became_active":"2023-10-26T06:45:04.907508+0000","last_became_peered":"2023-10-26T06:45:04.907508+0000","last_unstale":"2023-10-26T06:46:02.226986+0000","last_undegraded":"2023-10-26T06:46:02.226986+0000","last_fullsized":"2023-10-26T06:46:02.226986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:34.852243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":321,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473730+0000","last_change":"2023-10-26T06:43:59.765466+0000","last_active":"2023-10-26T06:46:02.473730+0000","last_peered":"2023-10-26T06:46:02.473730+0000","last_clean":"2023-10-26T06:46:02.473730+0000","last_became_active":"2023-10-26T06:43:59.765005+0000","last_became_peered":"2023-10-26T06:43:59.765005+0000","last_unstale":"2023-10-26T06:46:02.473730+0000","last_undegraded":"2023-10-26T06:46:02.473730+0000","last_fullsized":"2023-10-26T06:46:02.473730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:40:50.556448+0000","last_clean_scrub_stamp":"2023-10-26T06:40:50.556448+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:18.688941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774103+0000","last_change":"2023-10-26T06:45:05.906909+0000","last_active":"2023-10-26T06:46:31.774103+0000","last_peered":"2023-10-26T06:46:31.774103+0000","last_clean":"2023-10-26T06:46:31.774103+0000","last_became_active":"2023-10-26T06:45:05.906531+0000","last_became_peered":"2023-10-26T06:45:05.906531+0000","last_unstale":"2023-10-26T06:46:31.774103+0000","last_undegraded":"2023-10-26T06:46:31.774103+0000","last_fullsized":"2023-10-26T06:46:31.774103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.473805+0000","last_change":"2023-10-26T06:45:04.911635+0000","last_active":"2023-10-26T06:46:02.473805+0000","last_peered":"2023-10-26T06:46:02.473805+0000","last_clean":"2023-10-26T06:46:02.473805+0000","last_became_active":"2023-10-26T06:45:04.911289+0000","last_became_peered":"2023-10-26T06:45:04.911289+0000","last_unstale":"2023-10-26T06:46:02.473805+0000","last_undegraded":"2023-10-26T06:46:02.473805+0000","last_fullsized":"2023-10-26T06:46:02.473805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.115211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227809+0000","last_change":"2023-10-26T06:45:04.919853+0000","last_active":"2023-10-26T06:46:02.227809+0000","last_peered":"2023-10-26T06:46:02.227809+0000","last_clean":"2023-10-26T06:46:02.227809+0000","last_became_active":"2023-10-26T06:45:04.919488+0000","last_became_peered":"2023-10-26T06:45:04.919488+0000","last_unstale":"2023-10-26T06:46:02.227809+0000","last_undegraded":"2023-10-26T06:46:02.227809+0000","last_fullsized":"2023-10-26T06:46:02.227809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:06.555851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.776607+0000","last_change":"2023-10-26T06:45:05.903975+0000","last_active":"2023-10-26T06:46:31.776607+0000","last_peered":"2023-10-26T06:46:31.776607+0000","last_clean":"2023-10-26T06:46:31.776607+0000","last_became_active":"2023-10-26T06:45:05.903539+0000","last_became_peered":"2023-10-26T06:45:05.903539+0000","last_unstale":"2023-10-26T06:46:31.776607+0000","last_undegraded":"2023-10-26T06:46:31.776607+0000","last_fullsized":"2023-10-26T06:46:31.776607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:30.648947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:33.946402+0000","last_change":"2023-10-26T06:45:04.919815+0000","last_active":"2023-10-26T06:46:33.946402+0000","last_peered":"2023-10-26T06:46:33.946402+0000","last_clean":"2023-10-26T06:46:33.946402+0000","last_became_active":"2023-10-26T06:45:04.918141+0000","last_became_peered":"2023-10-26T06:45:04.918141+0000","last_unstale":"2023-10-26T06:46:33.946402+0000","last_undegraded":"2023-10-26T06:46:33.946402+0000","last_fullsized":"2023-10-26T06:46:33.946402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:53.024534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.774509+0000","last_change":"2023-10-26T06:45:05.898180+0000","last_active":"2023-10-26T06:46:31.774509+0000","last_peered":"2023-10-26T06:46:31.774509+0000","last_clean":"2023-10-26T06:46:31.774509+0000","last_became_active":"2023-10-26T06:45:05.896937+0000","last_became_peered":"2023-10-26T06:45:05.896937+0000","last_unstale":"2023-10-26T06:46:31.774509+0000","last_undegraded":"2023-10-26T06:46:31.774509+0000","last_fullsized":"2023-10-26T06:46:31.774509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:48.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227006+0000","last_change":"2023-10-26T06:45:04.922456+0000","last_active":"2023-10-26T06:46:02.227006+0000","last_peered":"2023-10-26T06:46:02.227006+0000","last_clean":"2023-10-26T06:46:02.227006+0000","last_became_active":"2023-10-26T06:45:04.922112+0000","last_became_peered":"2023-10-26T06:45:04.922112+0000","last_unstale":"2023-10-26T06:46:02.227006+0000","last_undegraded":"2023-10-26T06:46:02.227006+0000","last_fullsized":"2023-10-26T06:46:02.227006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:44.055569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781350+0000","last_change":"2023-10-26T06:45:05.895440+0000","last_active":"2023-10-26T06:46:31.781350+0000","last_peered":"2023-10-26T06:46:31.781350+0000","last_clean":"2023-10-26T06:46:31.781350+0000","last_became_active":"2023-10-26T06:45:05.892605+0000","last_became_peered":"2023-10-26T06:45:05.892605+0000","last_unstale":"2023-10-26T06:46:31.781350+0000","last_undegraded":"2023-10-26T06:46:31.781350+0000","last_fullsized":"2023-10-26T06:46:31.781350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:09:27.392772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.225650+0000","last_change":"2023-10-26T06:45:04.916042+0000","last_active":"2023-10-26T06:46:02.225650+0000","last_peered":"2023-10-26T06:46:02.225650+0000","last_clean":"2023-10-26T06:46:02.225650+0000","last_became_active":"2023-10-26T06:45:04.915649+0000","last_became_peered":"2023-10-26T06:45:04.915649+0000","last_unstale":"2023-10-26T06:46:02.225650+0000","last_undegraded":"2023-10-26T06:46:02.225650+0000","last_fullsized":"2023-10-26T06:46:02.225650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:30.060074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.735910+0000","last_change":"2023-10-26T06:45:05.908179+0000","last_active":"2023-10-26T06:46:31.735910+0000","last_peered":"2023-10-26T06:46:31.735910+0000","last_clean":"2023-10-26T06:46:31.735910+0000","last_became_active":"2023-10-26T06:45:05.907857+0000","last_became_peered":"2023-10-26T06:45:05.907857+0000","last_unstale":"2023-10-26T06:46:31.735910+0000","last_undegraded":"2023-10-26T06:46:31.735910+0000","last_fullsized":"2023-10-26T06:46:31.735910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:20.017194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230341+0000","last_change":"2023-10-26T06:45:04.921016+0000","last_active":"2023-10-26T06:46:02.230341+0000","last_peered":"2023-10-26T06:46:02.230341+0000","last_clean":"2023-10-26T06:46:02.230341+0000","last_became_active":"2023-10-26T06:45:04.920741+0000","last_became_peered":"2023-10-26T06:45:04.920741+0000","last_unstale":"2023-10-26T06:46:02.230341+0000","last_undegraded":"2023-10-26T06:46:02.230341+0000","last_fullsized":"2023-10-26T06:46:02.230341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:05:06.218205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778934+0000","last_change":"2023-10-26T06:45:05.893562+0000","last_active":"2023-10-26T06:46:31.778934+0000","last_peered":"2023-10-26T06:46:31.778934+0000","last_clean":"2023-10-26T06:46:31.778934+0000","last_became_active":"2023-10-26T06:45:05.893111+0000","last_became_peered":"2023-10-26T06:45:05.893111+0000","last_unstale":"2023-10-26T06:46:31.778934+0000","last_undegraded":"2023-10-26T06:46:31.778934+0000","last_fullsized":"2023-10-26T06:46:31.778934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:29.305664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226475+0000","last_change":"2023-10-26T06:45:04.914936+0000","last_active":"2023-10-26T06:46:02.226475+0000","last_peered":"2023-10-26T06:46:02.226475+0000","last_clean":"2023-10-26T06:46:02.226475+0000","last_became_active":"2023-10-26T06:45:04.914617+0000","last_became_peered":"2023-10-26T06:45:04.914617+0000","last_unstale":"2023-10-26T06:46:02.226475+0000","last_undegraded":"2023-10-26T06:46:02.226475+0000","last_fullsized":"2023-10-26T06:46:02.226475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:15.462597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.778284+0000","last_change":"2023-10-26T06:45:05.909101+0000","last_active":"2023-10-26T06:46:31.778284+0000","last_peered":"2023-10-26T06:46:31.778284+0000","last_clean":"2023-10-26T06:46:31.778284+0000","last_became_active":"2023-10-26T06:45:05.908800+0000","last_became_peered":"2023-10-26T06:45:05.908800+0000","last_unstale":"2023-10-26T06:46:31.778284+0000","last_undegraded":"2023-10-26T06:46:31.778284+0000","last_fullsized":"2023-10-26T06:46:31.778284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:37:18.605102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227537+0000","last_change":"2023-10-26T06:45:04.911714+0000","last_active":"2023-10-26T06:46:02.227537+0000","last_peered":"2023-10-26T06:46:02.227537+0000","last_clean":"2023-10-26T06:46:02.227537+0000","last_became_active":"2023-10-26T06:45:04.911343+0000","last_became_peered":"2023-10-26T06:45:04.911343+0000","last_unstale":"2023-10-26T06:46:02.227537+0000","last_undegraded":"2023-10-26T06:46:02.227537+0000","last_fullsized":"2023-10-26T06:46:02.227537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:30:35.120642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773867+0000","last_change":"2023-10-26T06:45:05.906804+0000","last_active":"2023-10-26T06:46:31.773867+0000","last_peered":"2023-10-26T06:46:31.773867+0000","last_clean":"2023-10-26T06:46:31.773867+0000","last_became_active":"2023-10-26T06:45:05.906522+0000","last_became_peered":"2023-10-26T06:45:05.906522+0000","last_unstale":"2023-10-26T06:46:31.773867+0000","last_undegraded":"2023-10-26T06:46:31.773867+0000","last_fullsized":"2023-10-26T06:46:31.773867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:32.707204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.226957+0000","last_change":"2023-10-26T06:45:04.906345+0000","last_active":"2023-10-26T06:46:02.226957+0000","last_peered":"2023-10-26T06:46:02.226957+0000","last_clean":"2023-10-26T06:46:02.226957+0000","last_became_active":"2023-10-26T06:45:04.906034+0000","last_became_peered":"2023-10-26T06:45:04.906034+0000","last_unstale":"2023-10-26T06:46:02.226957+0000","last_undegraded":"2023-10-26T06:46:02.226957+0000","last_fullsized":"2023-10-26T06:46:02.226957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:53.690788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779248+0000","last_change":"2023-10-26T06:45:05.906368+0000","last_active":"2023-10-26T06:46:31.779248+0000","last_peered":"2023-10-26T06:46:31.779248+0000","last_clean":"2023-10-26T06:46:31.779248+0000","last_became_active":"2023-10-26T06:45:05.905471+0000","last_became_peered":"2023-10-26T06:45:05.905471+0000","last_unstale":"2023-10-26T06:46:31.779248+0000","last_undegraded":"2023-10-26T06:46:31.779248+0000","last_fullsized":"2023-10-26T06:46:31.779248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:45.493704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.230243+0000","last_change":"2023-10-26T06:45:04.917530+0000","last_active":"2023-10-26T06:46:02.230243+0000","last_peered":"2023-10-26T06:46:02.230243+0000","last_clean":"2023-10-26T06:46:02.230243+0000","last_became_active":"2023-10-26T06:45:04.917193+0000","last_became_peered":"2023-10-26T06:45:04.917193+0000","last_unstale":"2023-10-26T06:46:02.230243+0000","last_undegraded":"2023-10-26T06:46:02.230243+0000","last_fullsized":"2023-10-26T06:46:02.230243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.965048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.765239+0000","last_change":"2023-10-26T06:45:05.899429+0000","last_active":"2023-10-26T06:46:31.765239+0000","last_peered":"2023-10-26T06:46:31.765239+0000","last_clean":"2023-10-26T06:46:31.765239+0000","last_became_active":"2023-10-26T06:45:05.899081+0000","last_became_peered":"2023-10-26T06:45:05.899081+0000","last_unstale":"2023-10-26T06:46:31.765239+0000","last_undegraded":"2023-10-26T06:46:31.765239+0000","last_fullsized":"2023-10-26T06:46:31.765239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:41.783840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T06:46:02.227783+0000","last_change":"2023-10-26T06:45:04.920295+0000","last_active":"2023-10-26T06:46:02.227783+0000","last_peered":"2023-10-26T06:46:02.227783+0000","last_clean":"2023-10-26T06:46:02.227783+0000","last_became_active":"2023-10-26T06:45:04.919920+0000","last_became_peered":"2023-10-26T06:45:04.919920+0000","last_unstale":"2023-10-26T06:46:02.227783+0000","last_undegraded":"2023-10-26T06:46:02.227783+0000","last_fullsized":"2023-10-26T06:46:02.227783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:03.889613+0000","last_clean_scrub_stamp":"2023-10-26T06:45:03.889613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:59.260841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.764977+0000","last_change":"2023-10-26T06:45:05.899610+0000","last_active":"2023-10-26T06:46:31.764977+0000","last_peered":"2023-10-26T06:46:31.764977+0000","last_clean":"2023-10-26T06:46:31.764977+0000","last_became_active":"2023-10-26T06:45:05.899283+0000","last_became_peered":"2023-10-26T06:45:05.899283+0000","last_unstale":"2023-10-26T06:46:31.764977+0000","last_undegraded":"2023-10-26T06:46:31.764977+0000","last_fullsized":"2023-10-26T06:46:31.764977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:48.730703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.759894+0000","last_change":"2023-10-26T06:45:05.892012+0000","last_active":"2023-10-26T06:46:31.759894+0000","last_peered":"2023-10-26T06:46:31.759894+0000","last_clean":"2023-10-26T06:46:31.759894+0000","last_became_active":"2023-10-26T06:45:05.891744+0000","last_became_peered":"2023-10-26T06:45:05.891744+0000","last_unstale":"2023-10-26T06:46:31.759894+0000","last_undegraded":"2023-10-26T06:46:31.759894+0000","last_fullsized":"2023-10-26T06:46:31.759894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:29.599184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.763039+0000","last_change":"2023-10-26T06:45:05.904348+0000","last_active":"2023-10-26T06:46:31.763039+0000","last_peered":"2023-10-26T06:46:31.763039+0000","last_clean":"2023-10-26T06:46:31.763039+0000","last_became_active":"2023-10-26T06:45:05.903949+0000","last_became_peered":"2023-10-26T06:45:05.903949+0000","last_unstale":"2023-10-26T06:46:31.763039+0000","last_undegraded":"2023-10-26T06:46:31.763039+0000","last_fullsized":"2023-10-26T06:46:31.763039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:29.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.771138+0000","last_change":"2023-10-26T06:46:03.237595+0000","last_active":"2023-10-26T06:46:31.771138+0000","last_peered":"2023-10-26T06:46:31.771138+0000","last_clean":"2023-10-26T06:46:31.771138+0000","last_became_active":"2023-10-26T06:46:03.237136+0000","last_became_peered":"2023-10-26T06:46:03.237136+0000","last_unstale":"2023-10-26T06:46:31.771138+0000","last_undegraded":"2023-10-26T06:46:31.771138+0000","last_fullsized":"2023-10-26T06:46:31.771138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:03.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.780714+0000","last_change":"2023-10-26T06:45:05.901225+0000","last_active":"2023-10-26T06:46:31.780714+0000","last_peered":"2023-10-26T06:46:31.780714+0000","last_clean":"2023-10-26T06:46:31.780714+0000","last_became_active":"2023-10-26T06:45:05.900743+0000","last_became_peered":"2023-10-26T06:45:05.900743+0000","last_unstale":"2023-10-26T06:46:31.780714+0000","last_undegraded":"2023-10-26T06:46:31.780714+0000","last_fullsized":"2023-10-26T06:46:31.780714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:02.359654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781128+0000","last_change":"2023-10-26T06:45:05.906805+0000","last_active":"2023-10-26T06:46:31.781128+0000","last_peered":"2023-10-26T06:46:31.781128+0000","last_clean":"2023-10-26T06:46:31.781128+0000","last_became_active":"2023-10-26T06:45:05.906464+0000","last_became_peered":"2023-10-26T06:45:05.906464+0000","last_unstale":"2023-10-26T06:46:31.781128+0000","last_undegraded":"2023-10-26T06:46:31.781128+0000","last_fullsized":"2023-10-26T06:46:31.781128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:24.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.759205+0000","last_change":"2023-10-26T06:45:05.898568+0000","last_active":"2023-10-26T06:46:31.759205+0000","last_peered":"2023-10-26T06:46:31.759205+0000","last_clean":"2023-10-26T06:46:31.759205+0000","last_became_active":"2023-10-26T06:45:05.897886+0000","last_became_peered":"2023-10-26T06:45:05.897886+0000","last_unstale":"2023-10-26T06:46:31.759205+0000","last_undegraded":"2023-10-26T06:46:31.759205+0000","last_fullsized":"2023-10-26T06:46:31.759205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:14.998233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779508+0000","last_change":"2023-10-26T06:45:05.899326+0000","last_active":"2023-10-26T06:46:31.779508+0000","last_peered":"2023-10-26T06:46:31.779508+0000","last_clean":"2023-10-26T06:46:31.779508+0000","last_became_active":"2023-10-26T06:45:05.899040+0000","last_became_peered":"2023-10-26T06:45:05.899040+0000","last_unstale":"2023-10-26T06:46:31.779508+0000","last_undegraded":"2023-10-26T06:46:31.779508+0000","last_fullsized":"2023-10-26T06:46:31.779508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:55:35.666122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773908+0000","last_change":"2023-10-26T06:45:05.904155+0000","last_active":"2023-10-26T06:46:31.773908+0000","last_peered":"2023-10-26T06:46:31.773908+0000","last_clean":"2023-10-26T06:46:31.773908+0000","last_became_active":"2023-10-26T06:45:05.903734+0000","last_became_peered":"2023-10-26T06:45:05.903734+0000","last_unstale":"2023-10-26T06:46:31.773908+0000","last_undegraded":"2023-10-26T06:46:31.773908+0000","last_fullsized":"2023-10-26T06:46:31.773908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:43.252789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.781034+0000","last_change":"2023-10-26T06:45:05.897174+0000","last_active":"2023-10-26T06:46:31.781034+0000","last_peered":"2023-10-26T06:46:31.781034+0000","last_clean":"2023-10-26T06:46:31.781034+0000","last_became_active":"2023-10-26T06:45:05.896893+0000","last_became_peered":"2023-10-26T06:45:05.896893+0000","last_unstale":"2023-10-26T06:46:31.781034+0000","last_undegraded":"2023-10-26T06:46:31.781034+0000","last_fullsized":"2023-10-26T06:46:31.781034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:14.291682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.773128+0000","last_change":"2023-10-26T06:45:05.905987+0000","last_active":"2023-10-26T06:46:31.773128+0000","last_peered":"2023-10-26T06:46:31.773128+0000","last_clean":"2023-10-26T06:46:31.773128+0000","last_became_active":"2023-10-26T06:45:05.904933+0000","last_became_peered":"2023-10-26T06:45:05.904933+0000","last_unstale":"2023-10-26T06:46:31.773128+0000","last_undegraded":"2023-10-26T06:46:31.773128+0000","last_fullsized":"2023-10-26T06:46:31.773128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:19.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.782354+0000","last_change":"2023-10-26T06:45:05.912781+0000","last_active":"2023-10-26T06:46:31.782354+0000","last_peered":"2023-10-26T06:46:31.782354+0000","last_clean":"2023-10-26T06:46:31.782354+0000","last_became_active":"2023-10-26T06:45:05.912415+0000","last_became_peered":"2023-10-26T06:45:05.912415+0000","last_unstale":"2023-10-26T06:46:31.782354+0000","last_undegraded":"2023-10-26T06:46:31.782354+0000","last_fullsized":"2023-10-26T06:46:31.782354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:25.452671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.779371+0000","last_change":"2023-10-26T06:45:05.909106+0000","last_active":"2023-10-26T06:46:31.779371+0000","last_peered":"2023-10-26T06:46:31.779371+0000","last_clean":"2023-10-26T06:46:31.779371+0000","last_became_active":"2023-10-26T06:45:05.908751+0000","last_became_peered":"2023-10-26T06:45:05.908751+0000","last_unstale":"2023-10-26T06:46:31.779371+0000","last_undegraded":"2023-10-26T06:46:31.779371+0000","last_fullsized":"2023-10-26T06:46:31.779371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:17.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T06:46:31.761209+0000","last_change":"2023-10-26T06:45:05.898262+0000","last_active":"2023-10-26T06:46:31.761209+0000","last_peered":"2023-10-26T06:46:31.761209+0000","last_clean":"2023-10-26T06:46:31.761209+0000","last_became_active":"2023-10-26T06:45:05.897947+0000","last_became_peered":"2023-10-26T06:45:05.897947+0000","last_unstale":"2023-10-26T06:46:31.761209+0000","last_undegraded":"2023-10-26T06:46:31.761209+0000","last_fullsized":"2023-10-26T06:46:31.761209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T06:45:04.891732+0000","last_clean_scrub_stamp":"2023-10-26T06:45:04.891732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:39:45.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65317,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":214,"num_read_kb":172,"num_write":77,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":195951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":462460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.624},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.48599999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.80700000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.432}]},{"osd":2,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.372},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88100000000000001,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":1.52,"15min":1.52},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.012,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95199999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93700000000000006,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47599999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1040000000000001,"5min":1.47,"15min":1.47},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0549999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.74299999999999999}]},{"osd":10,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96199999999999997,"5min":1.327,"15min":1.327},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.036,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.57799999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1117664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44500000000000001,"15min":0.44400000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.34200000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.498},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82499999999999996,"5min":1.004,"15min":1.004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67800000000000005,"15min":0.68500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79800000000000004,"5min":0.874,"15min":0.874},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.627}]},{"osd":5,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41699999999999998}]},{"osd":8,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.752},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95599999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.046,"5min":1.133,"15min":1.133},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76500000000000001,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Oct 26 06:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.73399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0089999999999999,"5min":1.04,"15min":1.04},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.60599999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.995,"5min":1.097,"15min":1.097},"last":0.748}]}]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":463042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.51400000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.70099999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":0.69499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.1099999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.85599999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.6319999999999997,"5min":4.6319999999999997,"15min":4.6319999999999997},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.84499999999999997,"15min":0.86499999999999999},"min":{"1min":0.55200000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.93100000000000005,"5min":1.173,"15min":1.173},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.87},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97199999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.877},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.496},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.877,"15min":0.90700000000000003},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.64600000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93700000000000006,"15min":0.96499999999999997},"min":{"1min":0.59799999999999998,"5min":0.503,"15min":0.503},"max":{"1min":2.8210000000000002,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94999999999999996,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.94099999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.762,"5min":1.885,"15min":1.885},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.70099999999999996,"15min":0.77100000000000002},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80000000000000004,"5min":5.0609999999999999,"15min":5.0609999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.41599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.505}]},{"osd":1,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.628,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.50700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.44600000000000001},"min":{"1min":0.34200000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.48799999999999999,"15min":0.48499999999999999},"min":{"1min":0.40000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71799999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.68500000000000005,"15min":0.65900000000000003},"min":{"1min":0.49199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.54700000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.41199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85699999999999998,"5min":0.88,"15min":0.88},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0580000000000001,"5min":1.095,"15min":1.095},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.81799999999999995,"15min":0.88200000000000001},"min":{"1min":0.56499999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.038,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.84699999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.63800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.69499999999999995,"15min":0.68000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.74299999999999999,"15min":0.71899999999999997},"min":{"1min":0.68400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.022,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72899999999999998,"15min":0.69699999999999995},"min":{"1min":0.63400000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Oct 26 06:46:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.86199999999999999}]},{"osd":11,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.94699999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1053533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.46000000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.48799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41699999999999998,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60299999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78500000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.501},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69599999999999995,"5min":0.748,"15min":0.748},"last":0.503},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.73199999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55200000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92300000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80100000000000005}]},{"osd":8,"last update":"Thu Oct 26 06:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.877},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70799999999999996,"15min":0.69099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83999999999999997}]},{"osd":9,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.753},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94899999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.88200000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.82099999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.747,"15min":0.76100000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.877,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.76600000000000001,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.90800000000000003,"5min":1.048,"15min":1.048},"last":0.77500000000000002}]},{"osd":11,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.94699999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45100000000000001,"15min":0.44400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70699999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79100000000000004,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.68600000000000005,"15min":0.64300000000000002},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68600000000000005,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.89700000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":5.5549999999999997,"5min":5.5549999999999997,"15min":5.5549999999999997},"last":0.91500000000000004}]},{"osd":3,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71499999999999997,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.70799999999999996,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.84199999999999997,"5min":1.208,"15min":1.208},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94999999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91200000000000003,"5min":1.385,"15min":1.385},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.749,"15min":0.747},"min":{"1min":0.57499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90600000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Oct 26 06:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.82399999999999995,"15min":0.76600000000000001},"min":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":5.532,"5min":5.532,"15min":5.532},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":0.79900000000000004,"15min":0.73299999999999998},"min":{"1min":0.56499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":5.5810000000000004,"5min":5.5810000000000004,"15min":5.5810000000000004},"last":0.876}]},{"osd":8,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.59299999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95299999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Thu Oct 26 06:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92200000000000004,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.82099999999999995,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84899999999999998,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96799999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.93400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":521600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57099999999999995,"15min":0.54100000000000004},"min":{"1min":0.47299999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.92300000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44700000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0209999999999999,"5min":1.256,"15min":1.256},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.56499999999999995,"15min":0.52000000000000002},"min":{"1min":0.498,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.627},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.56899999999999995,"15min":0.52500000000000002},"min":{"1min":0.52600000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 06:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.49299999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61099999999999999,"15min":0.55700000000000005},"min":{"1min":0.47499999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Oct 26 06:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.45500000000000002,"15min":0.442},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.373},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.42799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39100000000000001}]},{"osd":5,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.63100000000000001,"15min":0.58399999999999996},"min":{"1min":0.496,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.65100000000000002,"15min":0.60599999999999998},"min":{"1min":0.56299999999999994,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.53700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.52400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96199999999999997,"5min":1.03,"15min":1.03},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Oct 26 06:45:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57999999999999996,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.71499999999999997,"15min":0.68300000000000005},"min":{"1min":0.497,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.72999999999999998,"15min":0.70399999999999996},"min":{"1min":0.58499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.996,"5min":1.298,"15min":1.298},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.64800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.70199999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.751},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.52300000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Oct 26 06:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.82799999999999996}]}]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 06:46:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65900000000000003,"15min":0.60099999999999998},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.58699999999999997,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.66500000000000004,"15min":0.61499999999999999},"min":{"1min":0.47399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5820000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.68200000000000005,"15min":0.626},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Oct 26 06:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70599999999999996,"15min":0.66300000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4450000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.68400000000000005,"15min":0.63100000000000001},"min":{"1min":0.53000000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.749,"15min":0.69399999999999995},"min":{"1min":0.53300000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":0.89900000000000002}]},{"osd":6,"last update":"Thu Oct 26 06:46:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.68700000000000006,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.4300000000000002,"5min":3.4300000000000002,"15min":3.4300000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.73599999999999999,"15min":0.68200000000000005},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0099999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Oct 26 06:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Oct 26 06:46:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.61599999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.054,"5min":0.93700000000000006,"15min":0.92900000000000005},"min":{"1min":0.66100000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.92300000000000004}]},{"osd":9,"last update":"Thu Oct 26 06:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.437,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.99199999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.626},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.87,"15min":0.89000000000000001},"min":{"1min":0.47299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.544,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.60599999999999998}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.64000000000000001,"5min":0.625,"15min":0.625},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Oct 26 06:46:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.089,"5min":0.92700000000000005,"15min":0.90000000000000002},"min":{"1min":0.64500000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.85999999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":526344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Oct 26 06:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.58099999999999996,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.56499999999999995,"15min":0.52300000000000002},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Oct 26 06:46:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.625},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89700000000000002,"5min":2.887,"15min":2.887},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.501,"15min":0.46899999999999997},"min":{"1min":0.40500000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.56399999999999995,"15min":0.51400000000000001},"min":{"1min":0.31900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.786,"5min":3.786,"15min":3.786},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Oct 26 06:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.623},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79000000000000004,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93400000000000005,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":0.88}]},{"osd":7,"last update":"Thu Oct 26 06:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.78700000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69099999999999995,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":0.997,"15min":0.997},"last":1.3480000000000001}]},{"osd":8,"last update":"Thu Oct 26 06:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89300000000000002,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.68100000000000005,"15min":0.66400000000000003},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86499999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.70199999999999996}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78500000000000003,"15min":0.77100000000000002},"min":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.86199999999999999,"5min":3.661,"15min":3.661},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.58099999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":3.431,"15min":3.431},"last":0.95099999999999996}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.746,"15min":0.72099999999999997},"min":{"1min":0.51700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.4649999999999999,"5min":3.4649999999999999,"15min":3.4649999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.77100000000000002,"15min":0.748},"min":{"1min":0.626,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.7040000000000002,"5min":3.7040000000000002,"15min":3.7040000000000002},"last":0.80300000000000005}]},{"osd":11,"last update":"Thu Oct 26 06:46:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78100000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":462479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44400000000000001,"15min":0.40899999999999997},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82899999999999996,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48399999999999999,"15min":0.45200000000000001},"min":{"1min":0.32300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.722,"15min":1.722},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63800000000000001,"15min":0.60599999999999998},"min":{"1min":0.48799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.875,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Oct 26 06:46:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.42899999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94599999999999995,"5min":1.427,"15min":1.427},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.66000000000000003},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85599999999999998,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Oct 26 06:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.65200000000000002},"min":{"1min":0.50900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":1.667,"15min":1.667},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.304,"15min":1.304},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.55600000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96699999999999997,"5min":1.456,"15min":1.456},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75,"15min":0.70199999999999996},"min":{"1min":0.623,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.115,"5min":1.637,"15min":1.637},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Oct 26 06:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.36799999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90800000000000003,"5min":1.589,"15min":1.589},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64700000000000002,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.91700000000000004,"5min":1.516,"15min":1.516},"last":0.81000000000000005}]},{"osd":7,"last update":"Thu Oct 26 06:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.52600000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Oct 26 06:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Oct 26 06:46:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96999999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.69899999999999995}]},{"osd":10,"last update":"Thu Oct 26 06:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.85299999999999998,"15min":0.86099999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.3200000000000001,"5min":1.395,"15min":1.395},"last":0.875},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.91100000000000003,"15min":0.92400000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0780000000000001,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.85399999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":462482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.53700000000000003,"15min":0.503},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.434}]},{"osd":1,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.35299999999999998},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.38},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.57899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Oct 26 06:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.66400000000000003,"5min":1.222,"15min":1.222},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Oct 26 06:46:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.58199999999999996},"min":{"1min":0.48199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.81899999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.42399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86199999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":1.28,"15min":1.28},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Oct 26 06:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84599999999999997,"5min":1.403,"15min":1.403},"last":0.57499999999999996}]},{"osd":8,"last update":"Thu Oct 26 06:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.63100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88200000000000001,"5min":0.998,"15min":0.998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.623,"5min":0.433,"15min":0.433},"max":{"1min":0.82099999999999995,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Oct 26 06:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.72699999999999998,"15min":0.71299999999999997},"min":{"1min":0.56999999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.748},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.024,"5min":1.175,"15min":1.175},"last":0.77000000000000002}]},{"osd":10,"last update":"Thu Oct 26 06:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94099999999999995,"5min":1.141,"15min":1.141},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70499999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91700000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Oct 26 06:46:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.78900000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1053133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.438,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":0.871,"15min":0.871},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59699999999999998,"15min":0.57399999999999995},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.85799999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.313,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73599999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.46200000000000002,"15min":0.439},"min":{"1min":0.37,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.496}]},{"osd":3,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.748,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.48599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.75}]},{"osd":4,"last update":"Thu Oct 26 06:46:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63500000000000001,"15min":0.60999999999999999},"min":{"1min":0.54400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Oct 26 06:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68100000000000005,"15min":0.64900000000000002},"min":{"1min":0.59699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70199999999999996,"15min":0.66400000000000003},"min":{"1min":0.63100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Oct 26 06:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.52600000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84399999999999997,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.69999999999999996,"15min":0.68200000000000005},"min":{"1min":0.56200000000000006,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.874}]},{"osd":8,"last update":"Thu Oct 26 06:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66500000000000004,"15min":0.63900000000000001},"min":{"1min":0.42099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64500000000000002}]},{"osd":9,"last update":"Thu Oct 26 06:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.54000000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.66000000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.83999999999999997}]},{"osd":10,"last update":"Thu Oct 26 06:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.66500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":64553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-26T06:46:48.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:49.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 -- 172.21.15.89:0/1151566288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec00100e60 msgr2=0x7fec0010d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 --2- 172.21.15.89:0/1151566288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec00100e60 0x7fec0010d320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7febf800b0d0 tx=0x7febf800b3e0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 -- 172.21.15.89:0/1151566288 shutdown_connections 2023-10-26T06:46:49.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 --2- 172.21.15.89:0/1151566288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec00100e60 0x7fec0010d320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 --2- 172.21.15.89:0/1151566288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec00106560 0x7fec00106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 --2- 172.21.15.89:0/1151566288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec001004a0 0x7fec00100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 -- 172.21.15.89:0/1151566288 >> 172.21.15.89:0/1151566288 conn(0x7fec000fbfb0 msgr2=0x7fec000fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:49.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 -- 172.21.15.89:0/1151566288 shutdown_connections 2023-10-26T06:46:49.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.125+0000 7fec0559d700 1 -- 172.21.15.89:0/1151566288 wait complete. 2023-10-26T06:46:49.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.126+0000 7fec0559d700 1 Processor -- start 2023-10-26T06:46:49.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7fec0559d700 1 -- start start 2023-10-26T06:46:49.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7fec0559d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec001004a0 0x7fec0019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7fec0559d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec00100e60 0x7fec0019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7fec0559d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec00106560 0x7fec00196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7fec0559d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00196ee0 con 0x7fec001004a0 2023-10-26T06:46:49.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7fec0559d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00197050 con 0x7fec00106560 2023-10-26T06:46:49.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7fec0559d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00197210 con 0x7fec00100e60 2023-10-26T06:46:49.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7febfffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec001004a0 0x7fec0019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7febfffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec001004a0 0x7fec0019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54176/0 (socket says 172.21.15.89:54176) 2023-10-26T06:46:49.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.127+0000 7febfffff700 1 -- 172.21.15.89:0/3354300279 learned_addr learned my addr 172.21.15.89:0/3354300279 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:49.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7fec04d9c700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec00106560 0x7fec00196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febff7fe700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec00100e60 0x7fec0019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfffff700 1 -- 172.21.15.89:0/3354300279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec00100e60 msgr2=0x7fec0019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfffff700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec00100e60 0x7fec0019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfffff700 1 -- 172.21.15.89:0/3354300279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec00106560 msgr2=0x7fec00196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfffff700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec00106560 0x7fec00196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfffff700 1 -- 172.21.15.89:0/3354300279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf8009d00 con 0x7fec001004a0 2023-10-26T06:46:49.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febff7fe700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec00100e60 0x7fec0019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:49.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7fec04d9c700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec00106560 0x7fec00196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:49.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfffff700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec001004a0 0x7fec0019c6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7febf000e9a0 tx=0x7febf000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:49.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfd7fa700 1 -- 172.21.15.89:0/3354300279 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0009790 con 0x7fec001004a0 2023-10-26T06:46:49.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec001974f0 con 0x7fec001004a0 2023-10-26T06:46:49.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfd7fa700 1 -- 172.21.15.89:0/3354300279 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf000cbd0 con 0x7fec001004a0 2023-10-26T06:46:49.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7febfd7fa700 1 -- 172.21.15.89:0/3354300279 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0018680 con 0x7fec001004a0 2023-10-26T06:46:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.128+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec001a36e0 con 0x7fec001004a0 2023-10-26T06:46:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.129+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec00108a00 con 0x7fec001004a0 2023-10-26T06:46:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.133+0000 7febfd7fa700 1 -- 172.21.15.89:0/3354300279 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7febf000cd40 con 0x7fec001004a0 2023-10-26T06:46:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.134+0000 7febfd7fa700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7febe80a1550 0x7febe80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.134+0000 7febfd7fa700 1 -- 172.21.15.89:0/3354300279 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7febf0014070 con 0x7fec001004a0 2023-10-26T06:46:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.134+0000 7febff7fe700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7febe80a1550 0x7febe80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.135+0000 7febff7fe700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7febe80a1550 0x7febe80a3a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec00197c40 tx=0x7febf4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.136+0000 7febfd7fa700 1 -- 172.21.15.89:0/3354300279 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7febf0096b50 con 0x7fec001004a0 2023-10-26T06:46:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.343+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec00101bc0 con 0x7fec001004a0 2023-10-26T06:46:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.345+0000 7febfd7fa700 1 -- 172.21.15.89:0/3354300279 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7febf009a170 con 0x7fec001004a0 2023-10-26T06:46:49.347 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:49.348 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7febe80a1550 msgr2=0x7febe80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7febe80a1550 0x7febe80a3a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec00197c40 tx=0x7febf4009260 comp rx=0 tx=0).stop 2023-10-26T06:46:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec001004a0 msgr2=0x7fec0019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec001004a0 0x7fec0019c6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7febf000e9a0 tx=0x7febf000ed60 comp rx=0 tx=0).stop 2023-10-26T06:46:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 shutdown_connections 2023-10-26T06:46:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7febe80a1550 0x7febe80a3a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec001004a0 0x7fec0019c6f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec00106560 0x7fec00196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 --2- 172.21.15.89:0/3354300279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec00100e60 0x7fec0019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.347+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 >> 172.21.15.89:0/3354300279 conn(0x7fec000fbfb0 msgr2=0x7fec000fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:49.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.348+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 shutdown_connections 2023-10-26T06:46:49.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.348+0000 7fec0559d700 1 -- 172.21.15.89:0/3354300279 wait complete. 2023-10-26T06:46:49.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:46:49.370 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:46:49.429 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:49 smithi089 ceph-mon[95031]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:49.429 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:49 smithi089 ceph-mon[95031]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:49.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:49 smithi135 ceph-mon[131403]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:49.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:49 smithi135 ceph-mon[131403]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.568+0000 7ff97ffff700 1 -- 172.21.15.89:0/2167406390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980106660 msgr2=0x7ff980106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.568+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2167406390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980106660 0x7ff980106a40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff970009990 tx=0x7ff970009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.568+0000 7ff97ffff700 1 -- 172.21.15.89:0/2167406390 shutdown_connections 2023-10-26T06:46:49.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.568+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2167406390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff980100f80 0x7ff98010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.568+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2167406390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980106660 0x7ff980106a40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.568+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2167406390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9801005c0 0x7ff980100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.568+0000 7ff97ffff700 1 -- 172.21.15.89:0/2167406390 >> 172.21.15.89:0/2167406390 conn(0x7ff9800fc0f0 msgr2=0x7ff9800fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:49.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.569+0000 7ff97ffff700 1 -- 172.21.15.89:0/2167406390 shutdown_connections 2023-10-26T06:46:49.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.569+0000 7ff97ffff700 1 -- 172.21.15.89:0/2167406390 wait complete. 2023-10-26T06:46:49.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.569+0000 7ff97ffff700 1 Processor -- start 2023-10-26T06:46:49.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97ffff700 1 -- start start 2023-10-26T06:46:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9801005c0 0x7ff980196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff980100f80 0x7ff980196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff980106660 0x7ff98019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97ffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019b910 con 0x7ff980106660 2023-10-26T06:46:49.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97ffff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019ba80 con 0x7ff9801005c0 2023-10-26T06:46:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97ffff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019bc40 con 0x7ff980100f80 2023-10-26T06:46:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff980106660 0x7ff98019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9801005c0 0x7ff980196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff980106660 0x7ff98019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54202/0 (socket says 172.21.15.89:54202) 2023-10-26T06:46:49.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97f7fe700 1 -- 172.21.15.89:0/2431624092 learned_addr learned my addr 172.21.15.89:0/2431624092 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.570+0000 7ff97e7fc700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff980100f80 0x7ff980196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97f7fe700 1 -- 172.21.15.89:0/2431624092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff980100f80 msgr2=0x7ff980196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97f7fe700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff980100f80 0x7ff980196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97f7fe700 1 -- 172.21.15.89:0/2431624092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9801005c0 msgr2=0x7ff980196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97f7fe700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9801005c0 0x7ff980196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97f7fe700 1 -- 172.21.15.89:0/2431624092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff978009c40 con 0x7ff980106660 2023-10-26T06:46:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97effd700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9801005c0 0x7ff980196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97f7fe700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff980106660 0x7ff98019b340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c370 tx=0x7ff97800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff9849fd700 1 -- 172.21.15.89:0/2431624092 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978017070 con 0x7ff980106660 2023-10-26T06:46:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9700095f0 con 0x7ff980106660 2023-10-26T06:46:49.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff9849fd700 1 -- 172.21.15.89:0/2431624092 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97800f040 con 0x7ff980106660 2023-10-26T06:46:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff9849fd700 1 -- 172.21.15.89:0/2431624092 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800e050 con 0x7ff980106660 2023-10-26T06:46:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.571+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9801a35f0 con 0x7ff980106660 2023-10-26T06:46:49.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.572+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980108b50 con 0x7ff980106660 2023-10-26T06:46:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.574+0000 7ff9849fd700 1 -- 172.21.15.89:0/2431624092 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7ff97800cb60 con 0x7ff980106660 2023-10-26T06:46:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.576+0000 7ff9849fd700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9680a14d0 0x7ff9680a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.576+0000 7ff9849fd700 1 -- 172.21.15.89:0/2431624092 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff9780cc7d0 con 0x7ff980106660 2023-10-26T06:46:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.576+0000 7ff97effd700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9680a14d0 0x7ff9680a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.576+0000 7ff97effd700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9680a14d0 0x7ff9680a3990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff97000b2e0 tx=0x7ff970005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.579+0000 7ff9849fd700 1 -- 172.21.15.89:0/2431624092 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff978095d50 con 0x7ff980106660 2023-10-26T06:46:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:49 smithi192 ceph-mon[131661]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:49.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:49 smithi192 ceph-mon[131661]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:49.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.780+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9800673e0 con 0x7ff980106660 2023-10-26T06:46:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.780+0000 7ff9849fd700 1 -- 172.21.15.89:0/2431624092 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff978099370 con 0x7ff980106660 2023-10-26T06:46:49.783 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:46:49.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9680a14d0 msgr2=0x7ff9680a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9680a14d0 0x7ff9680a3990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff97000b2e0 tx=0x7ff970005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff980106660 msgr2=0x7ff98019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:49.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff980106660 0x7ff98019b340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c370 tx=0x7ff97800c680 comp rx=0 tx=0).stop 2023-10-26T06:46:49.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 shutdown_connections 2023-10-26T06:46:49.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff9680a14d0 0x7ff9680a3990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff980106660 0x7ff98019b340 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9801005c0 0x7ff980196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 --2- 172.21.15.89:0/2431624092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff980100f80 0x7ff980196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:49.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 >> 172.21.15.89:0/2431624092 conn(0x7ff9800fc0f0 msgr2=0x7ff9800fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:49.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 shutdown_connections 2023-10-26T06:46:49.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:49.783+0000 7ff97ffff700 1 -- 172.21.15.89:0/2431624092 wait complete. 2023-10-26T06:46:49.804 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:46:50.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 -- 172.21.15.89:0/4215557362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d8106730 msgr2=0x7fb1d8106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/4215557362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d8106730 0x7fb1d8106b10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009990 tx=0x7fb1c8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 -- 172.21.15.89:0/4215557362 shutdown_connections 2023-10-26T06:46:50.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/4215557362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d8106730 0x7fb1d8106b10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/4215557362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d81010e0 0x7fb1d810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/4215557362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1d8100720 0x7fb1d8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 -- 172.21.15.89:0/4215557362 >> 172.21.15.89:0/4215557362 conn(0x7fb1d80fc1b0 msgr2=0x7fb1d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:50.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 -- 172.21.15.89:0/4215557362 shutdown_connections 2023-10-26T06:46:50.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.003+0000 7fb1de3ff700 1 -- 172.21.15.89:0/4215557362 wait complete. 2023-10-26T06:46:50.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 Processor -- start 2023-10-26T06:46:50.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 -- start start 2023-10-26T06:46:50.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1d8100720 0x7fb1d819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d81010e0 0x7fb1d819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d8106730 0x7fb1d81969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d8197050 con 0x7fb1d81010e0 2023-10-26T06:46:50.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81971c0 con 0x7fb1d8100720 2023-10-26T06:46:50.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.004+0000 7fb1de3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d8197380 con 0x7fb1d8106730 2023-10-26T06:46:50.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d81010e0 0x7fb1d819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d81010e0 0x7fb1d819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58910/0 (socket says 172.21.15.89:58910) 2023-10-26T06:46:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 -- 172.21.15.89:0/980148198 learned_addr learned my addr 172.21.15.89:0/980148198 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d7fff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1d8100720 0x7fb1d819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1dc99c700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d8106730 0x7fb1d81969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 -- 172.21.15.89:0/980148198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d8106730 msgr2=0x7fb1d81969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d8106730 0x7fb1d81969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 -- 172.21.15.89:0/980148198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1d8100720 msgr2=0x7fb1d819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1d8100720 0x7fb1d819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 -- 172.21.15.89:0/980148198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c80095f0 con 0x7fb1d81010e0 2023-10-26T06:46:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d77fe700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d81010e0 0x7fb1d819cdb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00e9a0 tx=0x7fb1cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d57fa700 1 -- 172.21.15.89:0/980148198 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc009790 con 0x7fb1d81010e0 2023-10-26T06:46:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d57fa700 1 -- 172.21.15.89:0/980148198 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1cc00cbd0 con 0x7fb1d81010e0 2023-10-26T06:46:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.005+0000 7fb1d57fa700 1 -- 172.21.15.89:0/980148198 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc018680 con 0x7fb1d81010e0 2023-10-26T06:46:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.006+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d8197660 con 0x7fb1d81010e0 2023-10-26T06:46:50.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.006+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d81a3860 con 0x7fb1d81010e0 2023-10-26T06:46:50.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.007+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d8108d20 con 0x7fb1d81010e0 2023-10-26T06:46:50.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.008+0000 7fb1d57fa700 1 -- 172.21.15.89:0/980148198 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb1cc010430 con 0x7fb1d81010e0 2023-10-26T06:46:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.010+0000 7fb1d57fa700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb1c00a1550 0x7fb1c00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.011+0000 7fb1d57fa700 1 -- 172.21.15.89:0/980148198 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb1cc014070 con 0x7fb1d81010e0 2023-10-26T06:46:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.011+0000 7fb1d7fff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb1c00a1550 0x7fb1c00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.011+0000 7fb1d7fff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb1c00a1550 0x7fb1c00a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800b2e0 tx=0x7fb1c8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:50.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.014+0000 7fb1d57fa700 1 -- 172.21.15.89:0/980148198 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb1cc052100 con 0x7fb1d81010e0 2023-10-26T06:46:50.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.224+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1d8197e60 con 0x7fb1d81010e0 2023-10-26T06:46:50.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.224+0000 7fb1d57fa700 1 -- 172.21.15.89:0/980148198 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1cc0965e0 con 0x7fb1d81010e0 2023-10-26T06:46:50.227 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:46:50.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.226+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb1c00a1550 msgr2=0x7fb1c00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb1c00a1550 0x7fb1c00a3a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800b2e0 tx=0x7fb1c8005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d81010e0 msgr2=0x7fb1d819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d81010e0 0x7fb1d819cdb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00e9a0 tx=0x7fb1cc00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 shutdown_connections 2023-10-26T06:46:50.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb1c00a1550 0x7fb1c00a3a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d8106730 0x7fb1d81969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1d8100720 0x7fb1d819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 --2- 172.21.15.89:0/980148198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1d81010e0 0x7fb1d819cdb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 >> 172.21.15.89:0/980148198 conn(0x7fb1d80fc1b0 msgr2=0x7fb1d80fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:50.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 shutdown_connections 2023-10-26T06:46:50.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.227+0000 7fb1de3ff700 1 -- 172.21.15.89:0/980148198 wait complete. 2023-10-26T06:46:50.247 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:46:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 -- 172.21.15.89:0/1749220856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 msgr2=0x7fcb1c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 --2- 172.21.15.89:0/1749220856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c10d5f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b0a0 tx=0x7fcb1400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 -- 172.21.15.89:0/1749220856 shutdown_connections 2023-10-26T06:46:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 --2- 172.21.15.89:0/1749220856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c10d5f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 --2- 172.21.15.89:0/1749220856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb1c106730 0x7fcb1c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 --2- 172.21.15.89:0/1749220856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb1c100720 0x7fcb1c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 -- 172.21.15.89:0/1749220856 >> 172.21.15.89:0/1749220856 conn(0x7fcb1c0fc1b0 msgr2=0x7fcb1c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 -- 172.21.15.89:0/1749220856 shutdown_connections 2023-10-26T06:46:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.446+0000 7fcb21dff700 1 -- 172.21.15.89:0/1749220856 wait complete. 2023-10-26T06:46:50.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 Processor -- start 2023-10-26T06:46:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 -- start start 2023-10-26T06:46:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb1c100720 0x7fcb1c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb1c106730 0x7fcb1c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c197080 con 0x7fcb1c1010e0 2023-10-26T06:46:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c1971f0 con 0x7fcb1c100720 2023-10-26T06:46:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.448+0000 7fcb21dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c1973b0 con 0x7fcb1c106730 2023-10-26T06:46:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb1b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb1c100720 0x7fcb1c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58928/0 (socket says 172.21.15.89:58928) 2023-10-26T06:46:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 -- 172.21.15.89:0/2547496022 learned_addr learned my addr 172.21.15.89:0/2547496022 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb1bfff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb1c106730 0x7fcb1c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 -- 172.21.15.89:0/2547496022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb1c106730 msgr2=0x7fcb1c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb1c106730 0x7fcb1c1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 -- 172.21.15.89:0/2547496022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb1c100720 msgr2=0x7fcb1c19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb1c100720 0x7fcb1c19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 -- 172.21.15.89:0/2547496022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14009d00 con 0x7fcb1c1010e0 2023-10-26T06:46:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb13fff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c19cd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e9a0 tx=0x7fcb0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb197fa700 1 -- 172.21.15.89:0/2547496022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04009790 con 0x7fcb1c1010e0 2023-10-26T06:46:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.449+0000 7fcb197fa700 1 -- 172.21.15.89:0/2547496022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb04010430 con 0x7fcb1c1010e0 2023-10-26T06:46:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.450+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c197690 con 0x7fcb1c1010e0 2023-10-26T06:46:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.450+0000 7fcb197fa700 1 -- 172.21.15.89:0/2547496022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04018680 con 0x7fcb1c1010e0 2023-10-26T06:46:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.450+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c1a37e0 con 0x7fcb1c1010e0 2023-10-26T06:46:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.451+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c108d40 con 0x7fcb1c1010e0 2023-10-26T06:46:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.453+0000 7fcb197fa700 1 -- 172.21.15.89:0/2547496022 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fcb040105a0 con 0x7fcb1c1010e0 2023-10-26T06:46:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.454+0000 7fcb197fa700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcb080a1550 0x7fcb080a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.454+0000 7fcb197fa700 1 -- 172.21.15.89:0/2547496022 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fcb04014070 con 0x7fcb1c1010e0 2023-10-26T06:46:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.454+0000 7fcb1b7fe700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcb080a1550 0x7fcb080a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.455+0000 7fcb1b7fe700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcb080a1550 0x7fcb080a3a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009be0 tx=0x7fcb0c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:50.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.458+0000 7fcb197fa700 1 -- 172.21.15.89:0/2547496022 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcb040964e0 con 0x7fcb1c1010e0 2023-10-26T06:46:50.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:50 smithi135 ceph-mon[131403]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:50.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:50 smithi135 ceph-mon[131403]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:50.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:50 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3354300279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:50.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:50 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2431624092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:50.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:50 smithi089 ceph-mon[95031]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:50.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:50 smithi089 ceph-mon[95031]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:50.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:50 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3354300279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:50.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:50 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2431624092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:50.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:50 smithi192 ceph-mon[131661]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:50.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:50 smithi192 ceph-mon[131661]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-26T06:46:50.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:50 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3354300279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:50.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:50 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2431624092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:46:50.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.660+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb1c197d90 con 0x7fcb1c1010e0 2023-10-26T06:46:50.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.660+0000 7fcb197fa700 1 -- 172.21.15.89:0/2547496022 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb04099b00 con 0x7fcb1c1010e0 2023-10-26T06:46:50.663 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:46:50.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcb080a1550 msgr2=0x7fcb080a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcb080a1550 0x7fcb080a3a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009be0 tx=0x7fcb0c009260 comp rx=0 tx=0).stop 2023-10-26T06:46:50.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 msgr2=0x7fcb1c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c19cd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400e9a0 tx=0x7fcb0400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 shutdown_connections 2023-10-26T06:46:50.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcb080a1550 0x7fcb080a3a10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb1c106730 0x7fcb1c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb1c100720 0x7fcb1c19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 --2- 172.21.15.89:0/2547496022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb1c1010e0 0x7fcb1c19cd50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 >> 172.21.15.89:0/2547496022 conn(0x7fcb1c0fc1b0 msgr2=0x7fcb1c0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 shutdown_connections 2023-10-26T06:46:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.663+0000 7fcb21dff700 1 -- 172.21.15.89:0/2547496022 wait complete. 2023-10-26T06:46:50.684 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:46:50.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.880+0000 7faf2fbff700 1 -- 172.21.15.89:0/2237459217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 msgr2=0x7faf28100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.880+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2237459217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf28100b20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf18009990 tx=0x7faf18009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.881+0000 7faf2fbff700 1 -- 172.21.15.89:0/2237459217 shutdown_connections 2023-10-26T06:46:50.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.881+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2237459217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf28101060 0x7faf2810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.881+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2237459217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf281066b0 0x7faf28106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.881+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2237459217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf28100b20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.881+0000 7faf2fbff700 1 -- 172.21.15.89:0/2237459217 >> 172.21.15.89:0/2237459217 conn(0x7faf280fc170 msgr2=0x7faf280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:50.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.881+0000 7faf2fbff700 1 -- 172.21.15.89:0/2237459217 shutdown_connections 2023-10-26T06:46:50.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.881+0000 7faf2fbff700 1 -- 172.21.15.89:0/2237459217 wait complete. 2023-10-26T06:46:50.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 Processor -- start 2023-10-26T06:46:50.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 -- start start 2023-10-26T06:46:50.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf2819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf28101060 0x7faf2819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf281066b0 0x7faf28198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf28199390 con 0x7faf281006a0 2023-10-26T06:46:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf28199500 con 0x7faf281066b0 2023-10-26T06:46:50.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2fbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf281996c0 con 0x7faf28101060 2023-10-26T06:46:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2d99b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf2819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2d99b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf2819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58954/0 (socket says 172.21.15.89:58954) 2023-10-26T06:46:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.882+0000 7faf2d99b700 1 -- 172.21.15.89:0/2776086054 learned_addr learned my addr 172.21.15.89:0/2776086054 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2d19a700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf28101060 0x7faf2819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2e19c700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf281066b0 0x7faf28198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2d99b700 1 -- 172.21.15.89:0/2776086054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf28101060 msgr2=0x7faf2819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2d99b700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf28101060 0x7faf2819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2d99b700 1 -- 172.21.15.89:0/2776086054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf281066b0 msgr2=0x7faf28198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2d99b700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf281066b0 0x7faf28198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2d99b700 1 -- 172.21.15.89:0/2776086054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24009520 con 0x7faf281006a0 2023-10-26T06:46:50.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2d99b700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf2819ead0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf2400eb30 tx=0x7faf2400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf1effd700 1 -- 172.21.15.89:0/2776086054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24009880 con 0x7faf281006a0 2023-10-26T06:46:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf1effd700 1 -- 172.21.15.89:0/2776086054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf240044e0 con 0x7faf281006a0 2023-10-26T06:46:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf180095f0 con 0x7faf281006a0 2023-10-26T06:46:50.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.883+0000 7faf1effd700 1 -- 172.21.15.89:0/2776086054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24003d80 con 0x7faf281006a0 2023-10-26T06:46:50.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.884+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf281a5a50 con 0x7faf281006a0 2023-10-26T06:46:50.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.885+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf28101de0 con 0x7faf281006a0 2023-10-26T06:46:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.887+0000 7faf1effd700 1 -- 172.21.15.89:0/2776086054 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faf24003f20 con 0x7faf281006a0 2023-10-26T06:46:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.888+0000 7faf1effd700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf140a1550 0x7faf140a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.888+0000 7faf1effd700 1 -- 172.21.15.89:0/2776086054 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7faf24014070 con 0x7faf281006a0 2023-10-26T06:46:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.888+0000 7faf2d19a700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf140a1550 0x7faf140a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.889+0000 7faf2d19a700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf140a1550 0x7faf140a3a10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf2819a2e0 tx=0x7faf18005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:50.892+0000 7faf1effd700 1 -- 172.21.15.89:0/2776086054 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faf24096800 con 0x7faf281006a0 2023-10-26T06:46:51.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.138+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf28067530 con 0x7faf281006a0 2023-10-26T06:46:51.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.138+0000 7faf1effd700 1 -- 172.21.15.89:0/2776086054 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7faf24099e20 con 0x7faf281006a0 2023-10-26T06:46:51.141 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:51.141 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:46:51.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf140a1550 msgr2=0x7faf140a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf140a1550 0x7faf140a3a10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf2819a2e0 tx=0x7faf18005fd0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 msgr2=0x7faf2819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf2819ead0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf2400eb30 tx=0x7faf2400ee40 comp rx=0 tx=0).stop 2023-10-26T06:46:51.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 shutdown_connections 2023-10-26T06:46:51.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7faf140a1550 0x7faf140a3a10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf281006a0 0x7faf2819ead0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf281066b0 0x7faf28198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 --2- 172.21.15.89:0/2776086054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf28101060 0x7faf2819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 >> 172.21.15.89:0/2776086054 conn(0x7faf280fc170 msgr2=0x7faf280fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 shutdown_connections 2023-10-26T06:46:51.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.141+0000 7faf2fbff700 1 -- 172.21.15.89:0/2776086054 wait complete. 2023-10-26T06:46:51.146 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:46:51.162 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:46:51.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 -- 172.21.15.89:0/1353842556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0072c20 msgr2=0x7fc8b0073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/1353842556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0072c20 0x7fc8b0073000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009990 tx=0x7fc8a4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 -- 172.21.15.89:0/1353842556 shutdown_connections 2023-10-26T06:46:51.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/1353842556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0072c20 0x7fc8b0073000 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/1353842556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b010cee0 0x7fc8b010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/1353842556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b0073540 0x7fc8b010c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 -- 172.21.15.89:0/1353842556 >> 172.21.15.89:0/1353842556 conn(0x7fc8b00fc170 msgr2=0x7fc8b00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 -- 172.21.15.89:0/1353842556 shutdown_connections 2023-10-26T06:46:51.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.360+0000 7fc8b6fff700 1 -- 172.21.15.89:0/1353842556 wait complete. 2023-10-26T06:46:51.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.361+0000 7fc8b6fff700 1 Processor -- start 2023-10-26T06:46:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.361+0000 7fc8b6fff700 1 -- start start 2023-10-26T06:46:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.361+0000 7fc8b6fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b0072c20 0x7fc8b0198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b6fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0073540 0x7fc8b01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b6fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b010cee0 0x7fc8b019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b6fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019dcb0 con 0x7fc8b0073540 2023-10-26T06:46:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b6fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019de20 con 0x7fc8b0072c20 2023-10-26T06:46:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b6fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019dfe0 con 0x7fc8b010cee0 2023-10-26T06:46:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0073540 0x7fc8b01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0073540 0x7fc8b01992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58968/0 (socket says 172.21.15.89:58968) 2023-10-26T06:46:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 -- 172.21.15.89:0/416816663 learned_addr learned my addr 172.21.15.89:0/416816663 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b559c700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b010cee0 0x7fc8b019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b4d9b700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b0072c20 0x7fc8b0198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 -- 172.21.15.89:0/416816663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b010cee0 msgr2=0x7fc8b019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b010cee0 0x7fc8b019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 -- 172.21.15.89:0/416816663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b0072c20 msgr2=0x7fc8b0198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b0072c20 0x7fc8b0198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 -- 172.21.15.89:0/416816663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40095f0 con 0x7fc8b0073540 2023-10-26T06:46:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8b4d9b700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b0072c20 0x7fc8b0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.362+0000 7fc8affff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0073540 0x7fc8b01992d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00e9a0 tx=0x7fc89c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.363+0000 7fc8adffb700 1 -- 172.21.15.89:0/416816663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c009790 con 0x7fc8b0073540 2023-10-26T06:46:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.363+0000 7fc8adffb700 1 -- 172.21.15.89:0/416816663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89c00cbd0 con 0x7fc8b0073540 2023-10-26T06:46:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.363+0000 7fc8adffb700 1 -- 172.21.15.89:0/416816663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c018680 con 0x7fc8b0073540 2023-10-26T06:46:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.363+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b019e2c0 con 0x7fc8b0073540 2023-10-26T06:46:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.363+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b01a5b00 con 0x7fc8b0073540 2023-10-26T06:46:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.364+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b01080f0 con 0x7fc8b0073540 2023-10-26T06:46:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.366+0000 7fc8adffb700 1 -- 172.21.15.89:0/416816663 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fc89c00cd40 con 0x7fc8b0073540 2023-10-26T06:46:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.367+0000 7fc8adffb700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc8a00a14d0 0x7fc8a00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.367+0000 7fc8adffb700 1 -- 172.21.15.89:0/416816663 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fc89c014070 con 0x7fc8b0073540 2023-10-26T06:46:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.367+0000 7fc8b4d9b700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc8a00a14d0 0x7fc8a00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.368+0000 7fc8b4d9b700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc8a00a14d0 0x7fc8a00a3990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400b2e0 tx=0x7fc8a4005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.371+0000 7fc8adffb700 1 -- 172.21.15.89:0/416816663 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc89c052100 con 0x7fc8b0073540 2023-10-26T06:46:51.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:51 smithi135 ceph-mon[131403]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:51.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:51 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/980148198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:51.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:51 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2547496022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:51.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:51 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2776086054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:51 smithi089 ceph-mon[95031]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:51 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/980148198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:51 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2547496022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:51 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2776086054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:51.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.578+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8b019a090 con 0x7fc8b0073540 2023-10-26T06:46:51.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.579+0000 7fc8adffb700 1 -- 172.21.15.89:0/416816663 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22888 (secure 0 0 0) 0x7fc89c096660 con 0x7fc8b0073540 2023-10-26T06:46:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:46:51.582 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":86,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:46:03.220293+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:46:51.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc8a00a14d0 msgr2=0x7fc8a00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc8a00a14d0 0x7fc8a00a3990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400b2e0 tx=0x7fc8a4005fb0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0073540 msgr2=0x7fc8b01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0073540 0x7fc8b01992d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00e9a0 tx=0x7fc89c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 shutdown_connections 2023-10-26T06:46:51.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc8a00a14d0 0x7fc8a00a3990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b010cee0 0x7fc8b019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8b0072c20 0x7fc8b0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 --2- 172.21.15.89:0/416816663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8b0073540 0x7fc8b01992d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 >> 172.21.15.89:0/416816663 conn(0x7fc8b00fc170 msgr2=0x7fc8b00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 shutdown_connections 2023-10-26T06:46:51.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.582+0000 7fc8b6fff700 1 -- 172.21.15.89:0/416816663 wait complete. 2023-10-26T06:46:51.603 DEBUG:tasks.fwd_scrub:Yielding 2023-10-26T06:46:51.604 INFO:teuthology.run_tasks:Running task exec... 2023-10-26T06:46:51.615 INFO:teuthology.task.exec:Executing custom commands... 2023-10-26T06:46:51.616 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T06:46:51.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-10-26T06:46:51.617 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-26T06:46:51.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:46:51.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:46:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:51 smithi192 ceph-mon[131661]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:51 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/980148198' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:46:51.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:51 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2547496022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:46:51.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:51 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2776086054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:46:51.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 -- 172.21.15.89:0/4165301486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01066d0 msgr2=0x7f91a0106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 --2- 172.21.15.89:0/4165301486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01066d0 0x7f91a0106ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f919c009990 tx=0x7f919c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 -- 172.21.15.89:0/4165301486 shutdown_connections 2023-10-26T06:46:51.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 --2- 172.21.15.89:0/4165301486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01066d0 0x7f91a0106ab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 --2- 172.21.15.89:0/4165301486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 0x7f91a010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 --2- 172.21.15.89:0/4165301486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91a01006c0 0x7f91a0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 -- 172.21.15.89:0/4165301486 >> 172.21.15.89:0/4165301486 conn(0x7f91a00fc190 msgr2=0x7f91a00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 -- 172.21.15.89:0/4165301486 shutdown_connections 2023-10-26T06:46:51.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.817+0000 7f91a7bff700 1 -- 172.21.15.89:0/4165301486 wait complete. 2023-10-26T06:46:51.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.818+0000 7f91a7bff700 1 Processor -- start 2023-10-26T06:46:51.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.818+0000 7f91a7bff700 1 -- start start 2023-10-26T06:46:51.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a7bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01006c0 0x7f91a019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a7bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 0x7f91a019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a7bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91a01066d0 0x7f91a0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a7bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a0199190 con 0x7f91a01006c0 2023-10-26T06:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a7bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a0199300 con 0x7f91a01066d0 2023-10-26T06:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a7bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a01994c0 con 0x7f91a0101080 2023-10-26T06:46:51.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a519a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 0x7f91a019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a599b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01006c0 0x7f91a019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a519a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 0x7f91a019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:53854/0 (socket says 172.21.15.89:53854) 2023-10-26T06:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a519a700 1 -- 172.21.15.89:0/598708846 learned_addr learned my addr 172.21.15.89:0/598708846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a619c700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91a01066d0 0x7f91a0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a599b700 1 -- 172.21.15.89:0/598708846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 msgr2=0x7f91a019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a599b700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 0x7f91a019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a599b700 1 -- 172.21.15.89:0/598708846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91a01066d0 msgr2=0x7f91a0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a599b700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91a01066d0 0x7f91a0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a599b700 1 -- 172.21.15.89:0/598708846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c0095f0 con 0x7f91a01006c0 2023-10-26T06:46:51.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.819+0000 7f91a519a700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 0x7f91a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.820+0000 7f91a599b700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01006c0 0x7f91a019ea00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.820+0000 7f9196ffd700 1 -- 172.21.15.89:0/598708846 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c01d070 con 0x7f91a01006c0 2023-10-26T06:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.820+0000 7f9196ffd700 1 -- 172.21.15.89:0/598708846 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919c021410 con 0x7f91a01006c0 2023-10-26T06:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.820+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a0199740 con 0x7f91a01006c0 2023-10-26T06:46:51.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.820+0000 7f9196ffd700 1 -- 172.21.15.89:0/598708846 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c017650 con 0x7f91a01006c0 2023-10-26T06:46:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.820+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a01a59f0 con 0x7f91a01006c0 2023-10-26T06:46:51.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.821+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a0101e00 con 0x7f91a01006c0 2023-10-26T06:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.824+0000 7f9196ffd700 1 -- 172.21.15.89:0/598708846 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f919c021580 con 0x7f91a01006c0 2023-10-26T06:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.826+0000 7f9196ffd700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f918c0a14d0 0x7f918c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.827+0000 7f9196ffd700 1 -- 172.21.15.89:0/598708846 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f919c0cdea0 con 0x7f91a01006c0 2023-10-26T06:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.827+0000 7f91a519a700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f918c0a14d0 0x7f918c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.827+0000 7f91a519a700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f918c0a14d0 0x7f918c0a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91a0199cf0 tx=0x7f9190009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.830+0000 7f9196ffd700 1 -- 172.21.15.89:0/598708846 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f919c0974b0 con 0x7f91a01006c0 2023-10-26T06:46:51.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.831+0000 7f4503fff700 1 -- 172.21.15.89:0/3889387919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45041005c0 msgr2=0x7f4504100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.831+0000 7f4503fff700 1 --2- 172.21.15.89:0/3889387919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45041005c0 0x7f4504100a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009990 tx=0x7f44f8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.832+0000 7f4503fff700 1 -- 172.21.15.89:0/3889387919 shutdown_connections 2023-10-26T06:46:51.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.832+0000 7f4503fff700 1 --2- 172.21.15.89:0/3889387919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4504100f80 0x7f450410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.832+0000 7f4503fff700 1 --2- 172.21.15.89:0/3889387919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4504106660 0x7f4504106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.832+0000 7f4503fff700 1 --2- 172.21.15.89:0/3889387919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45041005c0 0x7f4504100a40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.832+0000 7f4503fff700 1 -- 172.21.15.89:0/3889387919 >> 172.21.15.89:0/3889387919 conn(0x7f45040fc0f0 msgr2=0x7f45040fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.832+0000 7f4503fff700 1 -- 172.21.15.89:0/3889387919 shutdown_connections 2023-10-26T06:46:51.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.832+0000 7f4503fff700 1 -- 172.21.15.89:0/3889387919 wait complete. 2023-10-26T06:46:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 Processor -- start 2023-10-26T06:46:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 -- start start 2023-10-26T06:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45041005c0 0x7f450419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4504100f80 0x7f450419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4504106660 0x7f4504196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504196ea0 con 0x7f4504100f80 2023-10-26T06:46:51.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504197010 con 0x7f45041005c0 2023-10-26T06:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4503fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45041971d0 con 0x7f4504106660 2023-10-26T06:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f45027fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4504100f80 0x7f450419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45041005c0 0x7f450419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f45027fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4504100f80 0x7f450419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59022/0 (socket says 172.21.15.89:59022) 2023-10-26T06:46:51.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.833+0000 7f45027fc700 1 -- 172.21.15.89:0/1262247173 learned_addr learned my addr 172.21.15.89:0/1262247173 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f45037fe700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4504106660 0x7f4504196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f4502ffd700 1 -- 172.21.15.89:0/1262247173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4504106660 msgr2=0x7f4504196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f4502ffd700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4504106660 0x7f4504196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f4502ffd700 1 -- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4504100f80 msgr2=0x7f450419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f4502ffd700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4504100f80 0x7f450419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f4502ffd700 1 -- 172.21.15.89:0/1262247173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80095f0 con 0x7f45041005c0 2023-10-26T06:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f4502ffd700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45041005c0 0x7f450419c700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9c0 tx=0x7f44f400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f44ebfff700 1 -- 172.21.15.89:0/1262247173 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400cbd0 con 0x7f45041005c0 2023-10-26T06:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45041974b0 con 0x7f45041005c0 2023-10-26T06:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f44ebfff700 1 -- 172.21.15.89:0/1262247173 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f400eeb0 con 0x7f45041005c0 2023-10-26T06:46:51.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.834+0000 7f44ebfff700 1 -- 172.21.15.89:0/1262247173 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4003e30 con 0x7f45041005c0 2023-10-26T06:46:51.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.835+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45041a36f0 con 0x7f45041005c0 2023-10-26T06:46:51.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.838+0000 7f44ebfff700 1 -- 172.21.15.89:0/1262247173 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f44f4003f90 con 0x7f45041005c0 2023-10-26T06:46:51.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.839+0000 7f451cf97b80 1 Processor -- start 2023-10-26T06:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.839+0000 7f451cf97b80 1 -- start start 2023-10-26T06:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.839+0000 7f44ebfff700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44ec0a14d0 0x7f44ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.839+0000 7f44ebfff700 1 -- 172.21.15.89:0/1262247173 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f44f4014070 con 0x7f45041005c0 2023-10-26T06:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.839+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.839+0000 7f45037fe700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44ec0a14d0 0x7f44ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901f700 0x556a59019360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a590198a0 0x556a59019d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f451cf97b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a58f8c760 con 0x556a590198a0 2023-10-26T06:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f451cf97b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a58f72ce0 con 0x556a5901f700 2023-10-26T06:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f451cf97b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a58f70600 con 0x556a5901ede0 2023-10-26T06:46:51.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:53890/0 (socket says 172.21.15.89:53890) 2023-10-26T06:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 -- 172.21.15.89:0/4099615640 learned_addr learned my addr 172.21.15.89:0/4099615640 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f45037fe700 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a590198a0 0x556a59019d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 -- 172.21.15.89:0/4099615640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901f700 msgr2=0x556a59019360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f45027fc700 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901f700 0x556a59019360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901f700 0x556a59019360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 -- 172.21.15.89:0/4099615640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a590198a0 msgr2=0x556a59019d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a590198a0 0x556a59019d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f4502ffd700 1 -- 172.21.15.89:0/4099615640 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a58f29470 con 0x556a5901ede0 2023-10-26T06:46:51.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f45027fc700 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901f700 0x556a59019360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.840+0000 7f45037fe700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44ec0a14d0 0x7f44ec0a3990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f450419d400 tx=0x7f44fc007a80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.841+0000 7f4502ffd700 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901f1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44f40a2230 tx=0x7f44f409a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af275e11c693773 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.841+0000 7f44e97fa700 1 -- 172.21.15.89:0/4099615640 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f409a3d0 con 0x556a5901ede0 2023-10-26T06:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.841+0000 7f44e97fa700 1 -- 172.21.15.89:0/4099615640 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f40b2e90 con 0x556a5901ede0 2023-10-26T06:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.841+0000 7f44e97fa700 1 -- 172.21.15.89:0/4099615640 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f409a630 con 0x556a5901ede0 2023-10-26T06:46:51.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 -- 172.21.15.89:0/4099615640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 msgr2=0x556a5901f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901f1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44f40a2230 tx=0x7f44f409a010 comp rx=0 tx=0).stop 2023-10-26T06:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 -- 172.21.15.89:0/4099615640 shutdown_connections 2023-10-26T06:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a590198a0 0x556a59019d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901f1c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 --2- 172.21.15.89:0/4099615640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901f700 0x556a59019360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 -- 172.21.15.89:0/4099615640 >> 172.21.15.89:0/4099615640 conn(0x556a58f8d7a0 msgr2=0x556a59013d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 -- 172.21.15.89:0/4099615640 shutdown_connections 2023-10-26T06:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 -- 172.21.15.89:0/4099615640 wait complete. 2023-10-26T06:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 -- start start 2023-10-26T06:46:51.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.842+0000 7f451cf97b80 1 Processor -- start 2023-10-26T06:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a59025790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59022ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59023210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f451cf97b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a58f6bef0 con 0x556a5901f700 2023-10-26T06:46:51.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f451cf97b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a58f08790 con 0x556a5901ede0 2023-10-26T06:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f451cf97b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a58f08430 con 0x556a590198a0 2023-10-26T06:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f451cf97b80 10 client.0 _ll_register_callbacks cb 0x556a58e52b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f45037fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59023210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a59025790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a59025790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:53896/0 (socket says 172.21.15.89:53896) 2023-10-26T06:46:51.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f45037fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59023210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59042/0 (socket says 172.21.15.89:59042) 2023-10-26T06:46:51.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f4502ffd700 1 -- 172.21.15.89:0/3958785423 learned_addr learned my addr 172.21.15.89:0/3958785423 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:51.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.843+0000 7f45027fc700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59022ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f4502ffd700 1 -- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 msgr2=0x556a59022ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f4502ffd700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59022ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f4502ffd700 1 -- 172.21.15.89:0/3958785423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 msgr2=0x556a59023210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f4502ffd700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59023210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f4502ffd700 1 -- 172.21.15.89:0/3958785423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45041a36f0 con 0x556a590198a0 2023-10-26T06:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f45027fc700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59022ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f45037fe700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59023210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f4502ffd700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a59025790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f44f400c5b0 tx=0x7f44f409e5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c062bd7423a5e61 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f44e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f44e6ff5700 1 -- 172.21.15.89:0/3958785423 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4098770 con 0x556a590198a0 2023-10-26T06:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a58f70f70 con 0x556a590198a0 2023-10-26T06:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f44e6ff5700 1 -- 172.21.15.89:0/3958785423 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f44f40998c0 con 0x556a590198a0 2023-10-26T06:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f44e6ff5700 1 -- 172.21.15.89:0/3958785423 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44f4099a80 con 0x556a590198a0 2023-10-26T06:46:51.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f451cf97b80 10 client.25396 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.844+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556a58d96e60 con 0x556a590198a0 2023-10-26T06:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f44e6ff5700 1 -- 172.21.15.89:0/3958785423 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a58f70f70 con 0x556a590198a0 2023-10-26T06:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f44e6ff5700 1 -- 172.21.15.89:0/3958785423 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f44f40b1020 con 0x556a590198a0 2023-10-26T06:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f451cf97b80 10 client.25396 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f451cf97b80 10 client.25396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:46:51.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f451cf97b80 20 client.25396 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f451cf97b80 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a59032f40 0x556a59035360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f451cf97b80 4 client.25396 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a58fe8930 con 0x556a59032f40 2023-10-26T06:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.845+0000 7f45027fc700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a59032f40 0x556a59035360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.846+0000 7f45027fc700 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a59032f40 0x556a59035360 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd6f30f7e84876c server_cookie=54873ca9a323a2cc in_seq=0 out_seq=0 2023-10-26T06:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.846+0000 7f44e6ff5700 10 client.25396 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f44e6ff5700 1 -- 172.21.15.89:0/3958785423 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a58fe8930 con 0x556a59032f40 2023-10-26T06:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f44e6ff5700 10 client.25396 handle_command_reply: tid=0 2023-10-26T06:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 client.25396 shutdown 2023-10-26T06:46:51.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 msgr2=0x556a59025790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a59025790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f44f400c5b0 tx=0x7f44f409e5b0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 shutdown_connections 2023-10-26T06:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59023210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a59032f40 0x556a59035360 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a59025790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f44e6ff5700 0 client.25396 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 --2- 172.21.15.89:0/3958785423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59022ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 >> 172.21.15.89:0/3958785423 conn(0x556a58f8d7a0 msgr2=0x556a59018350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 shutdown_connections 2023-10-26T06:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 1 -- 172.21.15.89:0/3958785423 wait complete. 2023-10-26T06:46:51.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.847+0000 7f451cf97b80 20 client.25396 trim_cache size 0 max 16384 2023-10-26T06:46:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 Processor -- start 2023-10-26T06:46:51.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 -- start start 2023-10-26T06:46:51.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a590198a0 0x556a59022a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a5901ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a58dec9a0 con 0x556a5901f700 2023-10-26T06:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a5901f1c0 con 0x556a590198a0 2023-10-26T06:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f451cf97b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a59025e90 con 0x556a5901ede0 2023-10-26T06:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a590198a0 0x556a59022a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a590198a0 0x556a59022a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51842/0 (socket says 172.21.15.89:51842) 2023-10-26T06:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f4502ffd700 1 -- 172.21.15.89:0/122702608 learned_addr learned my addr 172.21.15.89:0/122702608 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f45037fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a5901ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.865+0000 7f45027fc700 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f4502ffd700 1 -- 172.21.15.89:0/122702608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 msgr2=0x556a5901c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f4502ffd700 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f4502ffd700 1 -- 172.21.15.89:0/122702608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 msgr2=0x556a5901ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f4502ffd700 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a5901ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f4502ffd700 1 -- 172.21.15.89:0/122702608 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a59039f50 con 0x556a590198a0 2023-10-26T06:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f4502ffd700 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a590198a0 0x556a59022a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44f409a0d0 tx=0x7f44f409a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c7c8af6b71e5856 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f44e9ffb700 1 -- 172.21.15.89:0/122702608 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f40a22d0 con 0x556a590198a0 2023-10-26T06:46:51.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f44e9ffb700 1 -- 172.21.15.89:0/122702608 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f4004240 con 0x556a590198a0 2023-10-26T06:46:51.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.866+0000 7f44e9ffb700 1 -- 172.21.15.89:0/122702608 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f409a630 con 0x556a590198a0 2023-10-26T06:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 -- 172.21.15.89:0/122702608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a590198a0 msgr2=0x556a59022a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a590198a0 0x556a59022a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f44f409a0d0 tx=0x7f44f409a040 comp rx=0 tx=0).stop 2023-10-26T06:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 -- 172.21.15.89:0/122702608 shutdown_connections 2023-10-26T06:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a5901ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a590198a0 0x556a59022a70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 --2- 172.21.15.89:0/122702608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5901ede0 0x556a5901c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 -- 172.21.15.89:0/122702608 >> 172.21.15.89:0/122702608 conn(0x556a58f8d7a0 msgr2=0x556a59022360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 -- 172.21.15.89:0/122702608 shutdown_connections 2023-10-26T06:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 -- 172.21.15.89:0/122702608 wait complete. 2023-10-26T06:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.867+0000 7f451cf97b80 1 -- start start 2023-10-26T06:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 1 Processor -- start 2023-10-26T06:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a5902a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59028740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59028e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a59029570 con 0x556a5901f700 2023-10-26T06:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a59086d40 con 0x556a5901ede0 2023-10-26T06:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a59086e80 con 0x556a590198a0 2023-10-26T06:46:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f451cf97b80 10 client.0 _ll_register_callbacks cb 0x556a58e52b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a5902a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.868+0000 7f4502ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a5902a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:53916/0 (socket says 172.21.15.89:53916) 2023-10-26T06:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f4502ffd700 1 -- 172.21.15.89:0/1925016251 learned_addr learned my addr 172.21.15.89:0/1925016251 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f45037fe700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59028e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f45027fc700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59028740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f4502ffd700 1 -- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 msgr2=0x556a59028740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f4502ffd700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59028740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f4502ffd700 1 -- 172.21.15.89:0/1925016251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 msgr2=0x556a59028e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f4502ffd700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59028e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f4502ffd700 1 -- 172.21.15.89:0/1925016251 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a58d96e60 con 0x556a590198a0 2023-10-26T06:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f45027fc700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59028740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f45037fe700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59028e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f4502ffd700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a5902a270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44f40a2230 tx=0x7f44f4099e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a53000094fe9e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f44e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f44e6ff5700 1 -- 172.21.15.89:0/1925016251 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4099ac0 con 0x556a590198a0 2023-10-26T06:46:51.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f44e6ff5700 1 -- 172.21.15.89:0/1925016251 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f44f40a22d0 con 0x556a590198a0 2023-10-26T06:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.869+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44f40b2e90 con 0x556a590198a0 2023-10-26T06:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f44e6ff5700 1 -- 172.21.15.89:0/1925016251 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44f4086070 con 0x556a590198a0 2023-10-26T06:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 10 client.25402 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556a58d7fce0 con 0x556a590198a0 2023-10-26T06:46:51.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f44e6ff5700 1 -- 172.21.15.89:0/1925016251 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f40b2e90 con 0x556a590198a0 2023-10-26T06:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f44e6ff5700 1 -- 172.21.15.89:0/1925016251 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f44f40b1040 con 0x556a590198a0 2023-10-26T06:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 10 client.25402 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 20 client.25402 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:51.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a590880f0 0x556a5908a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 4 client.25402 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.870+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556a58f64ed0 con 0x556a590880f0 2023-10-26T06:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.871+0000 7f45027fc700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a590880f0 0x556a5908a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.871+0000 7f44e6ff5700 10 client.25402 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.871+0000 7f45027fc700 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a590880f0 0x556a5908a4e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85efcf686e78cfe server_cookie=f1dce056f0a92860 in_seq=0 out_seq=0 2023-10-26T06:46:51.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.872+0000 7f44e6ff5700 1 -- 172.21.15.89:0/1925016251 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556a58f64ed0 con 0x556a590880f0 2023-10-26T06:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.872+0000 7f44e6ff5700 10 client.25402 handle_command_reply: tid=0 2023-10-26T06:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.872+0000 7f451cf97b80 1 client.25402 shutdown 2023-10-26T06:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 msgr2=0x556a5902a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a5902a270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44f40a2230 tx=0x7f44f4099e30 comp rx=0 tx=0).stop 2023-10-26T06:46:51.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 shutdown_connections 2023-10-26T06:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556a5901f700 0x556a59028e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556a590880f0 0x556a5908a4e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a590198a0 0x556a5902a270 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 --2- 172.21.15.89:0/1925016251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a5901ede0 0x556a59028740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f44e6ff5700 0 client.25402 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 >> 172.21.15.89:0/1925016251 conn(0x556a58f8d7a0 msgr2=0x556a5901c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 shutdown_connections 2023-10-26T06:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 1 -- 172.21.15.89:0/1925016251 wait complete. 2023-10-26T06:46:51.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.873+0000 7f451cf97b80 20 client.25402 trim_cache size 0 max 16384 2023-10-26T06:46:51.888 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:46:51.888 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:46:51.888 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "2ef29119-16ad-4220-b192-078b1ff7acc2", 2023-10-26T06:46:51.888 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:46:51.889 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:46:51.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44ec0a14d0 msgr2=0x7f44ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44ec0a14d0 0x7f44ec0a3990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f450419d400 tx=0x7f44fc007a80 comp rx=0 tx=0).stop 2023-10-26T06:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45041005c0 msgr2=0x7f450419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45041005c0 0x7f450419c700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9c0 tx=0x7f44f400ecd0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 shutdown_connections 2023-10-26T06:46:51.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44ec0a14d0 0x7f44ec0a3990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4504106660 0x7f4504196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45041005c0 0x7f450419c700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 --2- 172.21.15.89:0/1262247173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4504100f80 0x7f450419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 >> 172.21.15.89:0/1262247173 conn(0x7f45040fc0f0 msgr2=0x7f45040fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:51.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 shutdown_connections 2023-10-26T06:46:51.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:51.874+0000 7f4503fff700 1 -- 172.21.15.89:0/1262247173 wait complete. 2023-10-26T06:46:51.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:46:52.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.084+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f91a0110640 con 0x7f918c0a14d0 2023-10-26T06:46:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 -- 172.21.15.89:0/743222187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 msgr2=0x7f3b8c10f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 --2- 172.21.15.89:0/743222187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c10f7f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b0a0 tx=0x7f3b8400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 -- 172.21.15.89:0/743222187 shutdown_connections 2023-10-26T06:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 --2- 172.21.15.89:0/743222187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c10f7f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 --2- 172.21.15.89:0/743222187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c108930 0x7f3b8c108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 --2- 172.21.15.89:0/743222187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b8c102920 0x7f3b8c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 -- 172.21.15.89:0/743222187 >> 172.21.15.89:0/743222187 conn(0x7f3b8c0fe3b0 msgr2=0x7f3b8c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 -- 172.21.15.89:0/743222187 shutdown_connections 2023-10-26T06:46:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.097+0000 7f3b90fff700 1 -- 172.21.15.89:0/743222187 wait complete. 2023-10-26T06:46:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.098+0000 7f3b90fff700 1 Processor -- start 2023-10-26T06:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.098+0000 7f3b90fff700 1 -- start start 2023-10-26T06:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b90fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c102920 0x7f3b8c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b90fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b90fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b8c108930 0x7f3b8c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b90fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c19cb60 con 0x7f3b8c108930 2023-10-26T06:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b90fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c19ccd0 con 0x7f3b8c102920 2023-10-26T06:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b90fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c19ce90 con 0x7f3b8c1032e0 2023-10-26T06:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b8a59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c102920 0x7f3b8c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b8c108930 0x7f3b8c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:53928/0 (socket says 172.21.15.89:53928) 2023-10-26T06:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b8a59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c102920 0x7f3b8c198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51874/0 (socket says 172.21.15.89:51874) 2023-10-26T06:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1105050019 learned_addr learned my addr 172.21.15.89:0/1105050019 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1105050019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c102920 msgr2=0x7f3b8c198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c102920 0x7f3b8c198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b8c108930 msgr2=0x7f3b8c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b8c108930 0x7f3b8c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.099+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1105050019 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b84009d00 con 0x7f3b8c1032e0 2023-10-26T06:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.100+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c1992c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e9a0 tx=0x7f3b8000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.100+0000 7f3b7b7fe700 1 -- 172.21.15.89:0/1105050019 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80009790 con 0x7f3b8c1032e0 2023-10-26T06:46:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.100+0000 7f3b7b7fe700 1 -- 172.21.15.89:0/1105050019 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b80010430 con 0x7f3b8c1032e0 2023-10-26T06:46:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.100+0000 7f3b7b7fe700 1 -- 172.21.15.89:0/1105050019 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80018680 con 0x7f3b8c1032e0 2023-10-26T06:46:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.101+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c19d170 con 0x7f3b8c1032e0 2023-10-26T06:46:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.101+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c1a5ad0 con 0x7f3b8c1032e0 2023-10-26T06:46:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.105+0000 7f3b7b7fe700 1 -- 172.21.15.89:0/1105050019 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f3b800105a0 con 0x7f3b8c1032e0 2023-10-26T06:46:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.106+0000 7f3ba49bab80 1 Processor -- start 2023-10-26T06:46:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.106+0000 7f3ba49bab80 1 -- start start 2023-10-26T06:46:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.106+0000 7f3b7b7fe700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b740a1550 0x7f3b740a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.106+0000 7f3b7b7fe700 1 -- 172.21.15.89:0/1105050019 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3b80014070 con 0x7f3b8c1032e0 2023-10-26T06:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01b3f0 0x5583ae01b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b740a1550 0x7f3b740a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01bd10 0x5583ae0158a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae016260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583adf806e0 con 0x5583ae01bd10 2023-10-26T06:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583adf66c60 con 0x5583ae015de0 2023-10-26T06:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583adf64580 con 0x5583ae01b3f0 2023-10-26T06:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01bd10 0x5583ae0158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01bd10 0x5583ae0158a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59126/0 (socket says 172.21.15.89:59126) 2023-10-26T06:46:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 -- 172.21.15.89:0/2857940921 learned_addr learned my addr 172.21.15.89:0/2857940921 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01b3f0 0x5583ae01b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae016260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 -- 172.21.15.89:0/2857940921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01b3f0 msgr2=0x5583ae01b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01b3f0 0x5583ae01b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 -- 172.21.15.89:0/2857940921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 msgr2=0x5583ae016260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae016260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.107+0000 7f3b89d9b700 1 -- 172.21.15.89:0/2857940921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583adf1d3f0 con 0x5583ae01bd10 2023-10-26T06:46:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.108+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01b3f0 0x5583ae01b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.108+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b740a1550 0x7f3b740a3a10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009610 tx=0x7f3b84006230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.108+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01bd10 0x5583ae0158a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b80096070 tx=0x7f3b80096760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8597b041bd0ccc02 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.108+0000 7f3b78ff9700 1 -- 172.21.15.89:0/2857940921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800a2680 con 0x5583ae01bd10 2023-10-26T06:46:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.108+0000 7f3b78ff9700 1 -- 172.21.15.89:0/2857940921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b80098430 con 0x5583ae01bd10 2023-10-26T06:46:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.108+0000 7f3b78ff9700 1 -- 172.21.15.89:0/2857940921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8009a7b0 con 0x5583ae01bd10 2023-10-26T06:46:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2857940921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01bd10 msgr2=0x5583ae0158a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01bd10 0x5583ae0158a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b80096070 tx=0x7f3b80096760 comp rx=0 tx=0).stop 2023-10-26T06:46:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2857940921 shutdown_connections 2023-10-26T06:46:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae016260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01b3f0 0x5583ae01b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2857940921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01bd10 0x5583ae0158a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2857940921 >> 172.21.15.89:0/2857940921 conn(0x5583adf81720 msgr2=0x5583ae007b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2857940921 shutdown_connections 2023-10-26T06:46:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.109+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2857940921 wait complete. 2023-10-26T06:46:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.110+0000 7f3ba49bab80 1 -- start start 2023-10-26T06:46:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.110+0000 7f3ba49bab80 1 Processor -- start 2023-10-26T06:46:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae021d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae01eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae01f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583adf5fe70 con 0x5583ae015de0 2023-10-26T06:46:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583adefc710 con 0x5583ae01b3f0 2023-10-26T06:46:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583adefc3b0 con 0x5583ae01bd10 2023-10-26T06:46:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3ba49bab80 10 client.0 _ll_register_callbacks cb 0x5583ade46a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae01eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae01eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51906/0 (socket says 172.21.15.89:51906) 2023-10-26T06:46:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1917580643 learned_addr learned my addr 172.21.15.89:0/1917580643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae01f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae021d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1917580643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 msgr2=0x5583ae01f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae01f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1917580643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 msgr2=0x5583ae021d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae021d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.111+0000 7f3b89d9b700 1 -- 172.21.15.89:0/1917580643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c1a5ad0 con 0x5583ae01b3f0 2023-10-26T06:46:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae01eff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b800a2150 tx=0x7f3b8009e500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957acde60cde9b5b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3b71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:46:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3b71ffb700 1 -- 172.21.15.89:0/1917580643 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8009e230 con 0x5583ae01b3f0 2023-10-26T06:46:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3b71ffb700 1 -- 172.21.15.89:0/1917580643 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3b80010880 con 0x5583ae01b3f0 2023-10-26T06:46:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583adf64ef0 con 0x5583ae01b3f0 2023-10-26T06:46:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3b71ffb700 1 -- 172.21.15.89:0/1917580643 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b8009a550 con 0x5583ae01b3f0 2023-10-26T06:46:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3ba49bab80 10 client.25418 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.112+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5583add8ade0 con 0x5583ae01b3f0 2023-10-26T06:46:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3b71ffb700 1 -- 172.21.15.89:0/1917580643 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583adf64ef0 con 0x5583ae01b3f0 2023-10-26T06:46:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3b71ffb700 1 -- 172.21.15.89:0/1917580643 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b80099250 con 0x5583ae01b3f0 2023-10-26T06:46:52.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3ba49bab80 10 client.25418 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3ba49bab80 10 client.25418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:46:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3ba49bab80 20 client.25418 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae040350 0x5583ae042770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3ba49bab80 4 client.25418 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:46:52.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583adfdc8b0 con 0x5583ae040350 2023-10-26T06:46:52.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.113+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae040350 0x5583ae042770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.114+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae040350 0x5583ae042770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad5c26644461950 server_cookie=f3d88d0b4cc97416 in_seq=0 out_seq=0 2023-10-26T06:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.114+0000 7f3b71ffb700 10 client.25418 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.114+0000 7f3b71ffb700 1 -- 172.21.15.89:0/1917580643 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583adfdc8b0 con 0x5583ae040350 2023-10-26T06:46:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.114+0000 7f3b71ffb700 10 client.25418 handle_command_reply: tid=0 2023-10-26T06:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.114+0000 7f3ba49bab80 1 client.25418 shutdown 2023-10-26T06:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 msgr2=0x5583ae01eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae01eff0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b800a2150 tx=0x7f3b8009e500 comp rx=0 tx=0).stop 2023-10-26T06:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 shutdown_connections 2023-10-26T06:46:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae021d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae040350 0x5583ae042770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae01f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/1917580643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae01eff0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 >> 172.21.15.89:0/1917580643 conn(0x5583adf81720 msgr2=0x5583ae014890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3b71ffb700 0 client.25418 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 shutdown_connections 2023-10-26T06:46:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 1 -- 172.21.15.89:0/1917580643 wait complete. 2023-10-26T06:46:52.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.115+0000 7f3ba49bab80 20 client.25418 trim_cache size 0 max 16384 2023-10-26T06:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.139+0000 7f3ba49bab80 1 Processor -- start 2023-10-26T06:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.139+0000 7f3ba49bab80 1 -- start start 2023-10-26T06:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae0025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01b3f0 0x5583ae026720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae0095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583adde0920 con 0x5583ae01b3f0 2023-10-26T06:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ae004260 con 0x5583ae015de0 2023-10-26T06:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ae01fd30 con 0x5583ae01bd10 2023-10-26T06:46:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae0095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae0095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:53954/0 (socket says 172.21.15.89:53954) 2023-10-26T06:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01b3f0 0x5583ae026720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3b8ad9d700 1 -- 172.21.15.89:0/2725433103 learned_addr learned my addr 172.21.15.89:0/2725433103 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.140+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae0025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b8ad9d700 1 -- 172.21.15.89:0/2725433103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 msgr2=0x5583ae0025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae0025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b8ad9d700 1 -- 172.21.15.89:0/2725433103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01b3f0 msgr2=0x5583ae026720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01b3f0 0x5583ae026720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b8ad9d700 1 -- 172.21.15.89:0/2725433103 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583add73c60 con 0x5583ae01bd10 2023-10-26T06:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b89d9b700 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01b3f0 0x5583ae026720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae0095e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b0a0 tx=0x7f3b84024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd14e72f7a5b8515 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b797fa700 1 -- 172.21.15.89:0/2725433103 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400e040 con 0x5583ae01bd10 2023-10-26T06:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b797fa700 1 -- 172.21.15.89:0/2725433103 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b8400be90 con 0x5583ae01bd10 2023-10-26T06:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.141+0000 7f3b797fa700 1 -- 172.21.15.89:0/2725433103 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401cea0 con 0x5583ae01bd10 2023-10-26T06:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2725433103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 msgr2=0x5583ae0095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae0095e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b0a0 tx=0x7f3b84024000 comp rx=0 tx=0).stop 2023-10-26T06:46:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2725433103 shutdown_connections 2023-10-26T06:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae0095e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae015de0 0x5583ae0025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2725433103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae01b3f0 0x5583ae026720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2725433103 >> 172.21.15.89:0/2725433103 conn(0x5583adf81720 msgr2=0x5583ae01e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2725433103 shutdown_connections 2023-10-26T06:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.142+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2725433103 wait complete. 2023-10-26T06:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.143+0000 7f3ba49bab80 1 -- start start 2023-10-26T06:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.143+0000 7f3ba49bab80 1 Processor -- start 2023-10-26T06:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae026760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae024d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3ba49bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae025260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ae025940 con 0x5583ae015de0 2023-10-26T06:46:52.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ae025ab0 con 0x5583ae01b3f0 2023-10-26T06:46:52.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3ba49bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ae094190 con 0x5583ae01bd10 2023-10-26T06:46:52.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3ba49bab80 10 client.0 _ll_register_callbacks cb 0x5583ade46a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:46:52.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b8ad9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae025260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae024d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b89d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae024d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51924/0 (socket says 172.21.15.89:51924) 2023-10-26T06:46:52.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b89d9b700 1 -- 172.21.15.89:0/2526614244 learned_addr learned my addr 172.21.15.89:0/2526614244 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:52.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae026760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b8ad9d700 1 -- 172.21.15.89:0/2526614244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 msgr2=0x5583ae024d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae024d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b8ad9d700 1 -- 172.21.15.89:0/2526614244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 msgr2=0x5583ae026760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:46:52.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.144+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae026760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.145+0000 7f3b8ad9d700 1 -- 172.21.15.89:0/2526614244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583add8ade0 con 0x5583ae01bd10 2023-10-26T06:46:52.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.145+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:46:52.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.145+0000 7f3b8ad9d700 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae025260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b840062a0 tx=0x7f3b8401c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70dc14ff3e9a0ad server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.145+0000 7f3b71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:46:52.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.145+0000 7f3b71ffb700 1 -- 172.21.15.89:0/2526614244 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400e040 con 0x5583ae01bd10 2023-10-26T06:46:52.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.145+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b80098430 con 0x5583ae01bd10 2023-10-26T06:46:52.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.145+0000 7f3b71ffb700 1 -- 172.21.15.89:0/2526614244 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3b84005040 con 0x5583ae01bd10 2023-10-26T06:46:52.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3b71ffb700 1 -- 172.21.15.89:0/2526614244 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b84003e60 con 0x5583ae01bd10 2023-10-26T06:46:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 10 client.25426 fetch_fsmap learned FSMap version 20 2023-10-26T06:46:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f3b84024070 con 0x5583ae01bd10 2023-10-26T06:46:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3b71ffb700 1 -- 172.21.15.89:0/2526614244 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b80098430 con 0x5583ae01bd10 2023-10-26T06:46:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3b71ffb700 1 -- 172.21.15.89:0/2526614244 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b8402f030 con 0x5583ae01bd10 2023-10-26T06:46:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 10 client.25426 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:46:52.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:46:52.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 20 client.25426 populate_metadata read hostname 'smithi089' 2023-10-26T06:46:52.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae095500 0x5583ae097920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 4 client.25426 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:46:52.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.146+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5583adf58e50 con 0x5583ae095500 2023-10-26T06:46:52.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.147+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae095500 0x5583ae097920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.147+0000 7f3b8a59c700 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae095500 0x5583ae097920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24edde9e65da7bb server_cookie=e6f98b09f64c0df3 in_seq=0 out_seq=0 2023-10-26T06:46:52.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.147+0000 7f3b71ffb700 10 client.25426 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:52.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.147+0000 7f3b71ffb700 1 -- 172.21.15.89:0/2526614244 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5583adf58e50 con 0x5583ae095500 2023-10-26T06:46:52.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3b71ffb700 10 client.25426 handle_command_reply: tid=0 2023-10-26T06:46:52.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 client.25426 shutdown 2023-10-26T06:46:52.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 msgr2=0x5583ae025260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae025260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b840062a0 tx=0x7f3b8401c6f0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 shutdown_connections 2023-10-26T06:46:52.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5583ae015de0 0x5583ae026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5583ae095500 0x5583ae097920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3b71ffb700 0 client.25426 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:46:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583ae01bd10 0x5583ae025260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 --2- 172.21.15.89:0/2526614244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583ae01b3f0 0x5583ae024d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 >> 172.21.15.89:0/2526614244 conn(0x5583adf81720 msgr2=0x5583ae0095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 shutdown_connections 2023-10-26T06:46:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 1 -- 172.21.15.89:0/2526614244 wait complete. 2023-10-26T06:46:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.148+0000 7f3ba49bab80 20 client.25426 trim_cache size 0 max 16384 2023-10-26T06:46:52.163 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:46:52.163 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:46:52.164 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:46:52.164 INFO:teuthology.orchestra.run.smithi089.stdout: "2ef29119-16ad-4220-b192-078b1ff7acc2": { 2023-10-26T06:46:52.164 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:46:52.164 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "2ef29119-16ad-4220-b192-078b1ff7acc2", 2023-10-26T06:46:52.164 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:46:52.164 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:46:52.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:46:52.165 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:46:52.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b740a1550 msgr2=0x7f3b740a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b740a1550 0x7f3b740a3a10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009610 tx=0x7f3b84006230 comp rx=0 tx=0).stop 2023-10-26T06:46:52.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 msgr2=0x7f3b8c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c1992c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e9a0 tx=0x7f3b8000ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 shutdown_connections 2023-10-26T06:46:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b740a1550 0x7f3b740a3a10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b8c108930 0x7f3b8c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c102920 0x7f3b8c198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 --2- 172.21.15.89:0/1105050019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b8c1032e0 0x7f3b8c1992c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 >> 172.21.15.89:0/1105050019 conn(0x7f3b8c0fe3b0 msgr2=0x7f3b8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 shutdown_connections 2023-10-26T06:46:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.149+0000 7f3b90fff700 1 -- 172.21.15.89:0/1105050019 wait complete. 2023-10-26T06:46:52.172 INFO:tasks.cephfs.filesystem:scrub status for tag:2ef29119-16ad-4220-b192-078b1ff7acc2 - {'path': '/', 'tag': '2ef29119-16ad-4220-b192-078b1ff7acc2', 'options': 'recursive,force'} 2023-10-26T06:46:52.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.215+0000 7f9196ffd700 1 -- 172.21.15.89:0/598708846 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f91a0110640 con 0x7f918c0a14d0 2023-10-26T06:46:52.217 INFO:teuthology.orchestra.run.smithi089.stdout:Schedule set for path / 2023-10-26T06:46:52.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f918c0a14d0 msgr2=0x7f918c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f918c0a14d0 0x7f918c0a3990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91a0199cf0 tx=0x7f9190009260 comp rx=0 tx=0).stop 2023-10-26T06:46:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01006c0 msgr2=0x7f91a019ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01006c0 0x7f91a019ea00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c004730 comp rx=0 tx=0).stop 2023-10-26T06:46:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 shutdown_connections 2023-10-26T06:46:52.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f918c0a14d0 0x7f918c0a3990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01006c0 0x7f91a019ea00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91a01066d0 0x7f91a0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 --2- 172.21.15.89:0/598708846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a0101080 0x7f91a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 >> 172.21.15.89:0/598708846 conn(0x7f91a00fc190 msgr2=0x7f91a010bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.217+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 shutdown_connections 2023-10-26T06:46:52.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.218+0000 7f91a7bff700 1 -- 172.21.15.89:0/598708846 wait complete. 2023-10-26T06:46:52.238 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-10-26T06:46:52.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 -- 172.21.15.89:0/4103471176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41066f0 msgr2=0x7fecb4106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 --2- 172.21.15.89:0/4103471176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41066f0 0x7fecb4106ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feca8009990 tx=0x7feca8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 -- 172.21.15.89:0/4103471176 shutdown_connections 2023-10-26T06:46:52.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 --2- 172.21.15.89:0/4103471176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41066f0 0x7fecb4106ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 --2- 172.21.15.89:0/4103471176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb41010a0 0x7fecb410d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 --2- 172.21.15.89:0/4103471176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecb41006e0 0x7fecb4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 -- 172.21.15.89:0/4103471176 >> 172.21.15.89:0/4103471176 conn(0x7fecb40fc1f0 msgr2=0x7fecb40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 -- 172.21.15.89:0/4103471176 shutdown_connections 2023-10-26T06:46:52.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.433+0000 7fecbabff700 1 -- 172.21.15.89:0/4103471176 wait complete. 2023-10-26T06:46:52.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 Processor -- start 2023-10-26T06:46:52.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 -- start start 2023-10-26T06:46:52.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41006e0 0x7fecb4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb41010a0 0x7fecb4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecb41066f0 0x7fecb419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419cb40 con 0x7fecb41006e0 2023-10-26T06:46:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419ccb0 con 0x7fecb41010a0 2023-10-26T06:46:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecbabff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419ce70 con 0x7fecb41066f0 2023-10-26T06:46:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecb899b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41006e0 0x7fecb4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecb899b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41006e0 0x7fecb4198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59166/0 (socket says 172.21.15.89:59166) 2023-10-26T06:46:52.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecb899b700 1 -- 172.21.15.89:0/3180503578 learned_addr learned my addr 172.21.15.89:0/3180503578 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecb3fff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb41010a0 0x7fecb4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecb919c700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecb41066f0 0x7fecb419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecb899b700 1 -- 172.21.15.89:0/3180503578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecb41066f0 msgr2=0x7fecb419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.434+0000 7fecb899b700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecb41066f0 0x7fecb419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecb899b700 1 -- 172.21.15.89:0/3180503578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb41010a0 msgr2=0x7fecb4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecb899b700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb41010a0 0x7fecb4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecb899b700 1 -- 172.21.15.89:0/3180503578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80095f0 con 0x7fecb41006e0 2023-10-26T06:46:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecb899b700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41006e0 0x7fecb4198d10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feca8005910 tx=0x7feca8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecb1ffb700 1 -- 172.21.15.89:0/3180503578 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca801d070 con 0x7fecb41006e0 2023-10-26T06:46:52.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecb1ffb700 1 -- 172.21.15.89:0/3180503578 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca8022470 con 0x7fecb41006e0 2023-10-26T06:46:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb419d0f0 con 0x7fecb41006e0 2023-10-26T06:46:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecb1ffb700 1 -- 172.21.15.89:0/3180503578 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8017650 con 0x7fecb41006e0 2023-10-26T06:46:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.436+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb41a5ab0 con 0x7fecb41006e0 2023-10-26T06:46:52.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.437+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb4101e20 con 0x7fecb41006e0 2023-10-26T06:46:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.439+0000 7fecb1ffb700 1 -- 172.21.15.89:0/3180503578 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7feca80225e0 con 0x7fecb41006e0 2023-10-26T06:46:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.440+0000 7fecb1ffb700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feca40a14d0 0x7feca40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.441+0000 7fecb1ffb700 1 -- 172.21.15.89:0/3180503578 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7feca80ce050 con 0x7fecb41006e0 2023-10-26T06:46:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.441+0000 7fecb3fff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feca40a14d0 0x7feca40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.441+0000 7fecb3fff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feca40a14d0 0x7feca40a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fecb419a220 tx=0x7feca0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.444+0000 7fecb1ffb700 1 -- 172.21.15.89:0/3180503578 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feca8097780 con 0x7fecb41006e0 2023-10-26T06:46:52.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:52 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/416816663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:52.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:52 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:46:52.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:52 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:46:52.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:52 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:46:52.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:52 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:46:52.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:52 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/416816663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:52.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:52 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:46:52.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:52 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:46:52.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:52 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:46:52.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:52 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:46:52.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:52 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/416816663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:46:52.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:52 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:46:52.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:52 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:46:52.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:52 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:46:52.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:52 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:46:52.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.672+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fecb4110650 con 0x7feca40a14d0 2023-10-26T06:46:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.731+0000 7fecb1ffb700 1 -- 172.21.15.89:0/3180503578 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fecb4110650 con 0x7feca40a14d0 2023-10-26T06:46:52.734 INFO:teuthology.orchestra.run.smithi089.stdout:Retention added to path / 2023-10-26T06:46:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feca40a14d0 msgr2=0x7feca40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feca40a14d0 0x7feca40a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fecb419a220 tx=0x7feca0009260 comp rx=0 tx=0).stop 2023-10-26T06:46:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41006e0 msgr2=0x7fecb4198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41006e0 0x7fecb4198d10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feca8005910 tx=0x7feca8004730 comp rx=0 tx=0).stop 2023-10-26T06:46:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 shutdown_connections 2023-10-26T06:46:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7feca40a14d0 0x7feca40a3990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecb41006e0 0x7fecb4198d10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecb41066f0 0x7fecb419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 --2- 172.21.15.89:0/3180503578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb41010a0 0x7fecb4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 >> 172.21.15.89:0/3180503578 conn(0x7fecb40fc1f0 msgr2=0x7fecb410bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 shutdown_connections 2023-10-26T06:46:52.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.734+0000 7fecbabff700 1 -- 172.21.15.89:0/3180503578 wait complete. 2023-10-26T06:46:52.754 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-26T06:46:52.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 -- 172.21.15.89:0/959304250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 msgr2=0x7fb718100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 --2- 172.21.15.89:0/959304250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718100b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb708009990 tx=0x7fb708009ca0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 -- 172.21.15.89:0/959304250 shutdown_connections 2023-10-26T06:46:52.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 --2- 172.21.15.89:0/959304250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb718101080 0x7fb71810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 --2- 172.21.15.89:0/959304250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7181066d0 0x7fb718106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 --2- 172.21.15.89:0/959304250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718100b40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 -- 172.21.15.89:0/959304250 >> 172.21.15.89:0/959304250 conn(0x7fb7180fc190 msgr2=0x7fb7180fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 -- 172.21.15.89:0/959304250 shutdown_connections 2023-10-26T06:46:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.945+0000 7fb71ffff700 1 -- 172.21.15.89:0/959304250 wait complete. 2023-10-26T06:46:52.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 Processor -- start 2023-10-26T06:46:52.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 -- start start 2023-10-26T06:46:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb718101080 0x7fb718199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7181066d0 0x7fb71819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71819cb40 con 0x7fb7181006c0 2023-10-26T06:46:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71819ccb0 con 0x7fb7181066d0 2023-10-26T06:46:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.946+0000 7fb71ffff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71819ce70 con 0x7fb718101080 2023-10-26T06:46:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59188/0 (socket says 172.21.15.89:59188) 2023-10-26T06:46:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 -- 172.21.15.89:0/4262391559 learned_addr learned my addr 172.21.15.89:0/4262391559 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71e59c700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7181066d0 0x7fb71819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71d59a700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb718101080 0x7fb718199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 -- 172.21.15.89:0/4262391559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb718101080 msgr2=0x7fb718199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb718101080 0x7fb718199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 -- 172.21.15.89:0/4262391559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7181066d0 msgr2=0x7fb71819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7181066d0 0x7fb71819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 -- 172.21.15.89:0/4262391559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7080095f0 con 0x7fb7181006c0 2023-10-26T06:46:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71d59a700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb718101080 0x7fb718199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:46:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71dd9b700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718198d10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9a0 tx=0x7fb71400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb70effd700 1 -- 172.21.15.89:0/4262391559 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714009790 con 0x7fb7181006c0 2023-10-26T06:46:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb70effd700 1 -- 172.21.15.89:0/4262391559 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb71400cbd0 con 0x7fb7181006c0 2023-10-26T06:46:52.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71819d150 con 0x7fb7181006c0 2023-10-26T06:46:52.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.947+0000 7fb70effd700 1 -- 172.21.15.89:0/4262391559 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714018680 con 0x7fb7181006c0 2023-10-26T06:46:52.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.948+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7181a5ab0 con 0x7fb7181006c0 2023-10-26T06:46:52.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.949+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb718101e00 con 0x7fb7181006c0 2023-10-26T06:46:52.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.950+0000 7fb70effd700 1 -- 172.21.15.89:0/4262391559 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fb71400cd40 con 0x7fb7181006c0 2023-10-26T06:46:52.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.953+0000 7fb70effd700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7040a1550 0x7fb7040a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:52.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.953+0000 7fb70effd700 1 -- 172.21.15.89:0/4262391559 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb714014070 con 0x7fb7181006c0 2023-10-26T06:46:52.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.953+0000 7fb71d59a700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7040a1550 0x7fb7040a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:52.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.954+0000 7fb71d59a700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7040a1550 0x7fb7040a3a10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb708006010 tx=0x7fb70800b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:52.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:52.957+0000 7fb70effd700 1 -- 172.21.15.89:0/4262391559 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb714052100 con 0x7fb7181006c0 2023-10-26T06:46:53.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.184+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fb71819a070 con 0x7fb7040a1550 2023-10-26T06:46:53.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.192+0000 7fb70effd700 1 -- 172.21.15.89:0/4262391559 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fb71819a070 con 0x7fb7040a1550 2023-10-26T06:46:53.194 INFO:teuthology.orchestra.run.smithi089.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-26T00:00:00", "created": "2023-10-26T06:46:52", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-10-26T06:46:53.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7040a1550 msgr2=0x7fb7040a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:53.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7040a1550 0x7fb7040a3a10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb708006010 tx=0x7fb70800b260 comp rx=0 tx=0).stop 2023-10-26T06:46:53.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 msgr2=0x7fb718198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:53.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718198d10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9a0 tx=0x7fb71400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 shutdown_connections 2023-10-26T06:46:53.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7040a1550 0x7fb7040a3a10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7181006c0 0x7fb718198d10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7181066d0 0x7fb71819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 --2- 172.21.15.89:0/4262391559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb718101080 0x7fb718199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 >> 172.21.15.89:0/4262391559 conn(0x7fb7180fc190 msgr2=0x7fb7180fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 shutdown_connections 2023-10-26T06:46:53.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.195+0000 7fb71ffff700 1 -- 172.21.15.89:0/4262391559 wait complete. 2023-10-26T06:46:53.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-26T06:46:53.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 -- 172.21.15.89:0/109485137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c101080 msgr2=0x7fd21c10d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:53.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 --2- 172.21.15.89:0/109485137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c101080 0x7fd21c10d590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd21400b0a0 tx=0x7fd21400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 -- 172.21.15.89:0/109485137 shutdown_connections 2023-10-26T06:46:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 --2- 172.21.15.89:0/109485137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c101080 0x7fd21c10d590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 --2- 172.21.15.89:0/109485137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd21c1066d0 0x7fd21c106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 --2- 172.21.15.89:0/109485137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c1006c0 0x7fd21c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 -- 172.21.15.89:0/109485137 >> 172.21.15.89:0/109485137 conn(0x7fd21c0fc190 msgr2=0x7fd21c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 -- 172.21.15.89:0/109485137 shutdown_connections 2023-10-26T06:46:53.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.408+0000 7fd2245ff700 1 -- 172.21.15.89:0/109485137 wait complete. 2023-10-26T06:46:53.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.409+0000 7fd2245ff700 1 Processor -- start 2023-10-26T06:46:53.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.409+0000 7fd2245ff700 1 -- start start 2023-10-26T06:46:53.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd2245ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c1006c0 0x7fd21c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:53.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd2245ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd21c101080 0x7fd21c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:53.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd2245ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c1066d0 0x7fd21c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:53.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd2245ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c19dca0 con 0x7fd21c1006c0 2023-10-26T06:46:53.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd2245ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c19de10 con 0x7fd21c1066d0 2023-10-26T06:46:53.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd2245ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c19dfd0 con 0x7fd21c101080 2023-10-26T06:46:53.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c1006c0 0x7fd21c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:53.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c1006c0 0x7fd21c198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59202/0 (socket says 172.21.15.89:59202) 2023-10-26T06:46:53.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 -- 172.21.15.89:0/3925501095 learned_addr learned my addr 172.21.15.89:0/3925501095 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:46:53.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd221b9a700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd21c101080 0x7fd21c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:53.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd222b9c700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c1066d0 0x7fd21c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:53.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 -- 172.21.15.89:0/3925501095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd21c101080 msgr2=0x7fd21c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:53.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd21c101080 0x7fd21c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 -- 172.21.15.89:0/3925501095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c1066d0 msgr2=0x7fd21c19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:53.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c1066d0 0x7fd21c19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd22239b700 1 -- 172.21.15.89:0/3925501095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214009d00 con 0x7fd21c1006c0 2023-10-26T06:46:53.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.410+0000 7fd222b9c700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c1066d0 0x7fd21c19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:46:53.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.411+0000 7fd22239b700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c1006c0 0x7fd21c198d80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd21800e9a0 tx=0x7fd21800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:53.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.411+0000 7fd2137fe700 1 -- 172.21.15.89:0/3925501095 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218009790 con 0x7fd21c1006c0 2023-10-26T06:46:53.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.411+0000 7fd2137fe700 1 -- 172.21.15.89:0/3925501095 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd21800cbd0 con 0x7fd21c1006c0 2023-10-26T06:46:53.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.411+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c19e2b0 con 0x7fd21c1006c0 2023-10-26T06:46:53.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.411+0000 7fd2137fe700 1 -- 172.21.15.89:0/3925501095 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218018680 con 0x7fd21c1006c0 2023-10-26T06:46:53.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.411+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c1a5af0 con 0x7fd21c1006c0 2023-10-26T06:46:53.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.412+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c101e00 con 0x7fd21c1006c0 2023-10-26T06:46:53.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.414+0000 7fd2137fe700 1 -- 172.21.15.89:0/3925501095 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fd21800cd40 con 0x7fd21c1006c0 2023-10-26T06:46:53.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.416+0000 7fd2137fe700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2080a14d0 0x7fd2080a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:46:53.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.416+0000 7fd2137fe700 1 -- 172.21.15.89:0/3925501095 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd218014070 con 0x7fd21c1006c0 2023-10-26T06:46:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.416+0000 7fd221b9a700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2080a14d0 0x7fd2080a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:46:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.417+0000 7fd221b9a700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2080a14d0 0x7fd2080a3990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd21c19a2d0 tx=0x7fd20c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:46:53.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.420+0000 7fd2137fe700 1 -- 172.21.15.89:0/3925501095 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd218052100 con 0x7fd21c1006c0 2023-10-26T06:46:53.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:53 smithi135 ceph-mon[131403]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:53.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:53 smithi135 ceph-mon[131403]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:53 smithi089 ceph-mon[95031]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:53.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:53 smithi089 ceph-mon[95031]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:53 smithi192 ceph-mon[131661]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2023-10-26T06:46:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:53 smithi192 ceph-mon[131661]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:53.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.649+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd21c110640 con 0x7fd2080a14d0 2023-10-26T06:46:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.657+0000 7fd2137fe700 1 -- 172.21.15.89:0/3925501095 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd21c110640 con 0x7fd2080a14d0 2023-10-26T06:46:53.658 INFO:teuthology.orchestra.run.smithi089.stdout:/ 1m 6m3h 2023-10-26T06:46:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.659+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2080a14d0 msgr2=0x7fd2080a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.659+0000 7fd2245ff700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2080a14d0 0x7fd2080a3990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd21c19a2d0 tx=0x7fd20c009260 comp rx=0 tx=0).stop 2023-10-26T06:46:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.659+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c1006c0 msgr2=0x7fd21c198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:46:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.659+0000 7fd2245ff700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c1006c0 0x7fd21c198d80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd21800e9a0 tx=0x7fd21800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 shutdown_connections 2023-10-26T06:46:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2080a14d0 0x7fd2080a3990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd21c1006c0 0x7fd21c198d80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd21c1066d0 0x7fd21c19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 --2- 172.21.15.89:0/3925501095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd21c101080 0x7fd21c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:46:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 >> 172.21.15.89:0/3925501095 conn(0x7fd21c0fc190 msgr2=0x7fd21c0fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:46:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 shutdown_connections 2023-10-26T06:46:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:46:53.660+0000 7fd2245ff700 1 -- 172.21.15.89:0/3925501095 wait complete. 2023-10-26T06:46:53.679 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-10-26T06:46:53.756 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-10-26T06:46:53.768 INFO:teuthology.run_tasks:Running task workunit... 2023-10-26T06:46:53.807 INFO:tasks.workunit:Pulling workunits from ref 384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:46:53.807 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-26T06:46:53.807 INFO:tasks.workunit:timeout=3h 2023-10-26T06:46:53.808 INFO:tasks.workunit:cleanup=True 2023-10-26T06:46:53.808 DEBUG:teuthology.orchestra.run.smithi089:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:53.853 INFO:teuthology.orchestra.run.smithi089.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:53.853 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-26T06:46:53.854 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-10-26T06:46:53.854 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T06:46:53.854 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-26T06:46:53.854 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-10-26 06:45:11.543800388 +0000 2023-10-26T06:46:53.854 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-10-26 06:45:11.543800388 +0000 2023-10-26T06:46:53.855 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-10-26 06:46:42.961859915 +0000 2023-10-26T06:46:53.855 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2023-10-26 06:45:11.543800388 +0000 2023-10-26T06:46:53.855 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-26T06:46:53.855 DEBUG:teuthology.orchestra.run.smithi089:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-26T06:46:53.971 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 384993681abceb8cb8affea10538d607052db4b4 2023-10-26T06:46:54.018 INFO:tasks.workunit.client.0.smithi089.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-26T06:46:54.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:54 smithi089 ceph-mon[95031]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:54.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:54 smithi089 ceph-mon[95031]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:54.290 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:54 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:46:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:54 smithi135 ceph-mon[131403]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:54.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:54 smithi135 ceph-mon[131403]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:54.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:54 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:46:54.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:54 smithi192 ceph-mon[131661]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:54.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:54 smithi192 ceph-mon[131661]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:54.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:54 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:46:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:55 smithi135 ceph-mon[131403]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:55.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:55 smithi135 ceph-mon[131403]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2023-10-26T06:46:55.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:55 smithi135 ceph-mon[131403]: mgrmap e23: x(active, since 7m), standbys: z, y 2023-10-26T06:46:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:55 smithi089 ceph-mon[95031]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:55 smithi089 ceph-mon[95031]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2023-10-26T06:46:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:55 smithi089 ceph-mon[95031]: mgrmap e23: x(active, since 7m), standbys: z, y 2023-10-26T06:46:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:55 smithi192 ceph-mon[131661]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:46:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:55 smithi192 ceph-mon[131661]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2023-10-26T06:46:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:55 smithi192 ceph-mon[131661]: mgrmap e23: x(active, since 7m), standbys: z, y 2023-10-26T06:46:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:57 smithi135 ceph-mon[131403]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-26T06:46:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:57 smithi089 ceph-mon[95031]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-26T06:46:57.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:57 smithi192 ceph-mon[131661]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-26T06:46:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:46:59 smithi135 ceph-mon[131403]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 115 KiB/s wr, 37 op/s 2023-10-26T06:46:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:46:59 smithi089 ceph-mon[95031]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 115 KiB/s wr, 37 op/s 2023-10-26T06:46:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:46:59 smithi192 ceph-mon[131661]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 115 KiB/s wr, 37 op/s 2023-10-26T06:47:01.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:47:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:47:00.746+0000 7f576ff17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:47:01.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:47:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:47:00.746+0000 7f576ff17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:47:01.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:47:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:47:00.746+0000 7f576ff17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:47:01.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:47:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:47:00.746+0000 7f576ff17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:47:01.040 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:47:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:47:00.746+0000 7f576ff17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-26T06:47:01.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:01 smithi135 ceph-mon[131403]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 35 op/s 2023-10-26T06:47:01.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:01 smithi089 ceph-mon[95031]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 35 op/s 2023-10-26T06:47:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:01 smithi192 ceph-mon[131661]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 35 op/s 2023-10-26T06:47:03.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:03 smithi135 ceph-mon[131403]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-26T06:47:03.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:03 smithi135 ceph-mon[131403]: mgrmap e24: x(active, since 8m), standbys: z, y 2023-10-26T06:47:03.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:03 smithi089 ceph-mon[95031]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-26T06:47:03.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:03 smithi089 ceph-mon[95031]: mgrmap e24: x(active, since 8m), standbys: z, y 2023-10-26T06:47:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:03 smithi192 ceph-mon[131661]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 36 op/s 2023-10-26T06:47:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:03 smithi192 ceph-mon[131661]: mgrmap e24: x(active, since 8m), standbys: z, y 2023-10-26T06:47:05.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:05 smithi135 ceph-mon[131403]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 37 op/s 2023-10-26T06:47:05.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:05 smithi089 ceph-mon[95031]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 37 op/s 2023-10-26T06:47:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:05 smithi192 ceph-mon[131661]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 115 KiB/s wr, 37 op/s 2023-10-26T06:47:07.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:07 smithi135 ceph-mon[131403]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 137 KiB/s wr, 48 op/s 2023-10-26T06:47:07.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:07 smithi089 ceph-mon[95031]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 137 KiB/s wr, 48 op/s 2023-10-26T06:47:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:07 smithi192 ceph-mon[131661]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 137 KiB/s wr, 48 op/s 2023-10-26T06:47:09.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:09 smithi135 ceph-mon[131403]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-26T06:47:09.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:09 smithi089 ceph-mon[95031]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-26T06:47:09.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:09 smithi192 ceph-mon[131661]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-26T06:47:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:11 smithi135 ceph-mon[131403]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:11 smithi089 ceph-mon[95031]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:11 smithi192 ceph-mon[131661]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:13.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:13 smithi089 ceph-mon[95031]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:13.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:13 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:47:13.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:13 smithi192 ceph-mon[131661]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:13.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:13 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:47:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:13 smithi135 ceph-mon[131403]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:13 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:47:15.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:15 smithi089 ceph-mon[95031]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-26T06:47:15.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:15 smithi192 ceph-mon[131661]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-26T06:47:15.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:15 smithi135 ceph-mon[131403]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-10-26T06:47:16.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:16 smithi192 ceph-mon[131661]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:16.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:16 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:16 smithi135 ceph-mon[131403]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:16.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:16 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:16 smithi089 ceph-mon[95031]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:47:16.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:16.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:16 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:47:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:47:17.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:17 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:47:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:47:17.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:17 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:17.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:47:17.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:47:17.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:17 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:47:18.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:18 smithi089 ceph-mon[95031]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2023-10-26T06:47:18.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:18 smithi192 ceph-mon[131661]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2023-10-26T06:47:18.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:18 smithi135 ceph-mon[131403]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2023-10-26T06:47:21.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:21 smithi192 ceph-mon[131661]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:21 smithi135 ceph-mon[131403]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:21.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:21 smithi089 ceph-mon[95031]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:22.174 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:47:22.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 -- 172.21.15.89:0/3699718797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 msgr2=0x7fa61410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 --2- 172.21.15.89:0/3699718797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 0x7fa61410f310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b0a0 tx=0x7fa60c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 -- 172.21.15.89:0/3699718797 shutdown_connections 2023-10-26T06:47:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 --2- 172.21.15.89:0/3699718797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 0x7fa61410f310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 --2- 172.21.15.89:0/3699718797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6140ff370 0x7fa6140ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 --2- 172.21.15.89:0/3699718797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ffd20 0x7fa61410c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 -- 172.21.15.89:0/3699718797 >> 172.21.15.89:0/3699718797 conn(0x7fa6140fafb0 msgr2=0x7fa6140fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 -- 172.21.15.89:0/3699718797 shutdown_connections 2023-10-26T06:47:22.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.385+0000 7fa61a39d700 1 -- 172.21.15.89:0/3699718797 wait complete. 2023-10-26T06:47:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.386+0000 7fa61a39d700 1 Processor -- start 2023-10-26T06:47:22.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61a39d700 1 -- start start 2023-10-26T06:47:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61a39d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ff370 0x7fa614198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61a39d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6140ffd20 0x7fa614199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61a39d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 0x7fa61419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61a39d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61419dcf0 con 0x7fa6140ff370 2023-10-26T06:47:22.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61a39d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61419de60 con 0x7fa61410cf20 2023-10-26T06:47:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61a39d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61419e020 con 0x7fa6140ffd20 2023-10-26T06:47:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ff370 0x7fa614198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa619b9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 0x7fa61419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ff370 0x7fa614198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46010/0 (socket says 172.21.15.89:46010) 2023-10-26T06:47:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.387+0000 7fa61939b700 1 -- 172.21.15.89:0/2945037103 learned_addr learned my addr 172.21.15.89:0/2945037103 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:22.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa618b9a700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6140ffd20 0x7fa614199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa619b9c700 1 -- 172.21.15.89:0/2945037103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6140ffd20 msgr2=0x7fa614199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa619b9c700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6140ffd20 0x7fa614199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa619b9c700 1 -- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ff370 msgr2=0x7fa614198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa619b9c700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ff370 0x7fa614198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa619b9c700 1 -- 172.21.15.89:0/2945037103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c009d00 con 0x7fa61410cf20 2023-10-26T06:47:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa618b9a700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6140ffd20 0x7fa614199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa61939b700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ff370 0x7fa614198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa619b9c700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 0x7fa61419d6b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b070 tx=0x7fa60c007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa60a7fc700 1 -- 172.21.15.89:0/2945037103 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00e040 con 0x7fa61410cf20 2023-10-26T06:47:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61419e2a0 con 0x7fa61410cf20 2023-10-26T06:47:22.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.388+0000 7fa60a7fc700 1 -- 172.21.15.89:0/2945037103 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa60c01d470 con 0x7fa61410cf20 2023-10-26T06:47:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.389+0000 7fa60a7fc700 1 -- 172.21.15.89:0/2945037103 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0052a0 con 0x7fa61410cf20 2023-10-26T06:47:22.396 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.389+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6141a5b40 con 0x7fa61410cf20 2023-10-26T06:47:22.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.393+0000 7fa62eec3b80 1 Processor -- start 2023-10-26T06:47:22.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.393+0000 7fa60a7fc700 1 -- 172.21.15.89:0/2945037103 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa60c01d5e0 con 0x7fa61410cf20 2023-10-26T06:47:22.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa62eec3b80 1 -- start start 2023-10-26T06:47:22.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c352226e0 con 0x555c352af820 2023-10-26T06:47:22.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c35208c60 con 0x555c352b4d60 2023-10-26T06:47:22.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c35206580 con 0x555c352b5680 2023-10-26T06:47:22.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa618b9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa618b9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:39776/0 (socket says 172.21.15.89:39776) 2023-10-26T06:47:22.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.394+0000 7fa618b9a700 1 -- 172.21.15.89:0/3870334571 learned_addr learned my addr 172.21.15.89:0/3870334571 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:22.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa618b9a700 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa61939b700 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa618b9a700 1 -- 172.21.15.89:0/3870334571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 msgr2=0x555c352af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa618b9a700 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa618b9a700 1 -- 172.21.15.89:0/3870334571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 msgr2=0x555c352afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa618b9a700 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa618b9a700 1 -- 172.21.15.89:0/3870334571 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c351bf3f0 con 0x555c352b4d60 2023-10-26T06:47:22.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa60a7fc700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6000a15f0 0x7fa6000a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa618b9a700 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b5140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6040095f0 tx=0x7fa604009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ad1f0122eb3486 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa60a7fc700 1 -- 172.21.15.89:0/2945037103 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa60c0ce340 con 0x7fa61410cf20 2023-10-26T06:47:22.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.395+0000 7fa5ff7fe700 1 -- 172.21.15.89:0/3870334571 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604012e40 con 0x555c352b4d60 2023-10-26T06:47:22.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa61939b700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6000a15f0 0x7fa6000a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa5ff7fe700 1 -- 172.21.15.89:0/3870334571 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa60400f910 con 0x555c352b4d60 2023-10-26T06:47:22.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa5ff7fe700 1 -- 172.21.15.89:0/3870334571 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60401e8a0 con 0x555c352b4d60 2023-10-26T06:47:22.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 -- 172.21.15.89:0/3870334571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 msgr2=0x555c352b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa61939b700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6000a15f0 0x7fa6000a3ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x555c352b5ae0 tx=0x7fa610009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b5140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6040095f0 tx=0x7fa604009770 comp rx=0 tx=0).stop 2023-10-26T06:47:22.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 -- 172.21.15.89:0/3870334571 shutdown_connections 2023-10-26T06:47:22.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b5140 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/3870334571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 -- 172.21.15.89:0/3870334571 >> 172.21.15.89:0/3870334571 conn(0x555c35223720 msgr2=0x555c352a9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 -- 172.21.15.89:0/3870334571 shutdown_connections 2023-10-26T06:47:22.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.396+0000 7fa62eec3b80 1 -- 172.21.15.89:0/3870334571 wait complete. 2023-10-26T06:47:22.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.397+0000 7fa62eec3b80 1 -- start start 2023-10-26T06:47:22.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.397+0000 7fa62eec3b80 1 Processor -- start 2023-10-26T06:47:22.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b4d60 0x555c352b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b5680 0x555c352b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c35201e70 con 0x555c352af820 2023-10-26T06:47:22.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3519e710 con 0x555c352b5680 2023-10-26T06:47:22.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3519e3b0 con 0x555c352b4d60 2023-10-26T06:47:22.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa62eec3b80 10 client.0 _ll_register_callbacks cb 0x555c350e8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:22.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa61939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa61939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352b8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46028/0 (socket says 172.21.15.89:46028) 2023-10-26T06:47:22.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.398+0000 7fa61939b700 1 -- 172.21.15.89:0/2076568340 learned_addr learned my addr 172.21.15.89:0/2076568340 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:22.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa619b9c700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b5680 0x555c352b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa618b9a700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b4d60 0x555c352b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa61939b700 1 -- 172.21.15.89:0/2076568340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b4d60 msgr2=0x555c352b8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa61939b700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b4d60 0x555c352b8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa61939b700 1 -- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b5680 msgr2=0x555c352b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa61939b700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b5680 0x555c352b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa61939b700 1 -- 172.21.15.89:0/2076568340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60401a040 con 0x555c352af820 2023-10-26T06:47:22.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa618b9a700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b4d60 0x555c352b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:47:22.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa619b9c700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b5680 0x555c352b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:22.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa61939b700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352b8a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa610011a20 tx=0x7fa6100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dcfcd24f418b00 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.399+0000 7fa5fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:22.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa5fcff9700 1 -- 172.21.15.89:0/2076568340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610017070 con 0x555c352af820 2023-10-26T06:47:22.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c35206ef0 con 0x555c352af820 2023-10-26T06:47:22.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa5fcff9700 1 -- 172.21.15.89:0/2076568340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa610012390 con 0x555c352af820 2023-10-26T06:47:22.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa5fcff9700 1 -- 172.21.15.89:0/2076568340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa610011ea0 con 0x555c352af820 2023-10-26T06:47:22.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa62eec3b80 10 client.15447 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:22.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555c3502cde0 con 0x555c352af820 2023-10-26T06:47:22.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa5fcff9700 1 -- 172.21.15.89:0/2076568340 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c35206ef0 con 0x555c352af820 2023-10-26T06:47:22.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.400+0000 7fa5fcff9700 1 -- 172.21.15.89:0/2076568340 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa610003710 con 0x555c352af820 2023-10-26T06:47:22.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa62eec3b80 10 client.15447 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:22.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa62eec3b80 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:22.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa62eec3b80 20 client.15447 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:22.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c352c9240 0x555c352cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa62eec3b80 4 client.15447 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:22.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c3527e8b0 con 0x555c352c9240 2023-10-26T06:47:22.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa618b9a700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c352c9240 0x555c352cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa5fcff9700 10 client.15447 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:22.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.401+0000 7fa618b9a700 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c352c9240 0x555c352cb660 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315ef922def1a30f server_cookie=c6da73f84480ca26 in_seq=0 out_seq=0 2023-10-26T06:47:22.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.402+0000 7fa5fcff9700 1 -- 172.21.15.89:0/2076568340 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c3527e8b0 con 0x555c352c9240 2023-10-26T06:47:22.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.402+0000 7fa5fcff9700 10 client.15447 handle_command_reply: tid=0 2023-10-26T06:47:22.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.402+0000 7fa62eec3b80 1 client.15447 shutdown 2023-10-26T06:47:22.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.402+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 msgr2=0x555c352b8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.402+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352b8a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa610011a20 tx=0x7fa6100047c0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 shutdown_connections 2023-10-26T06:47:22.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352af820 0x555c352b8a40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c352c9240 0x555c352cb660 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b5680 0x555c352b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2076568340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b4d60 0x555c352b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 >> 172.21.15.89:0/2076568340 conn(0x555c35223720 msgr2=0x555c352ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa5fcff9700 0 client.15447 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:22.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 shutdown_connections 2023-10-26T06:47:22.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2076568340 wait complete. 2023-10-26T06:47:22.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.403+0000 7fa62eec3b80 20 client.15447 trim_cache size 0 max 16384 2023-10-26T06:47:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 Processor -- start 2023-10-26T06:47:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 -- start start 2023-10-26T06:47:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352af820 0x555c352b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b5680 0x555c352a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c35082920 con 0x555c352b5680 2023-10-26T06:47:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c352b9ff0 con 0x555c352b4d60 2023-10-26T06:47:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.420+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c352bbe10 con 0x555c352af820 2023-10-26T06:47:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa618b9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa619b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b5680 0x555c352a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa619b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b5680 0x555c352a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46038/0 (socket says 172.21.15.89:46038) 2023-10-26T06:47:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa619b9c700 1 -- 172.21.15.89:0/2778944269 learned_addr learned my addr 172.21.15.89:0/2778944269 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa61939b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352af820 0x555c352b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa61939b700 1 -- 172.21.15.89:0/2778944269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 msgr2=0x555c352b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa61939b700 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa61939b700 1 -- 172.21.15.89:0/2778944269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b5680 msgr2=0x555c352a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa61939b700 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b5680 0x555c352a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa61939b700 1 -- 172.21.15.89:0/2778944269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c35015c60 con 0x555c352af820 2023-10-26T06:47:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.421+0000 7fa618b9a700 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.422+0000 7fa61939b700 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352af820 0x555c352b2f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa610003bc0 tx=0x7fa610012410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9210cb1ed1c1d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.422+0000 7fa5fffff700 1 -- 172.21.15.89:0/2778944269 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610003680 con 0x555c352af820 2023-10-26T06:47:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.422+0000 7fa5fffff700 1 -- 172.21.15.89:0/2778944269 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6100037e0 con 0x555c352af820 2023-10-26T06:47:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.422+0000 7fa5fffff700 1 -- 172.21.15.89:0/2778944269 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100041f0 con 0x555c352af820 2023-10-26T06:47:22.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2778944269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352af820 msgr2=0x555c352b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352af820 0x555c352b2f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa610003bc0 tx=0x7fa610012410 comp rx=0 tx=0).stop 2023-10-26T06:47:22.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2778944269 shutdown_connections 2023-10-26T06:47:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b5680 0x555c352a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352af820 0x555c352b2f00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/2778944269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352b4d60 0x555c352b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2778944269 >> 172.21.15.89:0/2778944269 conn(0x555c35223720 msgr2=0x555c352b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2778944269 shutdown_connections 2023-10-26T06:47:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 -- 172.21.15.89:0/2778944269 wait complete. 2023-10-26T06:47:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 -- start start 2023-10-26T06:47:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.423+0000 7fa62eec3b80 1 Processor -- start 2023-10-26T06:47:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352af820 0x555c352be070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b4d60 0x555c352be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa62eec3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3531c1c0 con 0x555c352b4d60 2023-10-26T06:47:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3531c330 con 0x555c352af820 2023-10-26T06:47:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa62eec3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3531c4f0 con 0x555c352b5680 2023-10-26T06:47:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa62eec3b80 10 client.0 _ll_register_callbacks cb 0x555c350e8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa618b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b4d60 0x555c352be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa618b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b4d60 0x555c352be5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46052/0 (socket says 172.21.15.89:46052) 2023-10-26T06:47:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa618b9a700 1 -- 172.21.15.89:0/1444230473 learned_addr learned my addr 172.21.15.89:0/1444230473 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa61939b700 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352af820 0x555c352be070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.424+0000 7fa619b9c700 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa618b9a700 1 -- 172.21.15.89:0/1444230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 msgr2=0x555c352beaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa618b9a700 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352beaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa618b9a700 1 -- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352af820 msgr2=0x555c352be070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa618b9a700 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352af820 0x555c352be070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa618b9a700 1 -- 172.21.15.89:0/1444230473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa604016040 con 0x555c352b4d60 2023-10-26T06:47:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa618b9a700 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b4d60 0x555c352be5b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa604009740 tx=0x7fa60400fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24e6c516c1c6c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa5fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa5fcff9700 1 -- 172.21.15.89:0/1444230473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604024040 con 0x555c352b4d60 2023-10-26T06:47:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa5fcff9700 1 -- 172.21.15.89:0/1444230473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa60400f450 con 0x555c352b4d60 2023-10-26T06:47:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa60400f910 con 0x555c352b4d60 2023-10-26T06:47:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa5fcff9700 1 -- 172.21.15.89:0/1444230473 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa604009c50 con 0x555c352b4d60 2023-10-26T06:47:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa62eec3b80 10 client.15453 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.425+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555c3502cde0 con 0x555c352b4d60 2023-10-26T06:47:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa5fcff9700 1 -- 172.21.15.89:0/1444230473 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa604036020 con 0x555c352b4d60 2023-10-26T06:47:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa5fcff9700 1 -- 172.21.15.89:0/1444230473 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa604030070 con 0x555c352b4d60 2023-10-26T06:47:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa62eec3b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa62eec3b80 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa62eec3b80 20 client.15453 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c3531dd40 0x555c35320160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa62eec3b80 4 client.15453 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:47:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555c351fae50 con 0x555c3531dd40 2023-10-26T06:47:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.426+0000 7fa61939b700 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c3531dd40 0x555c35320160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa61939b700 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c3531dd40 0x555c35320160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d219feca090ee3b server_cookie=75367d48c3de5294 in_seq=0 out_seq=0 2023-10-26T06:47:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa5fcff9700 10 client.15453 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa5fcff9700 1 -- 172.21.15.89:0/1444230473 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555c351fae50 con 0x555c3531dd40 2023-10-26T06:47:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa5fcff9700 10 client.15453 handle_command_reply: tid=0 2023-10-26T06:47:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 client.15453 shutdown 2023-10-26T06:47:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b4d60 msgr2=0x555c352be5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b4d60 0x555c352be5b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa604009740 tx=0x7fa60400fa80 comp rx=0 tx=0).stop 2023-10-26T06:47:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 shutdown_connections 2023-10-26T06:47:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c3531dd40 0x555c35320160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c352b5680 0x555c352beaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c352af820 0x555c352be070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa5fcff9700 0 client.15453 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 --2- 172.21.15.89:0/1444230473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c352b4d60 0x555c352be5b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 >> 172.21.15.89:0/1444230473 conn(0x555c35223720 msgr2=0x555c352a3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 shutdown_connections 2023-10-26T06:47:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.427+0000 7fa62eec3b80 1 -- 172.21.15.89:0/1444230473 wait complete. 2023-10-26T06:47:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa62eec3b80 20 client.15453 trim_cache size 0 max 16384 2023-10-26T06:47:22.444 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:47:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:47:22.444 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:47:22.445 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:47:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6000a15f0 msgr2=0x7fa6000a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6000a15f0 0x7fa6000a3ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x555c352b5ae0 tx=0x7fa610009260 comp rx=0 tx=0).stop 2023-10-26T06:47:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 msgr2=0x7fa61419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 0x7fa61419d6b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b070 tx=0x7fa60c007860 comp rx=0 tx=0).stop 2023-10-26T06:47:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 shutdown_connections 2023-10-26T06:47:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6000a15f0 0x7fa6000a3ab0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6140ff370 0x7fa614198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa61410cf20 0x7fa61419d6b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 --2- 172.21.15.89:0/2945037103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6140ffd20 0x7fa614199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 >> 172.21.15.89:0/2945037103 conn(0x7fa6140fafb0 msgr2=0x7fa6140fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 shutdown_connections 2023-10-26T06:47:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.428+0000 7fa61a39d700 1 -- 172.21.15.89:0/2945037103 wait complete. 2023-10-26T06:47:22.449 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:47:22.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:47:22.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 -- 172.21.15.89:0/161628726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 msgr2=0x7f9074072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 --2- 172.21.15.89:0/161628726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f9074072fa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9070009990 tx=0x7f9070009ca0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 -- 172.21.15.89:0/161628726 shutdown_connections 2023-10-26T06:47:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 --2- 172.21.15.89:0/161628726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907410cd30 0x7f907410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 --2- 172.21.15.89:0/161628726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f9074072fa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 --2- 172.21.15.89:0/161628726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90740734e0 0x7f907410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 -- 172.21.15.89:0/161628726 >> 172.21.15.89:0/161628726 conn(0x7f90740fbfe0 msgr2=0x7f90740fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 -- 172.21.15.89:0/161628726 shutdown_connections 2023-10-26T06:47:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.670+0000 7f907b39d700 1 -- 172.21.15.89:0/161628726 wait complete. 2023-10-26T06:47:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 Processor -- start 2023-10-26T06:47:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 -- start start 2023-10-26T06:47:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f907419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90740734e0 0x7f907419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907410cd30 0x7f9074198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9074198ff0 con 0x7f907410cd30 2023-10-26T06:47:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9074199160 con 0x7f90740734e0 2023-10-26T06:47:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.671+0000 7f907b39d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9074199320 con 0x7f9074072bc0 2023-10-26T06:47:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907a39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f907419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907ab9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907410cd30 0x7f9074198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907a39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f907419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:54382/0 (socket says 172.21.15.89:54382) 2023-10-26T06:47:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907a39b700 1 -- 172.21.15.89:0/62598132 learned_addr learned my addr 172.21.15.89:0/62598132 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f9079b9a700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90740734e0 0x7f907419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907ab9c700 1 -- 172.21.15.89:0/62598132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 msgr2=0x7f907419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907ab9c700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f907419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907ab9c700 1 -- 172.21.15.89:0/62598132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90740734e0 msgr2=0x7f907419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907ab9c700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90740734e0 0x7f907419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907ab9c700 1 -- 172.21.15.89:0/62598132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700095f0 con 0x7f907410cd30 2023-10-26T06:47:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907a39b700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f907419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.672+0000 7f907ab9c700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907410cd30 0x7f9074198990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f906c00ef90 tx=0x7f906c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.673+0000 7f906b7fe700 1 -- 172.21.15.89:0/62598132 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c018070 con 0x7f907410cd30 2023-10-26T06:47:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.673+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90741995a0 con 0x7f907410cd30 2023-10-26T06:47:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.673+0000 7f906b7fe700 1 -- 172.21.15.89:0/62598132 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906c00f040 con 0x7f907410cd30 2023-10-26T06:47:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.673+0000 7f906b7fe700 1 -- 172.21.15.89:0/62598132 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c00cc30 con 0x7f907410cd30 2023-10-26T06:47:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.673+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90741a5850 con 0x7f907410cd30 2023-10-26T06:47:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.674+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9074107f40 con 0x7f907410cd30 2023-10-26T06:47:22.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.676+0000 7f906b7fe700 1 -- 172.21.15.89:0/62598132 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f906c003a00 con 0x7f907410cd30 2023-10-26T06:47:22.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.677+0000 7f906b7fe700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90600a15f0 0x7f90600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:22.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.677+0000 7f906b7fe700 1 -- 172.21.15.89:0/62598132 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f906c0ccc30 con 0x7f907410cd30 2023-10-26T06:47:22.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.678+0000 7f907a39b700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90600a15f0 0x7f90600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:22.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.678+0000 7f907a39b700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90600a15f0 0x7f90600a3ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f907000b2e0 tx=0x7f9070005bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.683+0000 7f906b7fe700 1 -- 172.21.15.89:0/62598132 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f906c095fc0 con 0x7f907410cd30 2023-10-26T06:47:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.938+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9074074270 con 0x7f907410cd30 2023-10-26T06:47:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.938+0000 7f906b7fe700 1 -- 172.21.15.89:0/62598132 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f906c0995e0 con 0x7f907410cd30 2023-10-26T06:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:47:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90600a15f0 msgr2=0x7f90600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90600a15f0 0x7f90600a3ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f907000b2e0 tx=0x7f9070005bc0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907410cd30 msgr2=0x7f9074198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907410cd30 0x7f9074198990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f906c00ef90 tx=0x7f906c00c3a0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 shutdown_connections 2023-10-26T06:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f90600a15f0 0x7f90600a3ab0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f907410cd30 0x7f9074198990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9074072bc0 0x7f907419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 --2- 172.21.15.89:0/62598132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90740734e0 0x7f907419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 >> 172.21.15.89:0/62598132 conn(0x7f90740fbfe0 msgr2=0x7f90740fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 shutdown_connections 2023-10-26T06:47:22.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:22.941+0000 7f907b39d700 1 -- 172.21.15.89:0/62598132 wait complete. 2023-10-26T06:47:22.946 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:47:22.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:47:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 -- 172.21.15.89:0/749934396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4101850 msgr2=0x7f56e4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 --2- 172.21.15.89:0/749934396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4101850 0x7f56e4101c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009990 tx=0x7f56d4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 -- 172.21.15.89:0/749934396 shutdown_connections 2023-10-26T06:47:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 --2- 172.21.15.89:0/749934396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4101850 0x7f56e4101c30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 --2- 172.21.15.89:0/749934396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e410ac20 0x7f56e410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 --2- 172.21.15.89:0/749934396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e4102170 0x7f56e410a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 -- 172.21.15.89:0/749934396 >> 172.21.15.89:0/749934396 conn(0x7f56e40fafb0 msgr2=0x7f56e40fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 -- 172.21.15.89:0/749934396 shutdown_connections 2023-10-26T06:47:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.175+0000 7f56e939d700 1 -- 172.21.15.89:0/749934396 wait complete. 2023-10-26T06:47:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.176+0000 7f56e939d700 1 Processor -- start 2023-10-26T06:47:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.176+0000 7f56e939d700 1 -- start start 2023-10-26T06:47:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e939d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e4101850 0x7f56e419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e939d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4102170 0x7f56e419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e939d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e410ac20 0x7f56e4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e939d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41992c0 con 0x7f56e4102170 2023-10-26T06:47:23.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e939d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4199430 con 0x7f56e4101850 2023-10-26T06:47:23.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e939d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41995f0 con 0x7f56e410ac20 2023-10-26T06:47:23.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4102170 0x7f56e419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e4101850 0x7f56e419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4102170 0x7f56e419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46074/0 (socket says 172.21.15.89:46074) 2023-10-26T06:47:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e37fe700 1 -- 172.21.15.89:0/712426096 learned_addr learned my addr 172.21.15.89:0/712426096 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.177+0000 7f56e8b9c700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e410ac20 0x7f56e4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e37fe700 1 -- 172.21.15.89:0/712426096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e410ac20 msgr2=0x7f56e4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e37fe700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e410ac20 0x7f56e4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e37fe700 1 -- 172.21.15.89:0/712426096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e4101850 msgr2=0x7f56e419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e37fe700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e4101850 0x7f56e419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e37fe700 1 -- 172.21.15.89:0/712426096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40095f0 con 0x7f56e4102170 2023-10-26T06:47:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e3fff700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e4101850 0x7f56e419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e37fe700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4102170 0x7f56e419ef90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f56d800d980 tx=0x7f56d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e17fa700 1 -- 172.21.15.89:0/712426096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8004220 con 0x7f56e4102170 2023-10-26T06:47:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e17fa700 1 -- 172.21.15.89:0/712426096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d8007640 con 0x7f56e4102170 2023-10-26T06:47:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e41998d0 con 0x7f56e4102170 2023-10-26T06:47:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e17fa700 1 -- 172.21.15.89:0/712426096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8005690 con 0x7f56e4102170 2023-10-26T06:47:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.178+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41a5a20 con 0x7f56e4102170 2023-10-26T06:47:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.182+0000 7f56e17fa700 1 -- 172.21.15.89:0/712426096 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f56d8004750 con 0x7f56e4102170 2023-10-26T06:47:23.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.183+0000 7f56e17fa700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f56cc0a1670 0x7f56cc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.184+0000 7f56e17fa700 1 -- 172.21.15.89:0/712426096 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f56d80ccdd0 con 0x7f56e4102170 2023-10-26T06:47:23.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.184+0000 7f56e3fff700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f56cc0a1670 0x7f56cc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.184+0000 7f56e3fff700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f56cc0a1670 0x7f56cc0a3b30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56d4006010 tx=0x7f56d400b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:23.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.185+0000 7f56fe020b80 1 Processor -- start 2023-10-26T06:47:23.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.185+0000 7f56fe020b80 1 -- start start 2023-10-26T06:47:23.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003906140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003906680 0x55d0039002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003900820 0x55d003900c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56fe020b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0038736e0 con 0x55d003906680 2023-10-26T06:47:23.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56fe020b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d003859c60 con 0x55d003905d60 2023-10-26T06:47:23.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56fe020b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d003857580 con 0x55d003900820 2023-10-26T06:47:23.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56e37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003906680 0x55d0039002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003906140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003906140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:39862/0 (socket says 172.21.15.89:39862) 2023-10-26T06:47:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56e3fff700 1 -- 172.21.15.89:0/1130593648 learned_addr learned my addr 172.21.15.89:0/1130593648 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.186+0000 7f56e8b9c700 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003900820 0x55d003900c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56e3fff700 1 -- 172.21.15.89:0/1130593648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003900820 msgr2=0x55d003900c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56e3fff700 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003900820 0x55d003900c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56e3fff700 1 -- 172.21.15.89:0/1130593648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003906680 msgr2=0x55d0039002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56e3fff700 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003906680 0x55d0039002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56e3fff700 1 -- 172.21.15.89:0/1130593648 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0038103f0 con 0x55d003905d60 2023-10-26T06:47:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56e3fff700 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003906140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56d4015000 tx=0x7f56d4020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b17cb904231b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56ca7fc700 1 -- 172.21.15.89:0/1130593648 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d402b070 con 0x55d003905d60 2023-10-26T06:47:23.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56ca7fc700 1 -- 172.21.15.89:0/1130593648 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d401e040 con 0x55d003905d60 2023-10-26T06:47:23.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.187+0000 7f56ca7fc700 1 -- 172.21.15.89:0/1130593648 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400fa70 con 0x55d003905d60 2023-10-26T06:47:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 -- 172.21.15.89:0/1130593648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 msgr2=0x55d003906140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003906140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56d4015000 tx=0x7f56d4020000 comp rx=0 tx=0).stop 2023-10-26T06:47:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 -- 172.21.15.89:0/1130593648 shutdown_connections 2023-10-26T06:47:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003900820 0x55d003900c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003906140 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 --2- 172.21.15.89:0/1130593648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003906680 0x55d0039002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 -- 172.21.15.89:0/1130593648 >> 172.21.15.89:0/1130593648 conn(0x55d003874720 msgr2=0x55d0038facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 -- 172.21.15.89:0/1130593648 shutdown_connections 2023-10-26T06:47:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.188+0000 7f56fe020b80 1 -- 172.21.15.89:0/1130593648 wait complete. 2023-10-26T06:47:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.189+0000 7f56fe020b80 1 -- start start 2023-10-26T06:47:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.189+0000 7f56fe020b80 1 Processor -- start 2023-10-26T06:47:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d00390c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003909900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d003909f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56fe020b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d003852e70 con 0x55d003900820 2023-10-26T06:47:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56fe020b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0037ef710 con 0x55d003905d60 2023-10-26T06:47:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56fe020b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0037ef3b0 con 0x55d003906680 2023-10-26T06:47:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56fe020b80 10 client.0 _ll_register_callbacks cb 0x55d003739a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d00390c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d00390c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46098/0 (socket says 172.21.15.89:46098) 2023-10-26T06:47:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56e3fff700 1 -- 172.21.15.89:0/2437535789 learned_addr learned my addr 172.21.15.89:0/2437535789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56e37fe700 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003909900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.190+0000 7f56e8b9c700 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d003909f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56e3fff700 1 -- 172.21.15.89:0/2437535789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 msgr2=0x55d003909f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56e3fff700 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d003909f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56e3fff700 1 -- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 msgr2=0x55d003909900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56e3fff700 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003909900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56e3fff700 1 -- 172.21.15.89:0/2437535789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e410dd80 con 0x55d003900820 2023-10-26T06:47:23.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56e3fff700 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d00390c630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f56d400bb40 tx=0x7f56d4004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a3b44b1e974215 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:23.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:23.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56c37fe700 1 -- 172.21.15.89:0/2437535789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d402b040 con 0x55d003900820 2023-10-26T06:47:23.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56c37fe700 1 -- 172.21.15.89:0/2437535789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f56d400f460 con 0x55d003900820 2023-10-26T06:47:23.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d003857ef0 con 0x55d003900820 2023-10-26T06:47:23.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56c37fe700 1 -- 172.21.15.89:0/2437535789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56d400fb00 con 0x55d003900820 2023-10-26T06:47:23.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56fe020b80 10 client.15471 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:23.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.191+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d00367dde0 con 0x55d003900820 2023-10-26T06:47:23.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56c37fe700 1 -- 172.21.15.89:0/2437535789 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d003857ef0 con 0x55d003900820 2023-10-26T06:47:23.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56c37fe700 1 -- 172.21.15.89:0/2437535789 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56d402f400 con 0x55d003900820 2023-10-26T06:47:23.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56fe020b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:23.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56fe020b80 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:23.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56fe020b80 20 client.15471 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:23.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56fe020b80 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d003919a90 0x55d00391beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56fe020b80 4 client.15471 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:23.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0038cf8b0 con 0x55d003919a90 2023-10-26T06:47:23.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.192+0000 7f56e37fe700 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d003919a90 0x55d00391beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.193+0000 7f56e37fe700 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d003919a90 0x55d00391beb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58205c230881ec1c server_cookie=c122924804dc7769 in_seq=0 out_seq=0 2023-10-26T06:47:23.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.193+0000 7f56c37fe700 10 client.15471 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:23.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56c37fe700 1 -- 172.21.15.89:0/2437535789 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d0038cf8b0 con 0x55d003919a90 2023-10-26T06:47:23.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56c37fe700 10 client.15471 handle_command_reply: tid=0 2023-10-26T06:47:23.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 client.15471 shutdown 2023-10-26T06:47:23.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 msgr2=0x55d00390c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d00390c630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f56d400bb40 tx=0x7f56d4004dc0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 shutdown_connections 2023-10-26T06:47:23.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d00390c630 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d003919a90 0x55d00391beb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d003909f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 --2- 172.21.15.89:0/2437535789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003909900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56c37fe700 0 client.15471 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:23.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 >> 172.21.15.89:0/2437535789 conn(0x55d003874720 msgr2=0x55d0038ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:23.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 shutdown_connections 2023-10-26T06:47:23.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 1 -- 172.21.15.89:0/2437535789 wait complete. 2023-10-26T06:47:23.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.194+0000 7f56fe020b80 20 client.15471 trim_cache size 0 max 16384 2023-10-26T06:47:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 Processor -- start 2023-10-26T06:47:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 -- start start 2023-10-26T06:47:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d003910fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d0038fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d0039081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0036d3920 con 0x55d003900820 2023-10-26T06:47:23.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d003900680 con 0x55d003905d60 2023-10-26T06:47:23.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.209+0000 7f56fe020b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0039114e0 con 0x55d003906680 2023-10-26T06:47:23.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d003910fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e37fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d0038fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d003910fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46110/0 (socket says 172.21.15.89:46110) 2023-10-26T06:47:23.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 -- 172.21.15.89:0/4123237916 learned_addr learned my addr 172.21.15.89:0/4123237916 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:23.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e8b9c700 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d0039081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 -- 172.21.15.89:0/4123237916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 msgr2=0x55d0039081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d0039081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 -- 172.21.15.89:0/4123237916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 msgr2=0x55d0038fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d0038fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 -- 172.21.15.89:0/4123237916 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d003666c60 con 0x55d003900820 2023-10-26T06:47:23.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e8b9c700 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d0039081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e37fe700 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d0038fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56e3fff700 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d003910fa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005f30 tx=0x7f56d400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742f7672df56d5aa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.210+0000 7f56caffd700 1 -- 172.21.15.89:0/4123237916 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d402b040 con 0x55d003900820 2023-10-26T06:47:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56caffd700 1 -- 172.21.15.89:0/4123237916 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d400fdc0 con 0x55d003900820 2023-10-26T06:47:23.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56caffd700 1 -- 172.21.15.89:0/4123237916 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d402fb90 con 0x55d003900820 2023-10-26T06:47:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 -- 172.21.15.89:0/4123237916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 msgr2=0x55d003910fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d003910fa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005f30 tx=0x7f56d400b390 comp rx=0 tx=0).stop 2023-10-26T06:47:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 -- 172.21.15.89:0/4123237916 shutdown_connections 2023-10-26T06:47:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d003910fa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d0039081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 --2- 172.21.15.89:0/4123237916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d0038fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 -- 172.21.15.89:0/4123237916 >> 172.21.15.89:0/4123237916 conn(0x55d003874720 msgr2=0x55d0038faa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 -- 172.21.15.89:0/4123237916 shutdown_connections 2023-10-26T06:47:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.211+0000 7f56fe020b80 1 -- 172.21.15.89:0/4123237916 wait complete. 2023-10-26T06:47:23.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.212+0000 7f56fe020b80 1 -- start start 2023-10-26T06:47:23.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.212+0000 7f56fe020b80 1 Processor -- start 2023-10-26T06:47:23.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.212+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d0039158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.212+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003915e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56fe020b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d00390b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56fe020b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00390b690 con 0x55d003900820 2023-10-26T06:47:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56fe020b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00390b800 con 0x55d003905d60 2023-10-26T06:47:23.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56fe020b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00390b9c0 con 0x55d003906680 2023-10-26T06:47:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56fe020b80 10 client.0 _ll_register_callbacks cb 0x55d003739a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d0039158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d0039158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46126/0 (socket says 172.21.15.89:46126) 2023-10-26T06:47:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 -- 172.21.15.89:0/3229442673 learned_addr learned my addr 172.21.15.89:0/3229442673 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:23.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e8b9c700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d00390b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e37fe700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003915e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 -- 172.21.15.89:0/3229442673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 msgr2=0x55d00390b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d00390b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 -- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 msgr2=0x55d003915e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003915e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e3fff700 1 -- 172.21.15.89:0/3229442673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41a5a20 con 0x55d003900820 2023-10-26T06:47:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.213+0000 7f56e37fe700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003915e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56e3fff700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d0039158e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b6f0 tx=0x7f56d400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a263582bc6ef459 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56c37fe700 1 -- 172.21.15.89:0/3229442673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d402b040 con 0x55d003900820 2023-10-26T06:47:23.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56d401e040 con 0x55d003900820 2023-10-26T06:47:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56c37fe700 1 -- 172.21.15.89:0/3229442673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f56d402fbe0 con 0x55d003900820 2023-10-26T06:47:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56c37fe700 1 -- 172.21.15.89:0/3229442673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56d402fda0 con 0x55d003900820 2023-10-26T06:47:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56fe020b80 10 client.15477 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d00367dde0 con 0x55d003900820 2023-10-26T06:47:23.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.214+0000 7f56c37fe700 1 -- 172.21.15.89:0/3229442673 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d4058020 con 0x55d003900820 2023-10-26T06:47:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56c37fe700 1 -- 172.21.15.89:0/3229442673 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56d4040490 con 0x55d003900820 2023-10-26T06:47:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56fe020b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:23.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56fe020b80 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56fe020b80 20 client.15477 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56fe020b80 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d00396e370 0x55d003970760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56fe020b80 4 client.15477 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:47:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d00384be50 con 0x55d00396e370 2023-10-26T06:47:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.215+0000 7f56e37fe700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d00396e370 0x55d003970760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:23.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56c37fe700 10 client.15477 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56e37fe700 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d00396e370 0x55d003970760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe94b343aec0d79 server_cookie=b793d13534cba376 in_seq=0 out_seq=0 2023-10-26T06:47:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56c37fe700 1 -- 172.21.15.89:0/3229442673 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d00384be50 con 0x55d00396e370 2023-10-26T06:47:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56c37fe700 10 client.15477 handle_command_reply: tid=0 2023-10-26T06:47:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 client.15477 shutdown 2023-10-26T06:47:23.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 msgr2=0x55d0039158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d0039158e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b6f0 tx=0x7f56d400f9b0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 shutdown_connections 2023-10-26T06:47:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d003900820 0x55d0039158e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d00396e370 0x55d003970760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d003906680 0x55d00390b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 --2- 172.21.15.89:0/3229442673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d003905d60 0x55d003915e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56c37fe700 0 client.15477 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:23.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 >> 172.21.15.89:0/3229442673 conn(0x55d003874720 msgr2=0x55d0038fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:23.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 shutdown_connections 2023-10-26T06:47:23.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 1 -- 172.21.15.89:0/3229442673 wait complete. 2023-10-26T06:47:23.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.216+0000 7f56fe020b80 20 client.15477 trim_cache size 0 max 16384 2023-10-26T06:47:23.236 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:47:23.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f56cc0a1670 msgr2=0x7f56cc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f56cc0a1670 0x7f56cc0a3b30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56d4006010 tx=0x7f56d400b260 comp rx=0 tx=0).stop 2023-10-26T06:47:23.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4102170 msgr2=0x7f56e419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:23.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4102170 0x7f56e419ef90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f56d800d980 tx=0x7f56d800dc90 comp rx=0 tx=0).stop 2023-10-26T06:47:23.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 shutdown_connections 2023-10-26T06:47:23.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f56cc0a1670 0x7f56cc0a3b30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56e410ac20 0x7f56e4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56e4101850 0x7f56e419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 --2- 172.21.15.89:0/712426096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56e4102170 0x7f56e419ef90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:23.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 >> 172.21.15.89:0/712426096 conn(0x7f56e40fafb0 msgr2=0x7f56e41004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:23.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 shutdown_connections 2023-10-26T06:47:23.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:23.217+0000 7f56e939d700 1 -- 172.21.15.89:0/712426096 wait complete. 2023-10-26T06:47:23.240 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:23 smithi089 ceph-mon[95031]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:23.240 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:23 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/62598132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:47:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:23 smithi192 ceph-mon[131661]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:23.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:23 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/62598132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:47:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:23 smithi135 ceph-mon[131403]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:23.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:23 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/62598132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:47:24.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:47:24.243 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:47:24.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/100805628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8105170 msgr2=0x7f8ff81055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/100805628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8105170 0x7f8ff81055f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0099c0 tx=0x7f8fec009cd0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/100805628 shutdown_connections 2023-10-26T06:47:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/100805628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff8105b30 0x7f8ff810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/100805628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff8067d40 0x7f8ff8068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/100805628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8105170 0x7f8ff81055f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/100805628 >> 172.21.15.89:0/100805628 conn(0x7f8ff80767f0 msgr2=0x7f8ff8076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/100805628 shutdown_connections 2023-10-26T06:47:24.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.451+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/100805628 wait complete. 2023-10-26T06:47:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.452+0000 7f8ffe5ff700 1 Processor -- start 2023-10-26T06:47:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ffe5ff700 1 -- start start 2023-10-26T06:47:24.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ffe5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8067d40 0x7f8ff80ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ffe5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff8105170 0x7f8ff80ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ffe5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff8105b30 0x7f8ff8100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ffe5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff81a4fd0 con 0x7f8ff8067d40 2023-10-26T06:47:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ffe5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff81a5140 con 0x7f8ff8105b30 2023-10-26T06:47:24.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ffe5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff81a5300 con 0x7f8ff8105170 2023-10-26T06:47:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.453+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8067d40 0x7f8ff80ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8067d40 0x7f8ff80ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46150/0 (socket says 172.21.15.89:46150) 2023-10-26T06:47:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 -- 172.21.15.89:0/1004935253 learned_addr learned my addr 172.21.15.89:0/1004935253 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff8105170 0x7f8ff80ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ffcb9c700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff8105b30 0x7f8ff8100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 -- 172.21.15.89:0/1004935253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff8105170 msgr2=0x7f8ff80ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff8105170 0x7f8ff80ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 -- 172.21.15.89:0/1004935253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff8105b30 msgr2=0x7f8ff8100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff8105b30 0x7f8ff8100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 -- 172.21.15.89:0/1004935253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0095f0 con 0x7f8ff8067d40 2023-10-26T06:47:24.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff8105170 0x7f8ff80ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8067d40 0x7f8ff80ff6b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e9a0 tx=0x7f8fe800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff57fa700 1 -- 172.21.15.89:0/1004935253 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8009790 con 0x7f8ff8067d40 2023-10-26T06:47:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.454+0000 7f8ff57fa700 1 -- 172.21.15.89:0/1004935253 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe8010430 con 0x7f8ff8067d40 2023-10-26T06:47:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.455+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff81a55e0 con 0x7f8ff8067d40 2023-10-26T06:47:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.455+0000 7f8ff57fa700 1 -- 172.21.15.89:0/1004935253 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8018680 con 0x7f8ff8067d40 2023-10-26T06:47:24.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.455+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff81a5b10 con 0x7f8ff8067d40 2023-10-26T06:47:24.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.458+0000 7f8ff57fa700 1 -- 172.21.15.89:0/1004935253 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8fe80105a0 con 0x7f8ff8067d40 2023-10-26T06:47:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.459+0000 7f8ff57fa700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8fe00a1670 0x7f8fe00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.459+0000 7f8ff57fa700 1 -- 172.21.15.89:0/1004935253 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f8fe8014070 con 0x7f8ff8067d40 2023-10-26T06:47:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.460+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8fe00a1670 0x7f8fe00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.460+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8fe00a1670 0x7f8fe00a3b30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009990 tx=0x7f8fec005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.461+0000 7f9011eedb80 1 Processor -- start 2023-10-26T06:47:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.461+0000 7f9011eedb80 1 -- start start 2023-10-26T06:47:24.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2fc60 0x56357cb30040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb30580 0x56357cb2a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2a650 0x56357cb2aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f9011eedb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357ca9d760 con 0x56357cb2fc60 2023-10-26T06:47:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f9011eedb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357ca83ce0 con 0x56357cb30580 2023-10-26T06:47:24.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f9011eedb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357ca81600 con 0x56357cb2a650 2023-10-26T06:47:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2fc60 0x56357cb30040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f8ffcb9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2a650 0x56357cb2aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2fc60 0x56357cb30040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46164/0 (socket says 172.21.15.89:46164) 2023-10-26T06:47:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f8ff7fff700 1 -- 172.21.15.89:0/3445848946 learned_addr learned my addr 172.21.15.89:0/3445848946 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.462+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb30580 0x56357cb2a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ff77fe700 1 -- 172.21.15.89:0/3445848946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2a650 msgr2=0x56357cb2aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2a650 0x56357cb2aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ff77fe700 1 -- 172.21.15.89:0/3445848946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2fc60 msgr2=0x56357cb30040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2fc60 0x56357cb30040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ff77fe700 1 -- 172.21.15.89:0/3445848946 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357ca3a470 con 0x56357cb30580 2023-10-26T06:47:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ffcb9c700 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2a650 0x56357cb2aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2fc60 0x56357cb30040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb30580 0x56357cb2a110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0098e0 tx=0x7f8fec020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba248e7b037cecae server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8fde7fc700 1 -- 172.21.15.89:0/3445848946 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec003880 con 0x56357cb30580 2023-10-26T06:47:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8fde7fc700 1 -- 172.21.15.89:0/3445848946 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fec01e040 con 0x56357cb30580 2023-10-26T06:47:24.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.463+0000 7f8fde7fc700 1 -- 172.21.15.89:0/3445848946 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0107b0 con 0x56357cb30580 2023-10-26T06:47:24.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 -- 172.21.15.89:0/3445848946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb30580 msgr2=0x56357cb2a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb30580 0x56357cb2a110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0098e0 tx=0x7f8fec020000 comp rx=0 tx=0).stop 2023-10-26T06:47:24.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 -- 172.21.15.89:0/3445848946 shutdown_connections 2023-10-26T06:47:24.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2fc60 0x56357cb30040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2a650 0x56357cb2aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3445848946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb30580 0x56357cb2a110 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 -- 172.21.15.89:0/3445848946 >> 172.21.15.89:0/3445848946 conn(0x56357ca9e7a0 msgr2=0x56357cb24be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 -- 172.21.15.89:0/3445848946 shutdown_connections 2023-10-26T06:47:24.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.464+0000 7f9011eedb80 1 -- 172.21.15.89:0/3445848946 wait complete. 2023-10-26T06:47:24.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 -- start start 2023-10-26T06:47:24.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 Processor -- start 2023-10-26T06:47:24.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2a650 0x56357cb33840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2fc60 0x56357cb33d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb30580 0x56357cb342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357ca7cef0 con 0x56357cb2a650 2023-10-26T06:47:24.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357ca19790 con 0x56357cb2fc60 2023-10-26T06:47:24.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357ca19430 con 0x56357cb30580 2023-10-26T06:47:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.465+0000 7f9011eedb80 10 client.0 _ll_register_callbacks cb 0x56357c963b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2a650 0x56357cb33840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2a650 0x56357cb33840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46178/0 (socket says 172.21.15.89:46178) 2023-10-26T06:47:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 -- 172.21.15.89:0/3744493839 learned_addr learned my addr 172.21.15.89:0/3744493839 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2fc60 0x56357cb33d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ffcb9c700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb30580 0x56357cb342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 -- 172.21.15.89:0/3744493839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb30580 msgr2=0x56357cb342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb30580 0x56357cb342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 -- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2fc60 msgr2=0x56357cb33d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2fc60 0x56357cb33d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 -- 172.21.15.89:0/3744493839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff81a5b10 con 0x56357cb2a650 2023-10-26T06:47:24.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ffcb9c700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb30580 0x56357cb342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:24.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2a650 0x56357cb33840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c510 tx=0x7f8fe809a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1337d744214c5b05 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.466+0000 7f8fd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:24.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f8fd77fe700 1 -- 172.21.15.89:0/3744493839 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe809e390 con 0x56357cb2a650 2023-10-26T06:47:24.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f8fd77fe700 1 -- 172.21.15.89:0/3744493839 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f8fe80a25d0 con 0x56357cb2a650 2023-10-26T06:47:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56357ca81f70 con 0x56357cb2a650 2023-10-26T06:47:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f8fd77fe700 1 -- 172.21.15.89:0/3744493839 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe80b82c0 con 0x56357cb2a650 2023-10-26T06:47:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 10 client.15489 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f8ff810de70 con 0x56357cb2a650 2023-10-26T06:47:24.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f8fd77fe700 1 -- 172.21.15.89:0/3744493839 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56357ca81f70 con 0x56357cb2a650 2023-10-26T06:47:24.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f8fd77fe700 1 -- 172.21.15.89:0/3744493839 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8fe80a21d0 con 0x56357cb2a650 2023-10-26T06:47:24.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 10 client.15489 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:24.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:24.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 20 client.15489 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:24.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb43f40 0x56357cb46330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 4 client.15489 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:24.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.467+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56357caf9930 con 0x56357cb43f40 2023-10-26T06:47:24.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.468+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb43f40 0x56357cb46330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.468+0000 7f8fd77fe700 10 client.15489 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.468+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb43f40 0x56357cb46330 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a370407bb130f4 server_cookie=97afc5dfcb1339e3 in_seq=0 out_seq=0 2023-10-26T06:47:24.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f8fd77fe700 1 -- 172.21.15.89:0/3744493839 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56357caf9930 con 0x56357cb43f40 2023-10-26T06:47:24.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f8fd77fe700 10 client.15489 handle_command_reply: tid=0 2023-10-26T06:47:24.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 client.15489 shutdown 2023-10-26T06:47:24.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2a650 msgr2=0x56357cb33840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2a650 0x56357cb33840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c510 tx=0x7f8fe809a9c0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 shutdown_connections 2023-10-26T06:47:24.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb2a650 0x56357cb33840 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb43f40 0x56357cb46330 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb30580 0x56357cb342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 --2- 172.21.15.89:0/3744493839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2fc60 0x56357cb33d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 >> 172.21.15.89:0/3744493839 conn(0x56357ca9e7a0 msgr2=0x56357cb29100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f8fd77fe700 0 client.15489 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.469+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 shutdown_connections 2023-10-26T06:47:24.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.470+0000 7f9011eedb80 1 -- 172.21.15.89:0/3744493839 wait complete. 2023-10-26T06:47:24.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.470+0000 7f9011eedb80 20 client.15489 trim_cache size 0 max 16384 2023-10-26T06:47:24.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.487+0000 7f9011eedb80 1 Processor -- start 2023-10-26T06:47:24.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.487+0000 7f9011eedb80 1 -- start start 2023-10-26T06:47:24.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb32790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb3ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb1ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f9011eedb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357c8fd9a0 con 0x56357cb30580 2023-10-26T06:47:24.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f9011eedb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357cb26660 con 0x56357cb2a650 2023-10-26T06:47:24.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f9011eedb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357cb303d0 con 0x56357cb2fc60 2023-10-26T06:47:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb32790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ffcb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb1ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb32790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:39932/0 (socket says 172.21.15.89:39932) 2023-10-26T06:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 -- 172.21.15.89:0/303396376 learned_addr learned my addr 172.21.15.89:0/303396376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb3ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 -- 172.21.15.89:0/303396376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 msgr2=0x56357cb3ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb3ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 -- 172.21.15.89:0/303396376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 msgr2=0x56357cb1ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb1ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.488+0000 7f8ff7fff700 1 -- 172.21.15.89:0/303396376 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56357c8a7e60 con 0x56357cb2a650 2023-10-26T06:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.489+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb32790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8010c30 tx=0x7f8fe800c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284b667f3b3f4a57 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.489+0000 7f8fdeffd700 1 -- 172.21.15.89:0/303396376 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80a25d0 con 0x56357cb2a650 2023-10-26T06:47:24.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.489+0000 7f8fdeffd700 1 -- 172.21.15.89:0/303396376 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe809e390 con 0x56357cb2a650 2023-10-26T06:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.489+0000 7f8fdeffd700 1 -- 172.21.15.89:0/303396376 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8099950 con 0x56357cb2a650 2023-10-26T06:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 -- 172.21.15.89:0/303396376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 msgr2=0x56357cb32790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb32790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8010c30 tx=0x7f8fe800c480 comp rx=0 tx=0).stop 2023-10-26T06:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 -- 172.21.15.89:0/303396376 shutdown_connections 2023-10-26T06:47:24.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb1ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb32790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 --2- 172.21.15.89:0/303396376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb3ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 -- 172.21.15.89:0/303396376 >> 172.21.15.89:0/303396376 conn(0x56357ca9e7a0 msgr2=0x56357cb1e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 -- 172.21.15.89:0/303396376 shutdown_connections 2023-10-26T06:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 -- 172.21.15.89:0/303396376 wait complete. 2023-10-26T06:47:24.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 -- start start 2023-10-26T06:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.490+0000 7f9011eedb80 1 Processor -- start 2023-10-26T06:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb92550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb92a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f9011eedb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb3a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f9011eedb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357cb3a970 con 0x56357cb30580 2023-10-26T06:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f9011eedb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357cb3aae0 con 0x56357cb2a650 2023-10-26T06:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f9011eedb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357cb3aca0 con 0x56357cb2fc60 2023-10-26T06:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f9011eedb80 10 client.0 _ll_register_callbacks cb 0x56357c963b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb92550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f8ffcb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb3a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f8ff7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb92550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:39936/0 (socket says 172.21.15.89:39936) 2023-10-26T06:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f8ff7fff700 1 -- 172.21.15.89:0/4046357899 learned_addr learned my addr 172.21.15.89:0/4046357899 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.491+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb92a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8ff7fff700 1 -- 172.21.15.89:0/4046357899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 msgr2=0x56357cb92a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb92a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8ff7fff700 1 -- 172.21.15.89:0/4046357899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 msgr2=0x56357cb3a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb3a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8ff7fff700 1 -- 172.21.15.89:0/4046357899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff810de70 con 0x56357cb2a650 2023-10-26T06:47:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8ffcb9c700 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb3a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8ff7fff700 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb92550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8012040 tx=0x7f8fe8010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d44f5e55fc53d2b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8fd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:47:24.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8fd77fe700 1 -- 172.21.15.89:0/4046357899 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8099930 con 0x56357cb2a650 2023-10-26T06:47:24.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8fd77fe700 1 -- 172.21.15.89:0/4046357899 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f8fe80a2140 con 0x56357cb2a650 2023-10-26T06:47:24.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fec01e040 con 0x56357cb2a650 2023-10-26T06:47:24.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f8fd77fe700 1 -- 172.21.15.89:0/4046357899 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe8099400 con 0x56357cb2a650 2023-10-26T06:47:24.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f9011eedb80 10 client.25505 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:24.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.492+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56357c890ce0 con 0x56357cb2a650 2023-10-26T06:47:24.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f8fd77fe700 1 -- 172.21.15.89:0/4046357899 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fec01e040 con 0x56357cb2a650 2023-10-26T06:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f8fd77fe700 1 -- 172.21.15.89:0/4046357899 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8fe800ca10 con 0x56357cb2a650 2023-10-26T06:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f9011eedb80 10 client.25505 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f9011eedb80 10 client.25505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f9011eedb80 20 client.25505 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:24.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f9011eedb80 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb9a000 0x56357cb9c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f9011eedb80 4 client.25505 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56357ca75ed0 con 0x56357cb9a000 2023-10-26T06:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.493+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb9a000 0x56357cb9c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.494+0000 7f8ff77fe700 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb9a000 0x56357cb9c3f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80b14cc7edc064c server_cookie=65319662204b4e5f in_seq=0 out_seq=0 2023-10-26T06:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.494+0000 7f8fd77fe700 10 client.25505 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f8fd77fe700 1 -- 172.21.15.89:0/4046357899 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56357ca75ed0 con 0x56357cb9a000 2023-10-26T06:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f8fd77fe700 10 client.25505 handle_command_reply: tid=0 2023-10-26T06:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 client.25505 shutdown 2023-10-26T06:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 msgr2=0x56357cb92550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb92550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8012040 tx=0x7f8fe8010a90 comp rx=0 tx=0).stop 2023-10-26T06:47:24.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 shutdown_connections 2023-10-26T06:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56357cb30580 0x56357cb3a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56357cb9a000 0x56357cb9c3f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56357cb2a650 0x56357cb92550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 --2- 172.21.15.89:0/4046357899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357cb2fc60 0x56357cb92a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 >> 172.21.15.89:0/4046357899 conn(0x56357ca9e7a0 msgr2=0x56357cb30960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f8fd77fe700 0 client.25505 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.495+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 shutdown_connections 2023-10-26T06:47:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f9011eedb80 1 -- 172.21.15.89:0/4046357899 wait complete. 2023-10-26T06:47:24.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f9011eedb80 20 client.25505 trim_cache size 0 max 16384 2023-10-26T06:47:24.510 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:47:24.511 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:47:24.511 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "2317d627-2957-4ba4-9364-f39152ce69b4", 2023-10-26T06:47:24.511 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:47:24.511 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:47:24.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8fe00a1670 msgr2=0x7f8fe00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8fe00a1670 0x7f8fe00a3b30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009990 tx=0x7f8fec005fd0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8067d40 msgr2=0x7f8ff80ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8067d40 0x7f8ff80ff6b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e9a0 tx=0x7f8fe800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 shutdown_connections 2023-10-26T06:47:24.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8fe00a1670 0x7f8fe00a3b30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ff8067d40 0x7f8ff80ff6b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff8105b30 0x7f8ff8100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.496+0000 7f8ffe5ff700 1 --2- 172.21.15.89:0/1004935253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff8105170 0x7f8ff80ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.497+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 >> 172.21.15.89:0/1004935253 conn(0x7f8ff80767f0 msgr2=0x7f8ff80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.497+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 shutdown_connections 2023-10-26T06:47:24.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.497+0000 7f8ffe5ff700 1 -- 172.21.15.89:0/1004935253 wait complete. 2023-10-26T06:47:24.518 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:47:24.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.732+0000 7fbb3259d700 1 -- 172.21.15.89:0/1393485010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 msgr2=0x7fbb2c10c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.732+0000 7fbb3259d700 1 --2- 172.21.15.89:0/1393485010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c10c360 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400b0d0 tx=0x7fbb2400b3e0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.733+0000 7fbb3259d700 1 -- 172.21.15.89:0/1393485010 shutdown_connections 2023-10-26T06:47:24.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.733+0000 7fbb3259d700 1 --2- 172.21.15.89:0/1393485010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c10c360 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.733+0000 7fbb3259d700 1 --2- 172.21.15.89:0/1393485010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2c1055a0 0x7fbb2c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.733+0000 7fbb3259d700 1 --2- 172.21.15.89:0/1393485010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb2c0ff4e0 0x7fbb2c0ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.733+0000 7fbb3259d700 1 -- 172.21.15.89:0/1393485010 >> 172.21.15.89:0/1393485010 conn(0x7fbb2c0fafb0 msgr2=0x7fbb2c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.733+0000 7fbb3259d700 1 -- 172.21.15.89:0/1393485010 shutdown_connections 2023-10-26T06:47:24.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.733+0000 7fbb3259d700 1 -- 172.21.15.89:0/1393485010 wait complete. 2023-10-26T06:47:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 Processor -- start 2023-10-26T06:47:24.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 -- start start 2023-10-26T06:47:24.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2c0ff4e0 0x7fbb2c1062d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c106810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb2c1055a0 0x7fbb2c106d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c1a9390 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c1a94d0 con 0x7fbb2c0ff4e0 2023-10-26T06:47:24.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.734+0000 7fbb3259d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c1a9640 con 0x7fbb2c1055a0 2023-10-26T06:47:24.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46214/0 (socket says 172.21.15.89:46214) 2023-10-26T06:47:24.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 -- 172.21.15.89:0/953028461 learned_addr learned my addr 172.21.15.89:0/953028461 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb3159b700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2c0ff4e0 0x7fbb2c1062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb2c1055a0 0x7fbb2c106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 -- 172.21.15.89:0/953028461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb2c1055a0 msgr2=0x7fbb2c106d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb2c1055a0 0x7fbb2c106d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 -- 172.21.15.89:0/953028461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2c0ff4e0 msgr2=0x7fbb2c1062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2c0ff4e0 0x7fbb2c1062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb30d9a700 1 -- 172.21.15.89:0/953028461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb24009d00 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.735+0000 7fbb3159b700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2c0ff4e0 0x7fbb2c1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:24.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.736+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c106810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00d950 tx=0x7fbb1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.736+0000 7fbb227fc700 1 -- 172.21.15.89:0/953028461 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00f890 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.736+0000 7fbb227fc700 1 -- 172.21.15.89:0/953028461 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb1c010460 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.736+0000 7fbb227fc700 1 -- 172.21.15.89:0/953028461 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00e610 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.736+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c1a9920 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.736+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c1a9de0 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.740+0000 7fbb227fc700 1 -- 172.21.15.89:0/953028461 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbb1c0108d0 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.740+0000 7fbb4715cb80 1 Processor -- start 2023-10-26T06:47:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb4715cb80 1 -- start start 2023-10-26T06:47:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800e680 0x5653280082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327f7b6e0 con 0x56532800dd60 2023-10-26T06:47:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327f61c60 con 0x565328008820 2023-10-26T06:47:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327f5f580 con 0x56532800e680 2023-10-26T06:47:24.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb31d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb31d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46218/0 (socket says 172.21.15.89:46218) 2023-10-26T06:47:24.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb31d9c700 1 -- 172.21.15.89:0/1593872741 learned_addr learned my addr 172.21.15.89:0/1593872741 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.741+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb3159b700 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800e680 0x5653280082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb31d9c700 1 -- 172.21.15.89:0/1593872741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800e680 msgr2=0x5653280082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800e680 0x5653280082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb31d9c700 1 -- 172.21.15.89:0/1593872741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 msgr2=0x565328008c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328008c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb31d9c700 1 -- 172.21.15.89:0/1593872741 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565327f183f0 con 0x56532800dd60 2023-10-26T06:47:24.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800e140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009b80 tx=0x7fbb24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abbbdcacc652159 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb227fc700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb180a15f0 0x7fbb180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb227fc700 1 -- 172.21.15.89:0/953028461 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbb1c0ccec0 con 0x7fbb2c0ffea0 2023-10-26T06:47:24.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.742+0000 7fbb177fe700 1 -- 172.21.15.89:0/1593872741 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400e050 con 0x56532800dd60 2023-10-26T06:47:24.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.743+0000 7fbb177fe700 1 -- 172.21.15.89:0/1593872741 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb24007e70 con 0x56532800dd60 2023-10-26T06:47:24.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.743+0000 7fbb177fe700 1 -- 172.21.15.89:0/1593872741 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24007970 con 0x56532800dd60 2023-10-26T06:47:24.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.743+0000 7fbb3159b700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb180a15f0 0x7fbb180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.743+0000 7fbb4715cb80 1 -- 172.21.15.89:0/1593872741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 msgr2=0x56532800e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.743+0000 7fbb3159b700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb180a15f0 0x7fbb180a3ab0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x56532800eae0 tx=0x7fbb280073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.743+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800e140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009b80 tx=0x7fbb24009bb0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 -- 172.21.15.89:0/1593872741 shutdown_connections 2023-10-26T06:47:24.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800e140 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328008c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/1593872741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800e680 0x5653280082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 -- 172.21.15.89:0/1593872741 >> 172.21.15.89:0/1593872741 conn(0x565327f7c720 msgr2=0x565328002cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 -- 172.21.15.89:0/1593872741 shutdown_connections 2023-10-26T06:47:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 -- 172.21.15.89:0/1593872741 wait complete. 2023-10-26T06:47:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.744+0000 7fbb4715cb80 1 -- start start 2023-10-26T06:47:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.745+0000 7fbb4715cb80 1 Processor -- start 2023-10-26T06:47:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.745+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328011ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.745+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800dd60 0x565328012020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.745+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800e680 0x565328012560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.745+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327f5ae70 con 0x56532800e680 2023-10-26T06:47:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.745+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327ef7710 con 0x565328008820 2023-10-26T06:47:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.745+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327ef73b0 con 0x56532800dd60 2023-10-26T06:47:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb4715cb80 10 client.0 _ll_register_callbacks cb 0x565327e41a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800e680 0x565328012560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800e680 0x565328012560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46224/0 (socket says 172.21.15.89:46224) 2023-10-26T06:47:24.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 -- 172.21.15.89:0/382931893 learned_addr learned my addr 172.21.15.89:0/382931893 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb3159b700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328011ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800dd60 0x565328012020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 -- 172.21.15.89:0/382931893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800dd60 msgr2=0x565328012020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800dd60 0x565328012020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 -- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 msgr2=0x565328011ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328011ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 -- 172.21.15.89:0/382931893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c067830 con 0x56532800e680 2023-10-26T06:47:24.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb3159b700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328011ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:24.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800e680 0x565328012560 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400b420 tx=0x7fbb24007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193eef9280c99365 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.746+0000 7fbb14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:24.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb14ff9700 1 -- 172.21.15.89:0/382931893 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24027070 con 0x56532800e680 2023-10-26T06:47:24.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565327f5fef0 con 0x56532800e680 2023-10-26T06:47:24.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb14ff9700 1 -- 172.21.15.89:0/382931893 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbb24003c10 con 0x56532800e680 2023-10-26T06:47:24.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb14ff9700 1 -- 172.21.15.89:0/382931893 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb24007cf0 con 0x56532800e680 2023-10-26T06:47:24.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb4715cb80 10 client.15507 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:24.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x565327d85de0 con 0x56532800e680 2023-10-26T06:47:24.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb14ff9700 1 -- 172.21.15.89:0/382931893 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565327f5fef0 con 0x56532800e680 2023-10-26T06:47:24.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.747+0000 7fbb14ff9700 1 -- 172.21.15.89:0/382931893 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbb24021020 con 0x56532800e680 2023-10-26T06:47:24.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb4715cb80 10 client.15507 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:24.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb4715cb80 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:24.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb4715cb80 20 client.15507 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:24.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328021bf0 0x565328023fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb4715cb80 4 client.15507 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:24.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565327fd78b0 con 0x565328021bf0 2023-10-26T06:47:24.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb3159b700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328021bf0 0x565328023fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb14ff9700 10 client.15507 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.748+0000 7fbb3159b700 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328021bf0 0x565328023fe0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a51a0c240dd10e8 server_cookie=5661e78511c75aa6 in_seq=0 out_seq=0 2023-10-26T06:47:24.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.749+0000 7fbb14ff9700 1 -- 172.21.15.89:0/382931893 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565327fd78b0 con 0x565328021bf0 2023-10-26T06:47:24.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.749+0000 7fbb14ff9700 10 client.15507 handle_command_reply: tid=0 2023-10-26T06:47:24.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.749+0000 7fbb4715cb80 1 client.15507 shutdown 2023-10-26T06:47:24.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.749+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800e680 msgr2=0x565328012560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.749+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800e680 0x565328012560 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400b420 tx=0x7fbb24007ad0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.749+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 shutdown_connections 2023-10-26T06:47:24.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800e680 0x565328012560 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328021bf0 0x565328023fe0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565328008820 0x565328011ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb14ff9700 0 client.15507 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/382931893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56532800dd60 0x565328012020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 >> 172.21.15.89:0/382931893 conn(0x565327f7c720 msgr2=0x5653280072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 shutdown_connections 2023-10-26T06:47:24.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 1 -- 172.21.15.89:0/382931893 wait complete. 2023-10-26T06:47:24.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.750+0000 7fbb4715cb80 20 client.15507 trim_cache size 0 max 16384 2023-10-26T06:47:24.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 Processor -- start 2023-10-26T06:47:24.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 -- start start 2023-10-26T06:47:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 0x56532800d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327ffdd40 con 0x56532800dd60 2023-10-26T06:47:24.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x565327ddb920 con 0x56532800e680 2023-10-26T06:47:24.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.769+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653280047b0 con 0x565328008820 2023-10-26T06:47:24.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46236/0 (socket says 172.21.15.89:46236) 2023-10-26T06:47:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 -- 172.21.15.89:0/2615324019 learned_addr learned my addr 172.21.15.89:0/2615324019 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb3159b700 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb31d9c700 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 0x56532800d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 -- 172.21.15.89:0/2615324019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 msgr2=0x565328019030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 -- 172.21.15.89:0/2615324019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 msgr2=0x56532800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 0x56532800d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 -- 172.21.15.89:0/2615324019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565327d6ec60 con 0x56532800dd60 2023-10-26T06:47:24.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800cb20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c096380 tx=0x7fbb1c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbf81c344c34d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb17fff700 1 -- 172.21.15.89:0/2615324019 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0d0460 con 0x56532800dd60 2023-10-26T06:47:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb17fff700 1 -- 172.21.15.89:0/2615324019 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb1c0999c0 con 0x56532800dd60 2023-10-26T06:47:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.770+0000 7fbb17fff700 1 -- 172.21.15.89:0/2615324019 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c09a0e0 con 0x56532800dd60 2023-10-26T06:47:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 -- 172.21.15.89:0/2615324019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 msgr2=0x56532800cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800cb20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c096380 tx=0x7fbb1c000bc0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 -- 172.21.15.89:0/2615324019 shutdown_connections 2023-10-26T06:47:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 0x56532800d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/2615324019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x56532800cb20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 -- 172.21.15.89:0/2615324019 >> 172.21.15.89:0/2615324019 conn(0x565327f7c720 msgr2=0x565328011370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 -- 172.21.15.89:0/2615324019 shutdown_connections 2023-10-26T06:47:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.771+0000 7fbb4715cb80 1 -- 172.21.15.89:0/2615324019 wait complete. 2023-10-26T06:47:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 -- start start 2023-10-26T06:47:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 Processor -- start 2023-10-26T06:47:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x565328017560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 0x565328017aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565328018180 con 0x56532800dd60 2023-10-26T06:47:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653280182f0 con 0x56532800e680 2023-10-26T06:47:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565328075990 con 0x565328008820 2023-10-26T06:47:24.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.772+0000 7fbb4715cb80 10 client.0 _ll_register_callbacks cb 0x565327e41a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x565328017560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x565328017560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46250/0 (socket says 172.21.15.89:46250) 2023-10-26T06:47:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb3159b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 -- 172.21.15.89:0/950110529 learned_addr learned my addr 172.21.15.89:0/950110529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:24.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 -- 172.21.15.89:0/950110529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 msgr2=0x565328019070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 -- 172.21.15.89:0/950110529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 msgr2=0x565328017aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 0x565328017aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 -- 172.21.15.89:0/950110529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c1a9de0 con 0x56532800dd60 2023-10-26T06:47:24.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb30d9a700 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x565328017560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c010fd0 tx=0x7fbb1c098430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cecdfbb531e61b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb14ff9700 1 -- 172.21.15.89:0/950110529 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c099570 con 0x56532800dd60 2023-10-26T06:47:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb14ff9700 1 -- 172.21.15.89:0/950110529 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbb1c099e80 con 0x56532800dd60 2023-10-26T06:47:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.773+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb24007e70 con 0x56532800dd60 2023-10-26T06:47:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb14ff9700 1 -- 172.21.15.89:0/950110529 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1c0caa80 con 0x56532800dd60 2023-10-26T06:47:24.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 10 client.15513 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x565327dac250 con 0x56532800dd60 2023-10-26T06:47:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb14ff9700 1 -- 172.21.15.89:0/950110529 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb24007e70 con 0x56532800dd60 2023-10-26T06:47:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb14ff9700 1 -- 172.21.15.89:0/950110529 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbb1c0a1d80 con 0x56532800dd60 2023-10-26T06:47:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 10 client.15513 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:24.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 20 client.15513 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328076c30 0x565328079020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 4 client.15513 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:47:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.774+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565327f53e50 con 0x565328076c30 2023-10-26T06:47:24.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.775+0000 7fbb3159b700 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328076c30 0x565328079020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:24.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.775+0000 7fbb14ff9700 10 client.15513 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.775+0000 7fbb3159b700 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328076c30 0x565328079020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944328b939449fb server_cookie=235257b2d8ae58a0 in_seq=0 out_seq=0 2023-10-26T06:47:24.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.775+0000 7fbb14ff9700 1 -- 172.21.15.89:0/950110529 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x565327f53e50 con 0x565328076c30 2023-10-26T06:47:24.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.775+0000 7fbb14ff9700 10 client.15513 handle_command_reply: tid=0 2023-10-26T06:47:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.775+0000 7fbb4715cb80 1 client.15513 shutdown 2023-10-26T06:47:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 msgr2=0x565328017560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x565328017560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c010fd0 tx=0x7fbb1c098430 comp rx=0 tx=0).stop 2023-10-26T06:47:24.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 shutdown_connections 2023-10-26T06:47:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565328076c30 0x565328079020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56532800e680 0x565328017aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565328008820 0x565328019070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 --2- 172.21.15.89:0/950110529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56532800dd60 0x565328017560 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 >> 172.21.15.89:0/950110529 conn(0x565327f7c720 msgr2=0x5653280123c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb14ff9700 0 client.15513 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:24.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 shutdown_connections 2023-10-26T06:47:24.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 1 -- 172.21.15.89:0/950110529 wait complete. 2023-10-26T06:47:24.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.776+0000 7fbb4715cb80 20 client.15513 trim_cache size 0 max 16384 2023-10-26T06:47:24.793 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:47:24.793 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:47:24.793 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:47:24.793 INFO:teuthology.orchestra.run.smithi089.stdout: "2317d627-2957-4ba4-9364-f39152ce69b4": { 2023-10-26T06:47:24.793 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:47:24.794 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "2317d627-2957-4ba4-9364-f39152ce69b4", 2023-10-26T06:47:24.794 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:47:24.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:47:24.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:47:24.794 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:47:24.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb180a15f0 msgr2=0x7fbb180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb180a15f0 0x7fbb180a3ab0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x56532800eae0 tx=0x7fbb280073d0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 msgr2=0x7fbb2c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c106810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00d950 tx=0x7fbb1c00dc60 comp rx=0 tx=0).stop 2023-10-26T06:47:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 shutdown_connections 2023-10-26T06:47:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbb180a15f0 0x7fbb180a3ab0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb2c1055a0 0x7fbb2c106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb2c0ff4e0 0x7fbb2c1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 --2- 172.21.15.89:0/953028461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb2c0ffea0 0x7fbb2c106810 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 >> 172.21.15.89:0/953028461 conn(0x7fbb2c0fafb0 msgr2=0x7fbb2c0fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 shutdown_connections 2023-10-26T06:47:24.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:24.777+0000 7fbb3259d700 1 -- 172.21.15.89:0/953028461 wait complete. 2023-10-26T06:47:24.798 INFO:tasks.cephfs.filesystem:scrub status for tag:2317d627-2957-4ba4-9364-f39152ce69b4 - {'path': '/', 'tag': '2317d627-2957-4ba4-9364-f39152ce69b4', 'options': 'recursive,force'} 2023-10-26T06:47:25.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:25 smithi192 ceph-mon[131661]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:47:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:25 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:47:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:25 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:25.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:25 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:47:25.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:25 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:25 smithi135 ceph-mon[131403]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:47:25.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:25 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:47:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:25 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:25 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:47:25.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:25 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:25.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:25 smithi089 ceph-mon[95031]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:47:25.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:25 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:47:25.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:25 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:25.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:25 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:47:25.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:25 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:27.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:27 smithi192 ceph-mon[131661]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:27.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:27 smithi135 ceph-mon[131403]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:27.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:27 smithi089 ceph-mon[95031]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:29 smithi192 ceph-mon[131661]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 853 B/s wr, 2 op/s 2023-10-26T06:47:29.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:29 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:47:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:29 smithi135 ceph-mon[131403]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 853 B/s wr, 2 op/s 2023-10-26T06:47:29.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:29 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:47:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:29 smithi089 ceph-mon[95031]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 853 B/s wr, 2 op/s 2023-10-26T06:47:29.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:29 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:47:31.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:31 smithi192 ceph-mon[131661]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 853 B/s wr, 1 op/s 2023-10-26T06:47:31.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:31 smithi135 ceph-mon[131403]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 853 B/s wr, 1 op/s 2023-10-26T06:47:31.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:31 smithi089 ceph-mon[95031]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 853 B/s wr, 1 op/s 2023-10-26T06:47:33.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:33 smithi192 ceph-mon[131661]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:33.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:33 smithi135 ceph-mon[131403]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:33.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:33 smithi089 ceph-mon[95031]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:35.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:35 smithi192 ceph-mon[131661]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:35.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:35 smithi135 ceph-mon[131403]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:35.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:35 smithi089 ceph-mon[95031]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:37 smithi192 ceph-mon[131661]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:37.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:37 smithi135 ceph-mon[131403]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:37.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:37 smithi089 ceph-mon[95031]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:39 smithi192 ceph-mon[131661]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:39 smithi135 ceph-mon[131403]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:39.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:39 smithi089 ceph-mon[95031]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:47:41.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:41 smithi192 ceph-mon[131661]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:41 smithi135 ceph-mon[131403]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:41.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:41 smithi089 ceph-mon[95031]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:43 smithi192 ceph-mon[131661]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:43 smithi135 ceph-mon[131403]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:43.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:43 smithi089 ceph-mon[95031]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 1 op/s 2023-10-26T06:47:45.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:45 smithi192 ceph-mon[131661]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2023-10-26T06:47:45.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:45 smithi135 ceph-mon[131403]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2023-10-26T06:47:45.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:45 smithi089 ceph-mon[95031]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2023-10-26T06:47:47.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:47 smithi192 ceph-mon[131661]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:47 smithi135 ceph-mon[131403]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:47 smithi089 ceph-mon[95031]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:49.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:49 smithi192 ceph-mon[131661]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:47:49.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:49 smithi135 ceph-mon[131403]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:47:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:49 smithi089 ceph-mon[95031]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:47:51.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:51 smithi192 ceph-mon[131661]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:51.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:51 smithi135 ceph-mon[131403]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:51 smithi089 ceph-mon[95031]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:53.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:53 smithi192 ceph-mon[131661]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:53.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:53 smithi135 ceph-mon[131403]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:53 smithi089 ceph-mon[95031]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:54.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:47:55.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 -- 172.21.15.89:0/1779956564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c072c20 msgr2=0x7f368c073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 --2- 172.21.15.89:0/1779956564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c072c20 0x7f368c073000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3688009990 tx=0x7f3688009ca0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 -- 172.21.15.89:0/1779956564 shutdown_connections 2023-10-26T06:47:55.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 --2- 172.21.15.89:0/1779956564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c072c20 0x7f368c073000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 --2- 172.21.15.89:0/1779956564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f368c10cf60 0x7f368c10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 --2- 172.21.15.89:0/1779956564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f368c073540 0x7f368c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 -- 172.21.15.89:0/1779956564 >> 172.21.15.89:0/1779956564 conn(0x7f368c0fc1b0 msgr2=0x7f368c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 -- 172.21.15.89:0/1779956564 shutdown_connections 2023-10-26T06:47:55.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.004+0000 7f3693dff700 1 -- 172.21.15.89:0/1779956564 wait complete. 2023-10-26T06:47:55.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.006+0000 7f3693dff700 1 Processor -- start 2023-10-26T06:47:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.006+0000 7f3693dff700 1 -- start start 2023-10-26T06:47:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.006+0000 7f3693dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f368c072c20 0x7f368c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.006+0000 7f3693dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c073540 0x7f368c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3693dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f368c10cf60 0x7f368c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3693dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c19dce0 con 0x7f368c073540 2023-10-26T06:47:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3693dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c19de50 con 0x7f368c10cf60 2023-10-26T06:47:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3693dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c19e010 con 0x7f368c072c20 2023-10-26T06:47:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f369139a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c073540 0x7f368c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3691b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f368c072c20 0x7f368c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f369139a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c073540 0x7f368c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49254/0 (socket says 172.21.15.89:49254) 2023-10-26T06:47:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f369139a700 1 -- 172.21.15.89:0/1027208133 learned_addr learned my addr 172.21.15.89:0/1027208133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f369239c700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f368c10cf60 0x7f368c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3691b9b700 1 -- 172.21.15.89:0/1027208133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f368c10cf60 msgr2=0x7f368c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3691b9b700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f368c10cf60 0x7f368c19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3691b9b700 1 -- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c073540 msgr2=0x7f368c199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3691b9b700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c073540 0x7f368c199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f3691b9b700 1 -- 172.21.15.89:0/1027208133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c009520 con 0x7f368c072c20 2023-10-26T06:47:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.007+0000 7f369139a700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c073540 0x7f368c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.008+0000 7f3691b9b700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f368c072c20 0x7f368c198dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36880098e0 tx=0x7f3688005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.008+0000 7f3682ffd700 1 -- 172.21.15.89:0/1027208133 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368801d070 con 0x7f368c072c20 2023-10-26T06:47:55.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.008+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36880095f0 con 0x7f368c072c20 2023-10-26T06:47:55.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.008+0000 7f3682ffd700 1 -- 172.21.15.89:0/1027208133 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3688022470 con 0x7f368c072c20 2023-10-26T06:47:55.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.008+0000 7f3682ffd700 1 -- 172.21.15.89:0/1027208133 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688017650 con 0x7f368c072c20 2023-10-26T06:47:55.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.008+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1a5950 con 0x7f368c072c20 2023-10-26T06:47:55.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.011+0000 7f3682ffd700 1 -- 172.21.15.89:0/1027208133 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f36880225e0 con 0x7f368c072c20 2023-10-26T06:47:55.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.012+0000 7f36a7814b80 1 Processor -- start 2023-10-26T06:47:55.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.012+0000 7f3682ffd700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36780a15f0 0x7f36780a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f36a7814b80 1 -- start start 2023-10-26T06:47:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f3682ffd700 1 -- 172.21.15.89:0/1027208133 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f36880ce380 con 0x7f368c072c20 2023-10-26T06:47:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb564be0 0x562ceb564fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb55f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb55fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f36a7814b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb4d26e0 con 0x562ceb565500 2023-10-26T06:47:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f36a7814b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb4b8c60 con 0x562ceb55f5d0 2023-10-26T06:47:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f36a7814b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb4b6580 con 0x562ceb564be0 2023-10-26T06:47:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f369239c700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36780a15f0 0x7f36780a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f369139a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb55f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f3691b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb564be0 0x562ceb564fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f3691b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb564be0 0x562ceb564fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50108/0 (socket says 172.21.15.89:50108) 2023-10-26T06:47:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.013+0000 7f3691b9b700 1 -- 172.21.15.89:0/4128205219 learned_addr learned my addr 172.21.15.89:0/4128205219 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369239c700 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb55fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369139a700 1 -- 172.21.15.89:0/4128205219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb564be0 msgr2=0x562ceb564fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369139a700 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb564be0 0x562ceb564fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369139a700 1 -- 172.21.15.89:0/4128205219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 msgr2=0x562ceb55fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369139a700 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb55fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369139a700 1 -- 172.21.15.89:0/4128205219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ceb46f3f0 con 0x562ceb565500 2023-10-26T06:47:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369239c700 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb55fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:55.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369239c700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36780a15f0 0x7f36780a3ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f368400f540 tx=0x7f3684009920 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f369139a700 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb55f090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f367c0117e0 tx=0x7f367c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf8fa1227f2e1f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f3677fff700 1 -- 172.21.15.89:0/4128205219 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c004020 con 0x562ceb565500 2023-10-26T06:47:55.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f3677fff700 1 -- 172.21.15.89:0/4128205219 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f367c004830 con 0x562ceb565500 2023-10-26T06:47:55.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.014+0000 7f3677fff700 1 -- 172.21.15.89:0/4128205219 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c0053e0 con 0x562ceb565500 2023-10-26T06:47:55.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 -- 172.21.15.89:0/4128205219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 msgr2=0x562ceb55f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb55f090 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f367c0117e0 tx=0x7f367c011af0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 -- 172.21.15.89:0/4128205219 shutdown_connections 2023-10-26T06:47:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb55fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb564be0 0x562ceb564fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 --2- 172.21.15.89:0/4128205219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb55f090 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 -- 172.21.15.89:0/4128205219 >> 172.21.15.89:0/4128205219 conn(0x562ceb4d3720 msgr2=0x562ceb559b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 -- 172.21.15.89:0/4128205219 shutdown_connections 2023-10-26T06:47:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.015+0000 7f36a7814b80 1 -- 172.21.15.89:0/4128205219 wait complete. 2023-10-26T06:47:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 -- start start 2023-10-26T06:47:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 Processor -- start 2023-10-26T06:47:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb56b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb568760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb568ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb4b1e70 con 0x562ceb565500 2023-10-26T06:47:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb44e710 con 0x562ceb564be0 2023-10-26T06:47:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb44e3b0 con 0x562ceb55f5d0 2023-10-26T06:47:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.016+0000 7f36a7814b80 10 client.0 _ll_register_callbacks cb 0x562ceb398a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb568ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb568ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49270/0 (socket says 172.21.15.89:49270) 2023-10-26T06:47:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 -- 172.21.15.89:0/1073800987 learned_addr learned my addr 172.21.15.89:0/1073800987 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369139a700 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb568760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f3691b9b700 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb56b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 -- 172.21.15.89:0/1073800987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 msgr2=0x562ceb56b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb56b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 -- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 msgr2=0x562ceb568760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb568760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 -- 172.21.15.89:0/1073800987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c011440 con 0x562ceb565500 2023-10-26T06:47:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f369239c700 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb568ea0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3684019f20 tx=0x7f3684004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d113d55c15ed278b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f36757fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.017+0000 7f36757fa700 1 -- 172.21.15.89:0/1073800987 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368400e040 con 0x562ceb565500 2023-10-26T06:47:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36757fa700 1 -- 172.21.15.89:0/1073800987 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3684003a00 con 0x562ceb565500 2023-10-26T06:47:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ceb4b6ef0 con 0x562ceb565500 2023-10-26T06:47:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36757fa700 1 -- 172.21.15.89:0/1073800987 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3684003bc0 con 0x562ceb565500 2023-10-26T06:47:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 10 client.15525 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562ceb2dcde0 con 0x562ceb565500 2023-10-26T06:47:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36757fa700 1 -- 172.21.15.89:0/1073800987 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ceb4b6ef0 con 0x562ceb565500 2023-10-26T06:47:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36757fa700 1 -- 172.21.15.89:0/1073800987 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3684021480 con 0x562ceb565500 2023-10-26T06:47:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 20 client.15525 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb578f00 0x562ceb57b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 4 client.15525 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.018+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ceb52e8b0 con 0x562ceb578f00 2023-10-26T06:47:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.019+0000 7f3691b9b700 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb578f00 0x562ceb57b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.019+0000 7f36757fa700 10 client.15525 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.019+0000 7f3691b9b700 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb578f00 0x562ceb57b320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ebd3e4e39a7bde server_cookie=96aed063032d4bb9 in_seq=0 out_seq=0 2023-10-26T06:47:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36757fa700 1 -- 172.21.15.89:0/1073800987 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ceb52e8b0 con 0x562ceb578f00 2023-10-26T06:47:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36757fa700 10 client.15525 handle_command_reply: tid=0 2023-10-26T06:47:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 client.15525 shutdown 2023-10-26T06:47:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 msgr2=0x562ceb568ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb568ea0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3684019f20 tx=0x7f3684004b40 comp rx=0 tx=0).stop 2023-10-26T06:47:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 shutdown_connections 2023-10-26T06:47:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb568ea0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb578f00 0x562ceb57b320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb56b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 --2- 172.21.15.89:0/1073800987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb568760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 >> 172.21.15.89:0/1073800987 conn(0x562ceb4d3720 msgr2=0x562ceb55e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36757fa700 0 client.15525 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 shutdown_connections 2023-10-26T06:47:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.020+0000 7f36a7814b80 1 -- 172.21.15.89:0/1073800987 wait complete. 2023-10-26T06:47:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.021+0000 7f36a7814b80 20 client.15525 trim_cache size 0 max 16384 2023-10-26T06:47:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.038+0000 7f36a7814b80 1 Processor -- start 2023-10-26T06:47:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.038+0000 7f36a7814b80 1 -- start start 2023-10-26T06:47:55.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb5545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb559810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f36a7814b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb332920 con 0x562ceb565500 2023-10-26T06:47:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f36a7814b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb5703f0 con 0x562ceb564be0 2023-10-26T06:47:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f36a7814b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb566cb0 con 0x562ceb55f5d0 2023-10-26T06:47:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb5545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb5545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50124/0 (socket says 172.21.15.89:50124) 2023-10-26T06:47:55.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 -- 172.21.15.89:0/663392412 learned_addr learned my addr 172.21.15.89:0/663392412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f369239c700 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb559810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f369139a700 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 -- 172.21.15.89:0/663392412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 msgr2=0x562ceb56feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 -- 172.21.15.89:0/663392412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 msgr2=0x562ceb559810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb559810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.039+0000 7f3691b9b700 1 -- 172.21.15.89:0/663392412 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ceb2c5c60 con 0x562ceb55f5d0 2023-10-26T06:47:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.040+0000 7f3691b9b700 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb5545d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3688000c00 tx=0x7f3688022c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6629fdfb05d157 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.040+0000 7f3680bff700 1 -- 172.21.15.89:0/663392412 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880b7070 con 0x562ceb55f5d0 2023-10-26T06:47:55.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.040+0000 7f3680bff700 1 -- 172.21.15.89:0/663392412 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f368809ad30 con 0x562ceb55f5d0 2023-10-26T06:47:55.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.040+0000 7f3680bff700 1 -- 172.21.15.89:0/663392412 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880b2070 con 0x562ceb55f5d0 2023-10-26T06:47:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 -- 172.21.15.89:0/663392412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 msgr2=0x562ceb5545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb5545d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3688000c00 tx=0x7f3688022c60 comp rx=0 tx=0).stop 2023-10-26T06:47:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 -- 172.21.15.89:0/663392412 shutdown_connections 2023-10-26T06:47:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb565500 0x562ceb559810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb55f5d0 0x562ceb5545d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 --2- 172.21.15.89:0/663392412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 -- 172.21.15.89:0/663392412 >> 172.21.15.89:0/663392412 conn(0x562ceb4d3720 msgr2=0x562ceb567ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 -- 172.21.15.89:0/663392412 shutdown_connections 2023-10-26T06:47:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 -- 172.21.15.89:0/663392412 wait complete. 2023-10-26T06:47:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 -- start start 2023-10-26T06:47:55.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.041+0000 7f36a7814b80 1 Processor -- start 2023-10-26T06:47:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb56f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f36a7814b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb565500 0x562ceb56e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f36a7814b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb56e770 con 0x562ceb564be0 2023-10-26T06:47:55.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f36a7814b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb56e8e0 con 0x562ceb55f5d0 2023-10-26T06:47:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f36a7814b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb56eaa0 con 0x562ceb565500 2023-10-26T06:47:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f36a7814b80 10 client.0 _ll_register_callbacks cb 0x562ceb398a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f3691b9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb56f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f369139a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f3691b9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb56f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:54470/0 (socket says 172.21.15.89:54470) 2023-10-26T06:47:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f3691b9b700 1 -- 172.21.15.89:0/2864234672 learned_addr learned my addr 172.21.15.89:0/2864234672 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f369239c700 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb565500 0x562ceb56e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f3691b9b700 1 -- 172.21.15.89:0/2864234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb565500 msgr2=0x562ceb56e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f3691b9b700 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb565500 0x562ceb56e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f3691b9b700 1 -- 172.21.15.89:0/2864234672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb564be0 msgr2=0x562ceb56feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.042+0000 7f3691b9b700 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f3691b9b700 1 -- 172.21.15.89:0/2864234672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1a5950 con 0x562ceb55f5d0 2023-10-26T06:47:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f369139a700 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f3691b9b700 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb56f970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f368809b360 tx=0x7f368800b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770c5538392e4e31 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36757fa700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:47:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36757fa700 1 -- 172.21.15.89:0/2864234672 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880b7030 con 0x562ceb55f5d0 2023-10-26T06:47:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f367c004830 con 0x562ceb55f5d0 2023-10-26T06:47:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36757fa700 1 -- 172.21.15.89:0/2864234672 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f36880b8070 con 0x562ceb55f5d0 2023-10-26T06:47:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36757fa700 1 -- 172.21.15.89:0/2864234672 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36880a36d0 con 0x562ceb55f5d0 2023-10-26T06:47:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36a7814b80 10 client.25538 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:55.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562ceb303250 con 0x562ceb55f5d0 2023-10-26T06:47:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.043+0000 7f36757fa700 1 -- 172.21.15.89:0/2864234672 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3688081020 con 0x562ceb55f5d0 2023-10-26T06:47:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f36757fa700 1 -- 172.21.15.89:0/2864234672 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f368809b630 con 0x562ceb55f5d0 2023-10-26T06:47:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f36a7814b80 10 client.25538 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f36a7814b80 10 client.25538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f36a7814b80 20 client.25538 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:55.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f36a7814b80 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb5cdda0 0x562ceb5d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f36a7814b80 4 client.25538 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:47:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ceb4aae50 con 0x562ceb5cdda0 2023-10-26T06:47:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.044+0000 7f369139a700 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb5cdda0 0x562ceb5d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36757fa700 10 client.25538 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f369139a700 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb5cdda0 0x562ceb5d01c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe7a483da28eeaf server_cookie=c9545a91dcbe4335 in_seq=0 out_seq=0 2023-10-26T06:47:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36757fa700 1 -- 172.21.15.89:0/2864234672 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562ceb4aae50 con 0x562ceb5cdda0 2023-10-26T06:47:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36757fa700 10 client.25538 handle_command_reply: tid=0 2023-10-26T06:47:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 client.25538 shutdown 2023-10-26T06:47:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 msgr2=0x562ceb56f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb56f970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f368809b360 tx=0x7f368800b6b0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 shutdown_connections 2023-10-26T06:47:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x562ceb5cdda0 0x562ceb5d01c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ceb565500 0x562ceb56e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36757fa700 0 client.25538 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ceb55f5d0 0x562ceb56f970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 --2- 172.21.15.89:0/2864234672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562ceb564be0 0x562ceb56feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 >> 172.21.15.89:0/2864234672 conn(0x562ceb4d3720 msgr2=0x562ceb562720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 shutdown_connections 2023-10-26T06:47:55.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 1 -- 172.21.15.89:0/2864234672 wait complete. 2023-10-26T06:47:55.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.045+0000 7f36a7814b80 20 client.25538 trim_cache size 0 max 16384 2023-10-26T06:47:55.061 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:47:55.061 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:47:55.061 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:47:55.061 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:47:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36780a15f0 msgr2=0x7f36780a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36780a15f0 0x7f36780a3ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f368400f540 tx=0x7f3684009920 comp rx=0 tx=0).stop 2023-10-26T06:47:55.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f368c072c20 msgr2=0x7f368c198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f368c072c20 0x7f368c198dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36880098e0 tx=0x7f3688005740 comp rx=0 tx=0).stop 2023-10-26T06:47:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 shutdown_connections 2023-10-26T06:47:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36780a15f0 0x7f36780a3ab0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f368c10cf60 0x7f368c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f368c072c20 0x7f368c198dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 --2- 172.21.15.89:0/1027208133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c073540 0x7f368c199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 >> 172.21.15.89:0/1027208133 conn(0x7f368c0fc1b0 msgr2=0x7f368c0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 shutdown_connections 2023-10-26T06:47:55.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.046+0000 7f3693dff700 1 -- 172.21.15.89:0/1027208133 wait complete. 2023-10-26T06:47:55.066 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:47:55.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:47:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3863055811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa8073540 msgr2=0x7f7fa810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3863055811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa8073540 0x7f7fa810c9a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009990 tx=0x7f7f9c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3863055811 shutdown_connections 2023-10-26T06:47:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3863055811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fa810cee0 0x7f7fa810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3863055811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa8072c20 0x7f7fa8073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3863055811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa8073540 0x7f7fa810c9a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3863055811 >> 172.21.15.89:0/3863055811 conn(0x7f7fa80fc1d0 msgr2=0x7f7fa80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3863055811 shutdown_connections 2023-10-26T06:47:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.269+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3863055811 wait complete. 2023-10-26T06:47:55.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 Processor -- start 2023-10-26T06:47:55.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 -- start start 2023-10-26T06:47:55.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa8072c20 0x7f7fa8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fa8073540 0x7f7fa8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa810cee0 0x7f7fa819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa819dcf0 con 0x7f7fa810cee0 2023-10-26T06:47:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa819de60 con 0x7f7fa8073540 2023-10-26T06:47:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.270+0000 7f7fad7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa819e020 con 0x7f7fa8072c20 2023-10-26T06:47:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa810cee0 0x7f7fa819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa810cee0 0x7f7fa819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49324/0 (socket says 172.21.15.89:49324) 2023-10-26T06:47:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 -- 172.21.15.89:0/3089827666 learned_addr learned my addr 172.21.15.89:0/3089827666 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa67fc700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fa8073540 0x7f7fa8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa6ffd700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa8072c20 0x7f7fa8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 -- 172.21.15.89:0/3089827666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa8072c20 msgr2=0x7f7fa8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa8072c20 0x7f7fa8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 -- 172.21.15.89:0/3089827666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fa8073540 msgr2=0x7f7fa8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fa8073540 0x7f7fa8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 -- 172.21.15.89:0/3089827666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0009c40 con 0x7f7fa810cee0 2023-10-26T06:47:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.271+0000 7f7fa77fe700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa810cee0 0x7f7fa819d6b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c370 tx=0x7f7fa000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.272+0000 7f7f8ffff700 1 -- 172.21.15.89:0/3089827666 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0017070 con 0x7f7fa810cee0 2023-10-26T06:47:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.272+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c0095f0 con 0x7f7fa810cee0 2023-10-26T06:47:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.272+0000 7f7f8ffff700 1 -- 172.21.15.89:0/3089827666 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fa000f040 con 0x7f7fa810cee0 2023-10-26T06:47:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.272+0000 7f7f8ffff700 1 -- 172.21.15.89:0/3089827666 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000e050 con 0x7f7fa810cee0 2023-10-26T06:47:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.272+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa81a5960 con 0x7f7fa810cee0 2023-10-26T06:47:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.273+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa81080f0 con 0x7f7fa810cee0 2023-10-26T06:47:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.275+0000 7f7f8ffff700 1 -- 172.21.15.89:0/3089827666 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7fa000cb60 con 0x7f7fa810cee0 2023-10-26T06:47:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.276+0000 7f7f8ffff700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7f900a15f0 0x7f7f900a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.276+0000 7f7f8ffff700 1 -- 172.21.15.89:0/3089827666 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7fa00cca70 con 0x7f7fa810cee0 2023-10-26T06:47:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.276+0000 7f7fa6ffd700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7f900a15f0 0x7f7f900a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.277+0000 7f7fa6ffd700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7f900a15f0 0x7f7f900a3ab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009c60 tx=0x7f7f98009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.285+0000 7f7f8ffff700 1 -- 172.21.15.89:0/3089827666 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7fa0095d70 con 0x7f7fa810cee0 2023-10-26T06:47:55.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:55 smithi192 ceph-mon[131661]: pgmap v340: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:47:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:55 smithi135 ceph-mon[131403]: pgmap v340: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:47:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:55 smithi089 ceph-mon[95031]: pgmap v340: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:47:55.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.541+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fa80742d0 con 0x7f7fa810cee0 2023-10-26T06:47:55.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.542+0000 7f7f8ffff700 1 -- 172.21.15.89:0/3089827666 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f7fa0099390 con 0x7f7fa810cee0 2023-10-26T06:47:55.543 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:47:55.544 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:47:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.544+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7f900a15f0 msgr2=0x7f7f900a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.544+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7f900a15f0 0x7f7f900a3ab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009c60 tx=0x7f7f98009260 comp rx=0 tx=0).stop 2023-10-26T06:47:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.544+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa810cee0 msgr2=0x7f7fa819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.544+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa810cee0 0x7f7fa819d6b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c370 tx=0x7f7fa000c680 comp rx=0 tx=0).stop 2023-10-26T06:47:55.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 shutdown_connections 2023-10-26T06:47:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7f900a15f0 0x7f7f900a3ab0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fa810cee0 0x7f7fa819d6b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa8072c20 0x7f7fa8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 --2- 172.21.15.89:0/3089827666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fa8073540 0x7f7fa8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 >> 172.21.15.89:0/3089827666 conn(0x7f7fa80fc1d0 msgr2=0x7f7fa80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 shutdown_connections 2023-10-26T06:47:55.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.545+0000 7f7fad7ff700 1 -- 172.21.15.89:0/3089827666 wait complete. 2023-10-26T06:47:55.548 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:47:55.565 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:47:55.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 -- 172.21.15.89:0/138788679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c810cf60 msgr2=0x7f96c810f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/138788679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c810cf60 0x7f96c810f3d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96c000b0a0 tx=0x7f96c000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 -- 172.21.15.89:0/138788679 shutdown_connections 2023-10-26T06:47:55.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/138788679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c8072c20 0x7f96c8073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/138788679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c810cf60 0x7f96c810f3d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/138788679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96c8073540 0x7f96c810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 -- 172.21.15.89:0/138788679 >> 172.21.15.89:0/138788679 conn(0x7f96c80fc1b0 msgr2=0x7f96c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 -- 172.21.15.89:0/138788679 shutdown_connections 2023-10-26T06:47:55.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.772+0000 7f96ce5ff700 1 -- 172.21.15.89:0/138788679 wait complete. 2023-10-26T06:47:55.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.773+0000 7f96ce5ff700 1 Processor -- start 2023-10-26T06:47:55.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.773+0000 7f96ce5ff700 1 -- start start 2023-10-26T06:47:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ce5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8072c20 0x7f96c8198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ce5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96c8073540 0x7f96c8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ce5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c810cf60 0x7f96c819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ce5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c819de70 con 0x7f96c810cf60 2023-10-26T06:47:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ce5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c819dfe0 con 0x7f96c8073540 2023-10-26T06:47:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ce5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c819e1a0 con 0x7f96c8072c20 2023-10-26T06:47:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8072c20 0x7f96c8198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8072c20 0x7f96c8198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50162/0 (socket says 172.21.15.89:50162) 2023-10-26T06:47:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 -- 172.21.15.89:0/3629137542 learned_addr learned my addr 172.21.15.89:0/3629137542 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c810cf60 0x7f96c819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c77fe700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96c8073540 0x7f96c8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 -- 172.21.15.89:0/3629137542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96c8073540 msgr2=0x7f96c8199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96c8073540 0x7f96c8199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 -- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c810cf60 msgr2=0x7f96c819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c810cf60 0x7f96c819d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96c7fff700 1 -- 172.21.15.89:0/3629137542 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0009d00 con 0x7f96c8072c20 2023-10-26T06:47:55.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.774+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c810cf60 0x7f96c819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:55.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.775+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8072c20 0x7f96c8198f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96b800e9a0 tx=0x7f96b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.775+0000 7f96c57fa700 1 -- 172.21.15.89:0/3629137542 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8009790 con 0x7f96c8072c20 2023-10-26T06:47:55.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.775+0000 7f96c57fa700 1 -- 172.21.15.89:0/3629137542 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b8010430 con 0x7f96c8072c20 2023-10-26T06:47:55.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.775+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c819e480 con 0x7f96c8072c20 2023-10-26T06:47:55.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.775+0000 7f96c57fa700 1 -- 172.21.15.89:0/3629137542 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8018680 con 0x7f96c8072c20 2023-10-26T06:47:55.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.775+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c81a5cc0 con 0x7f96c8072c20 2023-10-26T06:47:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.778+0000 7f96c57fa700 1 -- 172.21.15.89:0/3629137542 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f96b80105a0 con 0x7f96c8072c20 2023-10-26T06:47:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.779+0000 7f96e1fc1b80 1 Processor -- start 2023-10-26T06:47:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.779+0000 7f96e1fc1b80 1 -- start start 2023-10-26T06:47:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96c57fa700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96b00a1670 0x7f96b00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96c57fa700 1 -- 172.21.15.89:0/3629137542 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f96b8014070 con 0x7f96c8072c20 2023-10-26T06:47:55.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa6ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228f9de6e0 con 0x56228fa71500 2023-10-26T06:47:55.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228f9c4c60 con 0x56228fa70be0 2023-10-26T06:47:55.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228f9c2580 con 0x56228fa6b5d0 2023-10-26T06:47:55.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96b00a1670 0x7f96b00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96c77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa70fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:54506/0 (socket says 172.21.15.89:54506) 2023-10-26T06:47:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96c7fff700 1 -- 172.21.15.89:0/3621212067 learned_addr learned my addr 172.21.15.89:0/3621212067 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.780+0000 7f96ccb9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa6ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96c7fff700 1 -- 172.21.15.89:0/3621212067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 msgr2=0x56228fa6ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3621212067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa6ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96c7fff700 1 -- 172.21.15.89:0/3621212067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 msgr2=0x56228fa6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3621212067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa6b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96c7fff700 1 -- 172.21.15.89:0/3621212067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228f97b3f0 con 0x56228fa70be0 2023-10-26T06:47:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3621212067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa70fc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96b8096c80 tx=0x7f96b8099e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd84281208cf42e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96b00a1670 0x7f96b00a3b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009610 tx=0x7f96c00093a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96ae7fc700 1 -- 172.21.15.89:0/3621212067 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b809e160 con 0x56228fa70be0 2023-10-26T06:47:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96ae7fc700 1 -- 172.21.15.89:0/3621212067 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b80b2070 con 0x56228fa70be0 2023-10-26T06:47:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.781+0000 7f96ae7fc700 1 -- 172.21.15.89:0/3621212067 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b809a6b0 con 0x56228fa70be0 2023-10-26T06:47:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3621212067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 msgr2=0x56228fa70fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3621212067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa70fc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96b8096c80 tx=0x7f96b8099e80 comp rx=0 tx=0).stop 2023-10-26T06:47:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3621212067 shutdown_connections 2023-10-26T06:47:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3621212067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa6ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3621212067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa70fc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3621212067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3621212067 >> 172.21.15.89:0/3621212067 conn(0x56228f9df720 msgr2=0x56228fa65b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3621212067 shutdown_connections 2023-10-26T06:47:55.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.782+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3621212067 wait complete. 2023-10-26T06:47:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.783+0000 7f96e1fc1b80 1 -- start start 2023-10-26T06:47:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.783+0000 7f96e1fc1b80 1 Processor -- start 2023-10-26T06:47:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.783+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa77490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.783+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa74780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa74e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228f9bde70 con 0x56228fa71500 2023-10-26T06:47:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228f95a710 con 0x56228fa70be0 2023-10-26T06:47:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228f95a3b0 con 0x56228fa6b5d0 2023-10-26T06:47:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96e1fc1b80 10 client.0 _ll_register_callbacks cb 0x56228f8a4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa74e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa74e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49362/0 (socket says 172.21.15.89:49362) 2023-10-26T06:47:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 -- 172.21.15.89:0/3210327511 learned_addr learned my addr 172.21.15.89:0/3210327511 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96c77fe700 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa74780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa77490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 -- 172.21.15.89:0/3210327511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 msgr2=0x56228fa77490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa77490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 -- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 msgr2=0x56228fa74780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa74780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 -- 172.21.15.89:0/3210327511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c81a5cc0 con 0x56228fa71500 2023-10-26T06:47:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa74e00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96c000b0a0 tx=0x7f96c000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d06e6de2c3925e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.784+0000 7f96a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96a77fe700 1 -- 172.21.15.89:0/3210327511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0030070 con 0x56228fa71500 2023-10-26T06:47:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56228f9c2ef0 con 0x56228fa71500 2023-10-26T06:47:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96a77fe700 1 -- 172.21.15.89:0/3210327511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f96c0007940 con 0x56228fa71500 2023-10-26T06:47:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96a77fe700 1 -- 172.21.15.89:0/3210327511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96c0026070 con 0x56228fa71500 2023-10-26T06:47:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96e1fc1b80 10 client.15549 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56228f7e8de0 con 0x56228fa71500 2023-10-26T06:47:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96a77fe700 1 -- 172.21.15.89:0/3210327511 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56228f9c2ef0 con 0x56228fa71500 2023-10-26T06:47:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.785+0000 7f96a77fe700 1 -- 172.21.15.89:0/3210327511 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f96c0033020 con 0x56228fa71500 2023-10-26T06:47:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96e1fc1b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96e1fc1b80 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96e1fc1b80 20 client.15549 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fa84f30 0x56228fa87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96e1fc1b80 4 client.15549 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56228fa3a8b0 con 0x56228fa84f30 2023-10-26T06:47:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fa84f30 0x56228fa87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fa84f30 0x56228fa87350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a821474fa128822 server_cookie=cf6f6862f8a23ae2 in_seq=0 out_seq=0 2023-10-26T06:47:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.786+0000 7f96a77fe700 10 client.15549 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.787+0000 7f96a77fe700 1 -- 172.21.15.89:0/3210327511 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56228fa3a8b0 con 0x56228fa84f30 2023-10-26T06:47:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.787+0000 7f96a77fe700 10 client.15549 handle_command_reply: tid=0 2023-10-26T06:47:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.787+0000 7f96e1fc1b80 1 client.15549 shutdown 2023-10-26T06:47:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.787+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 msgr2=0x56228fa74e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.787+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa74e00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96c000b0a0 tx=0x7f96c000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 shutdown_connections 2023-10-26T06:47:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa71500 0x56228fa74e00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fa84f30 0x56228fa87350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa77490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3210327511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa70be0 0x56228fa74780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96a77fe700 0 client.15549 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 >> 172.21.15.89:0/3210327511 conn(0x56228f9df720 msgr2=0x56228fa6a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 shutdown_connections 2023-10-26T06:47:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3210327511 wait complete. 2023-10-26T06:47:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.788+0000 7f96e1fc1b80 20 client.15549 trim_cache size 0 max 16384 2023-10-26T06:47:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.801+0000 7f96e1fc1b80 1 Processor -- start 2023-10-26T06:47:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.801+0000 7f96e1fc1b80 1 -- start start 2023-10-26T06:47:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228f83e920 con 0x56228fa70be0 2023-10-26T06:47:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228fa675e0 con 0x56228fa71500 2023-10-26T06:47:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228fa819c0 con 0x56228fa6b5d0 2023-10-26T06:47:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49368/0 (socket says 172.21.15.89:49368) 2023-10-26T06:47:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 -- 172.21.15.89:0/3698524257 learned_addr learned my addr 172.21.15.89:0/3698524257 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:50198/0 (socket says 172.21.15.89:50198) 2023-10-26T06:47:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 -- 172.21.15.89:0/3698524257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 msgr2=0x56228fa605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 -- 172.21.15.89:0/3698524257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 msgr2=0x56228fa658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c77fe700 1 -- 172.21.15.89:0/3698524257 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228f7d1c60 con 0x56228fa70be0 2023-10-26T06:47:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.802+0000 7f96c7fff700 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.803+0000 7f96c77fe700 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7bee0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f96c819a4a0 tx=0x7f96bc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea8ea59ecc392f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.803+0000 7f96aeffd700 1 -- 172.21.15.89:0/3698524257 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc016030 con 0x56228fa70be0 2023-10-26T06:47:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.803+0000 7f96aeffd700 1 -- 172.21.15.89:0/3698524257 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96bc017040 con 0x56228fa70be0 2023-10-26T06:47:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.803+0000 7f96aeffd700 1 -- 172.21.15.89:0/3698524257 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc003d60 con 0x56228fa70be0 2023-10-26T06:47:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3698524257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 msgr2=0x56228fa7bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7bee0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f96c819a4a0 tx=0x7f96bc011720 comp rx=0 tx=0).stop 2023-10-26T06:47:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3698524257 shutdown_connections 2023-10-26T06:47:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/3698524257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7bee0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3698524257 >> 172.21.15.89:0/3698524257 conn(0x56228f9df720 msgr2=0x56228fa6a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3698524257 shutdown_connections 2023-10-26T06:47:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/3698524257 wait complete. 2023-10-26T06:47:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.804+0000 7f96e1fc1b80 1 -- start start 2023-10-26T06:47:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 1 Processor -- start 2023-10-26T06:47:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa7a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228fa7c4d0 con 0x56228fa70be0 2023-10-26T06:47:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228fa7aaa0 con 0x56228fa71500 2023-10-26T06:47:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228fa7ac60 con 0x56228fa6b5d0 2023-10-26T06:47:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96e1fc1b80 10 client.0 _ll_register_callbacks cb 0x56228f8a4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96c77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96c77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49374/0 (socket says 172.21.15.89:49374) 2023-10-26T06:47:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96c77fe700 1 -- 172.21.15.89:0/1266617878 learned_addr learned my addr 172.21.15.89:0/1266617878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.805+0000 7f96c7fff700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa7a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96c77fe700 1 -- 172.21.15.89:0/1266617878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 msgr2=0x56228fa7bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96c77fe700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa7bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96c77fe700 1 -- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 msgr2=0x56228fa7a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96c77fe700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa7a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96c77fe700 1 -- 172.21.15.89:0/1266617878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228f97b3f0 con 0x56228fa70be0 2023-10-26T06:47:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96ccb9c700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa7a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:47:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96c77fe700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7a020 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f96bc011a70 tx=0x7f96bc011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da47f08868e4d159 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96a77fe700 1 -- 172.21.15.89:0/1266617878 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc016040 con 0x56228fa70be0 2023-10-26T06:47:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96b80b2070 con 0x56228fa70be0 2023-10-26T06:47:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.806+0000 7f96a77fe700 1 -- 172.21.15.89:0/1266617878 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f96bc003d50 con 0x56228fa70be0 2023-10-26T06:47:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96a77fe700 1 -- 172.21.15.89:0/1266617878 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96bc004050 con 0x56228fa70be0 2023-10-26T06:47:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 10 client.15555 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f96bc007ae0 con 0x56228fa70be0 2023-10-26T06:47:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96a77fe700 1 -- 172.21.15.89:0/1266617878 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b80b2070 con 0x56228fa70be0 2023-10-26T06:47:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96a77fe700 1 -- 172.21.15.89:0/1266617878 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f96bc01ccd0 con 0x56228fa70be0 2023-10-26T06:47:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 20 client.15555 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fada090 0x56228fadc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 4 client.15555 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:47:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56228f9b6e50 con 0x56228fada090 2023-10-26T06:47:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.807+0000 7f96c7fff700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fada090 0x56228fadc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96c7fff700 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fada090 0x56228fadc4b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f655fd9b3746a9 server_cookie=eeaa97d4490567e0 in_seq=0 out_seq=0 2023-10-26T06:47:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96a77fe700 10 client.15555 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96a77fe700 1 -- 172.21.15.89:0/1266617878 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56228f9b6e50 con 0x56228fada090 2023-10-26T06:47:55.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96a77fe700 10 client.15555 handle_command_reply: tid=0 2023-10-26T06:47:55.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 client.15555 shutdown 2023-10-26T06:47:55.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 msgr2=0x56228fa7a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7a020 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f96bc011a70 tx=0x7f96bc011e30 comp rx=0 tx=0).stop 2023-10-26T06:47:55.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 shutdown_connections 2023-10-26T06:47:55.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56228fada090 0x56228fadc4b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56228fa71500 0x56228fa7a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56228fa6b5d0 0x56228fa7bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 --2- 172.21.15.89:0/1266617878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56228fa70be0 0x56228fa7a020 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 >> 172.21.15.89:0/1266617878 conn(0x56228f9df720 msgr2=0x56228fa72fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96a77fe700 0 client.15555 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 shutdown_connections 2023-10-26T06:47:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.808+0000 7f96e1fc1b80 1 -- 172.21.15.89:0/1266617878 wait complete. 2023-10-26T06:47:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96e1fc1b80 20 client.15555 trim_cache size 0 max 16384 2023-10-26T06:47:55.827 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:47:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96b00a1670 msgr2=0x7f96b00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96b00a1670 0x7f96b00a3b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009610 tx=0x7f96c00093a0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8072c20 msgr2=0x7f96c8198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8072c20 0x7f96c8198f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96b800e9a0 tx=0x7f96b800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 shutdown_connections 2023-10-26T06:47:55.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f96b00a1670 0x7f96b00a3b30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96c810cf60 0x7f96c819d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8072c20 0x7f96c8198f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 --2- 172.21.15.89:0/3629137542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96c8073540 0x7f96c8199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:55.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 >> 172.21.15.89:0/3629137542 conn(0x7f96c80fc1b0 msgr2=0x7f96c80fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:55.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 shutdown_connections 2023-10-26T06:47:55.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:55.809+0000 7f96ce5ff700 1 -- 172.21.15.89:0/3629137542 wait complete. 2023-10-26T06:47:56.201 INFO:tasks.workunit.client.0.smithi089.stderr:Updating files: 63% (7707/12067) Updating files: 64% (7723/12067) Updating files: 65% (7844/12067) Updating files: 66% (7965/12067) Updating files: 67% (8085/12067) Updating files: 68% (8206/12067) Updating files: 69% (8327/12067) Updating files: 70% (8447/12067) Updating files: 71% (8568/12067) Updating files: 72% (8689/12067) Updating files: 73% (8809/12067) Updating files: 74% (8930/12067) Updating files: 75% (9051/12067) Updating files: 76% (9171/12067) Updating files: 77% (9292/12067) Updating files: 78% (9413/12067) Updating files: 79% (9533/12067) Updating files: 80% (9654/12067) Updating files: 81% (9775/12067) Updating files: 82% (9895/12067) Updating files: 83% (10016/12067) Updating files: 84% (10137/12067) Updating files: 85% (10257/12067) Updating files: 86% (10378/12067) Updating files: 87% (10499/12067) Updating files: 88% (10619/12067) Updating files: 89% (10740/12067) Updating files: 90% (10861/12067) Updating files: 91% (10981/12067) Updating files: 92% (11102/12067) Updating files: 93% (11223/12067) Updating files: 94% (11343/12067) Updating files: 95% (11464/12067) Updating files: 96% (11585/12067) Updating files: 97% (11705/12067) Updating files: 98% (11826/12067) Updating files: 99% (11947/12067) Updating files: 100% (12067/12067) Updating files: 100% (12067/12067), done. 2023-10-26T06:47:56.408 INFO:tasks.workunit.client.0.smithi089.stderr:Note: switching to '384993681abceb8cb8affea10538d607052db4b4'. 2023-10-26T06:47:56.408 INFO:tasks.workunit.client.0.smithi089.stderr: 2023-10-26T06:47:56.408 INFO:tasks.workunit.client.0.smithi089.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-26T06:47:56.408 INFO:tasks.workunit.client.0.smithi089.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-26T06:47:56.409 INFO:tasks.workunit.client.0.smithi089.stderr:state without impacting any branches by switching back to a branch. 2023-10-26T06:47:56.409 INFO:tasks.workunit.client.0.smithi089.stderr: 2023-10-26T06:47:56.409 INFO:tasks.workunit.client.0.smithi089.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-26T06:47:56.409 INFO:tasks.workunit.client.0.smithi089.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-26T06:47:56.409 INFO:tasks.workunit.client.0.smithi089.stderr: 2023-10-26T06:47:56.410 INFO:tasks.workunit.client.0.smithi089.stderr: git switch -c 2023-10-26T06:47:56.410 INFO:tasks.workunit.client.0.smithi089.stderr: 2023-10-26T06:47:56.410 INFO:tasks.workunit.client.0.smithi089.stderr:Or undo this operation with: 2023-10-26T06:47:56.410 INFO:tasks.workunit.client.0.smithi089.stderr: 2023-10-26T06:47:56.410 INFO:tasks.workunit.client.0.smithi089.stderr: git switch - 2023-10-26T06:47:56.411 INFO:tasks.workunit.client.0.smithi089.stderr: 2023-10-26T06:47:56.411 INFO:tasks.workunit.client.0.smithi089.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-26T06:47:56.411 INFO:tasks.workunit.client.0.smithi089.stderr: 2023-10-26T06:47:56.411 INFO:tasks.workunit.client.0.smithi089.stderr:HEAD is now at 384993681ab Merge PR #47121 into wip-batrick-testing-20231024.144545 2023-10-26T06:47:56.412 DEBUG:teuthology.orchestra.run.smithi089:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-26T06:47:56.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:56 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3089827666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:47:56.510 INFO:tasks.workunit.client.0.smithi089.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-26T06:47:56.515 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-26T06:47:56.516 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-26T06:47:56.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:56 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3089827666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:47:56.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:56 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3089827666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:47:56.832 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:47:56.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:47:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 -- 172.21.15.89:0/2156583838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6941010e0 msgr2=0x7fa69410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 --2- 172.21.15.89:0/2156583838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6941010e0 0x7fa69410d5f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00b0a0 tx=0x7fa68c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 -- 172.21.15.89:0/2156583838 shutdown_connections 2023-10-26T06:47:57.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 --2- 172.21.15.89:0/2156583838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6941010e0 0x7fa69410d5f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 --2- 172.21.15.89:0/2156583838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa694106730 0x7fa694106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 --2- 172.21.15.89:0/2156583838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa694100720 0x7fa694100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 -- 172.21.15.89:0/2156583838 >> 172.21.15.89:0/2156583838 conn(0x7fa6940fc1b0 msgr2=0x7fa6940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 -- 172.21.15.89:0/2156583838 shutdown_connections 2023-10-26T06:47:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.038+0000 7fa698dff700 1 -- 172.21.15.89:0/2156583838 wait complete. 2023-10-26T06:47:57.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.039+0000 7fa698dff700 1 Processor -- start 2023-10-26T06:47:57.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.039+0000 7fa698dff700 1 -- start start 2023-10-26T06:47:57.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.039+0000 7fa698dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa694100720 0x7fa694198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa698dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6941010e0 0x7fa6941992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa698dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa694106730 0x7fa69419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa698dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419dca0 con 0x7fa694106730 2023-10-26T06:47:57.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa698dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419de10 con 0x7fa6941010e0 2023-10-26T06:47:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa698dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69419dfd0 con 0x7fa694100720 2023-10-26T06:47:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa694106730 0x7fa69419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa694106730 0x7fa69419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49388/0 (socket says 172.21.15.89:49388) 2023-10-26T06:47:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 -- 172.21.15.89:0/1043194689 learned_addr learned my addr 172.21.15.89:0/1043194689 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa69259c700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa694100720 0x7fa694198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa691d9b700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6941010e0 0x7fa6941992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 -- 172.21.15.89:0/1043194689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa694100720 msgr2=0x7fa694198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa694100720 0x7fa694198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 -- 172.21.15.89:0/1043194689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6941010e0 msgr2=0x7fa6941992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6941010e0 0x7fa6941992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 -- 172.21.15.89:0/1043194689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c009d00 con 0x7fa694106730 2023-10-26T06:47:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.040+0000 7fa692d9d700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa694106730 0x7fa69419d660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00b070 tx=0x7fa68c0077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.041+0000 7fa6837fe700 1 -- 172.21.15.89:0/1043194689 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00e040 con 0x7fa694106730 2023-10-26T06:47:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.041+0000 7fa6837fe700 1 -- 172.21.15.89:0/1043194689 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa68c0094c0 con 0x7fa694106730 2023-10-26T06:47:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.041+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69419e250 con 0x7fa694106730 2023-10-26T06:47:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.041+0000 7fa6837fe700 1 -- 172.21.15.89:0/1043194689 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0043a0 con 0x7fa694106730 2023-10-26T06:47:57.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.041+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6941a5af0 con 0x7fa694106730 2023-10-26T06:47:57.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.045+0000 7fa6837fe700 1 -- 172.21.15.89:0/1043194689 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa68c004500 con 0x7fa694106730 2023-10-26T06:47:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.046+0000 7fa6ac822b80 1 Processor -- start 2023-10-26T06:47:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.046+0000 7fa6ac822b80 1 -- start start 2023-10-26T06:47:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.046+0000 7fa6837fe700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa6837fe700 1 -- 172.21.15.89:0/1043194689 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa68c0cdf20 con 0x7fa694106730 2023-10-26T06:47:57.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da3580 0x55c1e7d9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7d9d650 0x55c1e7d9dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d10760 con 0x55c1e7d9d650 2023-10-26T06:47:57.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7cf6ce0 con 0x55c1e7da2c60 2023-10-26T06:47:57.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7cf4600 con 0x55c1e7da3580 2023-10-26T06:47:57.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa691d9b700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa69259c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa692d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7d9d650 0x55c1e7d9dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa69259c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:54552/0 (socket says 172.21.15.89:54552) 2023-10-26T06:47:57.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa69259c700 1 -- 172.21.15.89:0/985602675 learned_addr learned my addr 172.21.15.89:0/985602675 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.047+0000 7fa691d9b700 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da3580 0x55c1e7d9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa692d9d700 1 -- 172.21.15.89:0/985602675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da3580 msgr2=0x55c1e7d9d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa692d9d700 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da3580 0x55c1e7d9d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa692d9d700 1 -- 172.21.15.89:0/985602675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 msgr2=0x55c1e7da3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa692d9d700 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa692d9d700 1 -- 172.21.15.89:0/985602675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e7cad470 con 0x55c1e7d9d650 2023-10-26T06:47:57.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa69259c700 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:57.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa691d9b700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa6880097a0 tx=0x7fa6880138b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa692d9d700 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7d9d650 0x55c1e7d9dad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0cb6b0 tx=0x7fa68c09aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8737b77a99c3f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa680ff9700 1 -- 172.21.15.89:0/985602675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00be20 con 0x55c1e7d9d650 2023-10-26T06:47:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa680ff9700 1 -- 172.21.15.89:0/985602675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa68c09b6d0 con 0x55c1e7d9d650 2023-10-26T06:47:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.048+0000 7fa680ff9700 1 -- 172.21.15.89:0/985602675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c099400 con 0x55c1e7d9d650 2023-10-26T06:47:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 -- 172.21.15.89:0/985602675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7d9d650 msgr2=0x55c1e7d9dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7d9d650 0x55c1e7d9dad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0cb6b0 tx=0x7fa68c09aea0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 -- 172.21.15.89:0/985602675 shutdown_connections 2023-10-26T06:47:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7d9d650 0x55c1e7d9dad0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/985602675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da3580 0x55c1e7d9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 -- 172.21.15.89:0/985602675 >> 172.21.15.89:0/985602675 conn(0x55c1e7d117a0 msgr2=0x55c1e7d97be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 -- 172.21.15.89:0/985602675 shutdown_connections 2023-10-26T06:47:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.049+0000 7fa6ac822b80 1 -- 172.21.15.89:0/985602675 wait complete. 2023-10-26T06:47:57.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 -- start start 2023-10-26T06:47:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 Processor -- start 2023-10-26T06:47:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7da9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da2c60 0x55c1e7da6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da3580 0x55c1e7da6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7cefef0 con 0x55c1e7da2c60 2023-10-26T06:47:57.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7c8c790 con 0x55c1e7da3580 2023-10-26T06:47:57.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7c8c430 con 0x55c1e7d9d650 2023-10-26T06:47:57.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.050+0000 7fa6ac822b80 10 client.0 _ll_register_callbacks cb 0x55c1e7bd6b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:57.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da2c60 0x55c1e7da6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da2c60 0x55c1e7da6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49402/0 (socket says 172.21.15.89:49402) 2023-10-26T06:47:57.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 -- 172.21.15.89:0/571294409 learned_addr learned my addr 172.21.15.89:0/571294409 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa692d9d700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da3580 0x55c1e7da6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa69259c700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7da9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 -- 172.21.15.89:0/571294409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 msgr2=0x55c1e7da9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7da9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 -- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da3580 msgr2=0x55c1e7da6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da3580 0x55c1e7da6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 -- 172.21.15.89:0/571294409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa684009960 con 0x55c1e7da2c60 2023-10-26T06:47:57.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa692d9d700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da3580 0x55c1e7da6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa691d9b700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da2c60 0x55c1e7da6880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa688004530 tx=0x7fa6880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9f2bebaf7d36ad server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.051+0000 7fa679ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa679ffb700 1 -- 172.21.15.89:0/571294409 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688018070 con 0x55c1e7da2c60 2023-10-26T06:47:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa679ffb700 1 -- 172.21.15.89:0/571294409 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa688012270 con 0x55c1e7da2c60 2023-10-26T06:47:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1e7cf4f70 con 0x55c1e7da2c60 2023-10-26T06:47:57.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa679ffb700 1 -- 172.21.15.89:0/571294409 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa688004560 con 0x55c1e7da2c60 2023-10-26T06:47:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 10 client.15567 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa6941a5af0 con 0x55c1e7da2c60 2023-10-26T06:47:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa679ffb700 1 -- 172.21.15.89:0/571294409 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1e7cf4f70 con 0x55c1e7da2c60 2023-10-26T06:47:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa679ffb700 1 -- 172.21.15.89:0/571294409 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa688021530 con 0x55c1e7da2c60 2023-10-26T06:47:57.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:57.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:57.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 20 client.15567 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:57.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7db6da0 0x55c1e7db91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 4 client.15567 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:57.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.052+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1e7d6c930 con 0x55c1e7db6da0 2023-10-26T06:47:57.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.053+0000 7fa69259c700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7db6da0 0x55c1e7db91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.053+0000 7fa679ffb700 10 client.15567 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.053+0000 7fa69259c700 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7db6da0 0x55c1e7db91c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c61ef56b1de4b6 server_cookie=2dcd9bdc74319585 in_seq=0 out_seq=0 2023-10-26T06:47:57.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.054+0000 7fa679ffb700 1 -- 172.21.15.89:0/571294409 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c1e7d6c930 con 0x55c1e7db6da0 2023-10-26T06:47:57.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.054+0000 7fa679ffb700 10 client.15567 handle_command_reply: tid=0 2023-10-26T06:47:57.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.054+0000 7fa6ac822b80 1 client.15567 shutdown 2023-10-26T06:47:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da2c60 msgr2=0x55c1e7da6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da2c60 0x55c1e7da6880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa688004530 tx=0x7fa6880047c0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 shutdown_connections 2023-10-26T06:47:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7db6da0 0x55c1e7db91c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da3580 0x55c1e7da6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7da9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/571294409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da2c60 0x55c1e7da6880 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa679ffb700 0 client.15567 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 >> 172.21.15.89:0/571294409 conn(0x55c1e7d117a0 msgr2=0x55c1e7d9c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 shutdown_connections 2023-10-26T06:47:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 1 -- 172.21.15.89:0/571294409 wait complete. 2023-10-26T06:47:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.055+0000 7fa6ac822b80 20 client.15567 trim_cache size 0 max 16384 2023-10-26T06:47:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.072+0000 7fa6ac822b80 1 Processor -- start 2023-10-26T06:47:57.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.072+0000 7fa6ac822b80 1 -- start start 2023-10-26T06:47:57.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7dadd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7da5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7b709a0 con 0x55c1e7da3580 2023-10-26T06:47:57.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7da77d0 con 0x55c1e7da2c60 2023-10-26T06:47:57.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d9e080 con 0x55c1e7d9d650 2023-10-26T06:47:57.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7da5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7da5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49410/0 (socket says 172.21.15.89:49410) 2023-10-26T06:47:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 -- 172.21.15.89:0/1793888616 learned_addr learned my addr 172.21.15.89:0/1793888616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa69259c700 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7dadd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa691d9b700 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 -- 172.21.15.89:0/1793888616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 msgr2=0x55c1e7dadd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7dadd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 -- 172.21.15.89:0/1793888616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 msgr2=0x55c1e7da5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 -- 172.21.15.89:0/1793888616 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e7d97df0 con 0x55c1e7da3580 2023-10-26T06:47:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.073+0000 7fa692d9d700 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7da5690 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0979f0 tx=0x7fa68c09b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ee55f6c4e4b00b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6817fa700 1 -- 172.21.15.89:0/1793888616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00be20 con 0x55c1e7da3580 2023-10-26T06:47:57.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6817fa700 1 -- 172.21.15.89:0/1793888616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa68c0a32d0 con 0x55c1e7da3580 2023-10-26T06:47:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6817fa700 1 -- 172.21.15.89:0/1793888616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c024360 con 0x55c1e7da3580 2023-10-26T06:47:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 -- 172.21.15.89:0/1793888616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 msgr2=0x55c1e7da5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7da5690 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0979f0 tx=0x7fa68c09b840 comp rx=0 tx=0).stop 2023-10-26T06:47:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 -- 172.21.15.89:0/1793888616 shutdown_connections 2023-10-26T06:47:57.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7da5690 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7d9d650 0x55c1e7dadd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/1793888616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7da2c60 0x55c1e7da5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 -- 172.21.15.89:0/1793888616 >> 172.21.15.89:0/1793888616 conn(0x55c1e7d117a0 msgr2=0x55c1e7da91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.074+0000 7fa6ac822b80 1 -- 172.21.15.89:0/1793888616 shutdown_connections 2023-10-26T06:47:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 -- 172.21.15.89:0/1793888616 wait complete. 2023-10-26T06:47:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 -- start start 2023-10-26T06:47:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 Processor -- start 2023-10-26T06:47:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7d9d650 0x55c1e7dacfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da2c60 0x55c1e7dad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7e03c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7e041c0 con 0x55c1e7da3580 2023-10-26T06:47:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7e04330 con 0x55c1e7d9d650 2023-10-26T06:47:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.075+0000 7fa6ac822b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7e044f0 con 0x55c1e7da2c60 2023-10-26T06:47:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa6ac822b80 10 client.0 _ll_register_callbacks cb 0x55c1e7bd6b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7e03c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7e03c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49414/0 (socket says 172.21.15.89:49414) 2023-10-26T06:47:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 -- 172.21.15.89:0/302903236 learned_addr learned my addr 172.21.15.89:0/302903236 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa69259c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7d9d650 0x55c1e7dacfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa691d9b700 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da2c60 0x55c1e7dad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 -- 172.21.15.89:0/302903236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da2c60 msgr2=0x55c1e7dad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da2c60 0x55c1e7dad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 -- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7d9d650 msgr2=0x55c1e7dacfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7d9d650 0x55c1e7dacfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 -- 172.21.15.89:0/302903236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1e7cad470 con 0x55c1e7da3580 2023-10-26T06:47:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa69259c700 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7d9d650 0x55c1e7dacfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa692d9d700 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7e03c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0974d0 tx=0x7fa68c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4be4b6fbf9f3707 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.076+0000 7fa679ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa679ffb700 1 -- 172.21.15.89:0/302903236 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c09f120 con 0x55c1e7da3580 2023-10-26T06:47:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa679ffb700 1 -- 172.21.15.89:0/302903236 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa68c09afd0 con 0x55c1e7da3580 2023-10-26T06:47:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa68c09b6d0 con 0x55c1e7da3580 2023-10-26T06:47:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa679ffb700 1 -- 172.21.15.89:0/302903236 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa68c0b9220 con 0x55c1e7da3580 2023-10-26T06:47:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 10 client.15573 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c1e7b1ae60 con 0x55c1e7da3580 2023-10-26T06:47:57.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa679ffb700 1 -- 172.21.15.89:0/302903236 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa68c09b6d0 con 0x55c1e7da3580 2023-10-26T06:47:57.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa679ffb700 1 -- 172.21.15.89:0/302903236 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa68c0aed20 con 0x55c1e7da3580 2023-10-26T06:47:57.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:57.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:57.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 20 client.15573 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:57.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7e0caa0 0x55c1e7e0ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 4 client.15573 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:47:57.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.077+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c1e7ce8ed0 con 0x55c1e7e0caa0 2023-10-26T06:47:57.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.078+0000 7fa69259c700 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7e0caa0 0x55c1e7e0ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.078+0000 7fa69259c700 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7e0caa0 0x55c1e7e0ee90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb3079828a60a55 server_cookie=95f45f71e57d5250 in_seq=0 out_seq=0 2023-10-26T06:47:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.078+0000 7fa679ffb700 10 client.15573 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.079+0000 7fa679ffb700 1 -- 172.21.15.89:0/302903236 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c1e7ce8ed0 con 0x55c1e7e0caa0 2023-10-26T06:47:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.079+0000 7fa679ffb700 10 client.15573 handle_command_reply: tid=0 2023-10-26T06:47:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.079+0000 7fa6ac822b80 1 client.15573 shutdown 2023-10-26T06:47:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 msgr2=0x55c1e7e03c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7e03c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0974d0 tx=0x7fa68c00bf60 comp rx=0 tx=0).stop 2023-10-26T06:47:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 shutdown_connections 2023-10-26T06:47:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1e7da3580 0x55c1e7e03c10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c1e7e0caa0 0x55c1e7e0ee90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1e7d9d650 0x55c1e7dacfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 --2- 172.21.15.89:0/302903236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1e7da2c60 0x55c1e7dad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 >> 172.21.15.89:0/302903236 conn(0x55c1e7d117a0 msgr2=0x55c1e7da0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa679ffb700 0 client.15573 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 shutdown_connections 2023-10-26T06:47:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 1 -- 172.21.15.89:0/302903236 wait complete. 2023-10-26T06:47:57.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.080+0000 7fa6ac822b80 20 client.15573 trim_cache size 0 max 16384 2023-10-26T06:47:57.094 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:47:57.094 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:47:57.094 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "d122659a-a8bf-4397-85f5-7c9f71d6e4f5", 2023-10-26T06:47:57.094 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:47:57.094 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:47:57.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa67c0a15f0 msgr2=0x7fa67c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa6880097a0 tx=0x7fa6880138b0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa694106730 msgr2=0x7fa69419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa694106730 0x7fa69419d660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00b070 tx=0x7fa68c0077f0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 shutdown_connections 2023-10-26T06:47:57.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa67c0a15f0 0x7fa67c0a3ab0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa694106730 0x7fa69419d660 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa694100720 0x7fa694198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 --2- 172.21.15.89:0/1043194689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6941010e0 0x7fa6941992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 >> 172.21.15.89:0/1043194689 conn(0x7fa6940fc1b0 msgr2=0x7fa69410c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 shutdown_connections 2023-10-26T06:47:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.081+0000 7fa698dff700 1 -- 172.21.15.89:0/1043194689 wait complete. 2023-10-26T06:47:57.097 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-26T06:47:57.101 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:47:57.174 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-26T06:47:57.241 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-26T06:47:57.245 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-26T06:47:57.245 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-26T06:47:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/3937410058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 msgr2=0x7fbf34072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/3937410058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf34072f70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbf300098f0 tx=0x7fbf30009c00 comp rx=0 tx=0).stop 2023-10-26T06:47:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/3937410058 shutdown_connections 2023-10-26T06:47:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/3937410058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf34072f70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/3937410058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf34111730 0x7fbf34113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/3937410058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf340734b0 0x7fbf341111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/3937410058 >> 172.21.15.89:0/3937410058 conn(0x7fbf340fc1b0 msgr2=0x7fbf340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/3937410058 shutdown_connections 2023-10-26T06:47:57.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.307+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/3937410058 wait complete. 2023-10-26T06:47:57.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.308+0000 7fbf3b5ff700 1 Processor -- start 2023-10-26T06:47:57.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.308+0000 7fbf3b5ff700 1 -- start start 2023-10-26T06:47:57.311 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-26T06:47:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3b5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf3419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3b5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf340734b0 0x7fbf3419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3b5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf34111730 0x7fbf341a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3b5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf341a21d0 con 0x7fbf34072b90 2023-10-26T06:47:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3b5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf341a2340 con 0x7fbf34111730 2023-10-26T06:47:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3b5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf341a2500 con 0x7fbf340734b0 2023-10-26T06:47:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf3419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf3419d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49428/0 (socket says 172.21.15.89:49428) 2023-10-26T06:47:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf3939b700 1 -- 172.21.15.89:0/953937781 learned_addr learned my addr 172.21.15.89:0/953937781 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf34111730 0x7fbf341a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.309+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf340734b0 0x7fbf3419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3939b700 1 -- 172.21.15.89:0/953937781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf340734b0 msgr2=0x7fbf3419d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3939b700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf340734b0 0x7fbf3419d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3939b700 1 -- 172.21.15.89:0/953937781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf34111730 msgr2=0x7fbf341a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3939b700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf34111730 0x7fbf341a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3939b700 1 -- 172.21.15.89:0/953937781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30009520 con 0x7fbf34072b90 2023-10-26T06:47:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3939b700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf3419d300 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005950 tx=0x7fbf3000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf2a7fc700 1 -- 172.21.15.89:0/953937781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3001c070 con 0x7fbf34072b90 2023-10-26T06:47:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf2a7fc700 1 -- 172.21.15.89:0/953937781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf3000bb20 con 0x7fbf34072b90 2023-10-26T06:47:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf341a2780 con 0x7fbf34072b90 2023-10-26T06:47:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf2a7fc700 1 -- 172.21.15.89:0/953937781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300038e0 con 0x7fbf34072b90 2023-10-26T06:47:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.310+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf341aa020 con 0x7fbf34072b90 2023-10-26T06:47:57.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.314+0000 7fbf2a7fc700 1 -- 172.21.15.89:0/953937781 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbf30020030 con 0x7fbf34072b90 2023-10-26T06:47:57.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 Processor -- start 2023-10-26T06:47:57.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 -- start start 2023-10-26T06:47:57.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c35d0 0x5651f91c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f91366e0 con 0x5651f91c8be0 2023-10-26T06:47:57.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f911cc60 con 0x5651f91c9500 2023-10-26T06:47:57.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f911a580 con 0x5651f91c35d0 2023-10-26T06:47:57.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf39b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.315+0000 7fbf3939b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:54602/0 (socket says 172.21.15.89:54602) 2023-10-26T06:47:57.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 -- 172.21.15.89:0/1177891603 learned_addr learned my addr 172.21.15.89:0/1177891603 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 -- 172.21.15.89:0/1177891603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c35d0 msgr2=0x5651f91c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf2a7fc700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf200a15f0 0x7fbf200a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1177891603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c35d0 0x5651f91c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 -- 172.21.15.89:0/1177891603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 msgr2=0x5651f91c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1177891603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 -- 172.21.15.89:0/1177891603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651f90d33f0 con 0x5651f91c9500 2023-10-26T06:47:57.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf200a15f0 0x7fbf200a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf2a7fc700 1 -- 172.21.15.89:0/953937781 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbf300cdef0 con 0x7fbf34072b90 2023-10-26T06:47:57.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.316+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1177891603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91c3090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf300cb560 tx=0x7fbf30097300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b060d3708aefcf server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf1f7fe700 1 -- 172.21.15.89:0/1177891603 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3009ad10 con 0x5651f91c9500 2023-10-26T06:47:57.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf1f7fe700 1 -- 172.21.15.89:0/1177891603 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf3009aa80 con 0x5651f91c9500 2023-10-26T06:47:57.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf1f7fe700 1 -- 172.21.15.89:0/1177891603 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3009c070 con 0x5651f91c9500 2023-10-26T06:47:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1177891603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 msgr2=0x5651f91c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1177891603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91c3090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf300cb560 tx=0x7fbf30097300 comp rx=0 tx=0).stop 2023-10-26T06:47:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf200a15f0 0x7fbf200a3ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009ec0 tx=0x7fbf2c009040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1177891603 shutdown_connections 2023-10-26T06:47:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1177891603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1177891603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c35d0 0x5651f91c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1177891603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91c3090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.317+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1177891603 >> 172.21.15.89:0/1177891603 conn(0x5651f9137720 msgr2=0x5651f91bdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.318+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1177891603 shutdown_connections 2023-10-26T06:47:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.318+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1177891603 wait complete. 2023-10-26T06:47:57.326 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:47:57.327 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-26T06:47:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.318+0000 7fbf4ef14b80 1 -- start start 2023-10-26T06:47:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.318+0000 7fbf4ef14b80 1 Processor -- start 2023-10-26T06:47:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f9115e70 con 0x5651f91c8be0 2023-10-26T06:47:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f90b2710 con 0x5651f91c35d0 2023-10-26T06:47:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f90b23b0 con 0x5651f91c9500 2023-10-26T06:47:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf4ef14b80 10 client.0 _ll_register_callbacks cb 0x5651f8ffca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49438/0 (socket says 172.21.15.89:49438) 2023-10-26T06:47:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421649713 learned_addr learned my addr 172.21.15.89:0/1421649713 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421649713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 msgr2=0x5651f91ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 msgr2=0x5651f91cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421649713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf341aa020 con 0x5651f91c8be0 2023-10-26T06:47:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.319+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91cc6e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf240116c0 tx=0x7fbf24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7d1182301decaf server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:47:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf1cff9700 1 -- 172.21.15.89:0/1421649713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24004220 con 0x5651f91c8be0 2023-10-26T06:47:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651f911aef0 con 0x5651f91c8be0 2023-10-26T06:47:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf1cff9700 1 -- 172.21.15.89:0/1421649713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbf24004380 con 0x5651f91c8be0 2023-10-26T06:47:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf1cff9700 1 -- 172.21.15.89:0/1421649713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf24004540 con 0x5651f91c8be0 2023-10-26T06:47:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf4ef14b80 10 client.15585 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5651f8f40de0 con 0x5651f91c8be0 2023-10-26T06:47:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.320+0000 7fbf1cff9700 1 -- 172.21.15.89:0/1421649713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5651f911aef0 con 0x5651f91c8be0 2023-10-26T06:47:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf1cff9700 1 -- 172.21.15.89:0/1421649713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf2400e9d0 con 0x5651f91c8be0 2023-10-26T06:47:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf4ef14b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf4ef14b80 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf4ef14b80 20 client.15585 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f91dcb60 0x5651f91def80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf4ef14b80 4 client.15585 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:47:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651f91928b0 con 0x5651f91dcb60 2023-10-26T06:47:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.321+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f91dcb60 0x5651f91def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.322+0000 7fbf1cff9700 10 client.15585 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.322+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f91dcb60 0x5651f91def80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7cfadb3c7edc9b5 server_cookie=a94d2113683b86b in_seq=0 out_seq=0 2023-10-26T06:47:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.322+0000 7fbf1cff9700 1 -- 172.21.15.89:0/1421649713 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5651f91928b0 con 0x5651f91dcb60 2023-10-26T06:47:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.322+0000 7fbf1cff9700 10 client.15585 handle_command_reply: tid=0 2023-10-26T06:47:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.322+0000 7fbf4ef14b80 1 client.15585 shutdown 2023-10-26T06:47:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 msgr2=0x5651f91cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91cc6e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf240116c0 tx=0x7fbf24009f90 comp rx=0 tx=0).stop 2023-10-26T06:47:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 shutdown_connections 2023-10-26T06:47:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f91dcb60 0x5651f91def80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf1cff9700 0 client.15585 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421649713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91cc6e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 >> 172.21.15.89:0/1421649713 conn(0x5651f9137720 msgr2=0x5651f91c2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 shutdown_connections 2023-10-26T06:47:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421649713 wait complete. 2023-10-26T06:47:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.323+0000 7fbf4ef14b80 20 client.15585 trim_cache size 0 max 16384 2023-10-26T06:47:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.340+0000 7fbf4ef14b80 1 Processor -- start 2023-10-26T06:47:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.340+0000 7fbf4ef14b80 1 -- start start 2023-10-26T06:47:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f8f96920 con 0x5651f91c8be0 2023-10-26T06:47:57.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f91928b0 con 0x5651f91c35d0 2023-10-26T06:47:57.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f91da170 con 0x5651f91c9500 2023-10-26T06:47:57.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49448/0 (socket says 172.21.15.89:49448) 2023-10-26T06:47:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421236691 learned_addr learned my addr 172.21.15.89:0/1421236691 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421236691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 msgr2=0x5651f91cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421236691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 msgr2=0x5651f91b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf38b9a700 1 -- 172.21.15.89:0/1421236691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651f8f29c60 con 0x5651f91c8be0 2023-10-26T06:47:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.341+0000 7fbf3939b700 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:57.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.342+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91d3c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ed30 tx=0x7fbf24009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce7a6f5a567ed7a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.342+0000 7fbf1ffff700 1 -- 172.21.15.89:0/1421236691 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400e900 con 0x5651f91c8be0 2023-10-26T06:47:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.342+0000 7fbf1ffff700 1 -- 172.21.15.89:0/1421236691 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2400ea60 con 0x5651f91c8be0 2023-10-26T06:47:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.342+0000 7fbf1ffff700 1 -- 172.21.15.89:0/1421236691 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24003cf0 con 0x5651f91c8be0 2023-10-26T06:47:57.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.342+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421236691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 msgr2=0x5651f91d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.342+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91d3c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ed30 tx=0x7fbf24009790 comp rx=0 tx=0).stop 2023-10-26T06:47:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421236691 shutdown_connections 2023-10-26T06:47:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c9500 0x5651f91cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c35d0 0x5651f91b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/1421236691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c8be0 0x5651f91d3c30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421236691 >> 172.21.15.89:0/1421236691 conn(0x5651f9137720 msgr2=0x5651f91cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421236691 shutdown_connections 2023-10-26T06:47:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/1421236691 wait complete. 2023-10-26T06:47:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 -- start start 2023-10-26T06:47:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.343+0000 7fbf4ef14b80 1 Processor -- start 2023-10-26T06:47:57.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c35d0 0x5651f91d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c8be0 0x5651f91d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf4ef14b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f91d2320 con 0x5651f91c35d0 2023-10-26T06:47:57.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f91d2490 con 0x5651f91c9500 2023-10-26T06:47:57.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf4ef14b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f91d2650 con 0x5651f91c8be0 2023-10-26T06:47:57.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf4ef14b80 10 client.0 _ll_register_callbacks cb 0x5651f8ffca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:47:57.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf3939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c35d0 0x5651f91d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf3939b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c35d0 0x5651f91d36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49464/0 (socket says 172.21.15.89:49464) 2023-10-26T06:47:57.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.344+0000 7fbf3939b700 1 -- 172.21.15.89:0/2807292078 learned_addr learned my addr 172.21.15.89:0/2807292078 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:47:57.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf38b9a700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c8be0 0x5651f91d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf39b9c700 1 -- 172.21.15.89:0/2807292078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c8be0 msgr2=0x5651f91d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c8be0 0x5651f91d3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf39b9c700 1 -- 172.21.15.89:0/2807292078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c35d0 msgr2=0x5651f91d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:47:57.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c35d0 0x5651f91d36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf39b9c700 1 -- 172.21.15.89:0/2807292078 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651f90d33f0 con 0x5651f91c9500 2023-10-26T06:47:57.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf3939b700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c35d0 0x5651f91d36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:47:57.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf39b9c700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91d1c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009c10 tx=0x7fbf2c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a1c2891f4e8d9d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:47:57.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:47:57.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf1cff9700 1 -- 172.21.15.89:0/2807292078 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c01c040 con 0x5651f91c9500 2023-10-26T06:47:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.345+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf3009aa80 con 0x5651f91c9500 2023-10-26T06:47:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf1cff9700 1 -- 172.21.15.89:0/2807292078 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbf2c018e90 con 0x5651f91c9500 2023-10-26T06:47:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf1cff9700 1 -- 172.21.15.89:0/2807292078 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf2c00e040 con 0x5651f91c9500 2023-10-26T06:47:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 10 client.25598 fetch_fsmap learned FSMap version 20 2023-10-26T06:47:57.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5651f8f40de0 con 0x5651f91c9500 2023-10-26T06:47:57.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf1cff9700 1 -- 172.21.15.89:0/2807292078 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf3009aa80 con 0x5651f91c9500 2023-10-26T06:47:57.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf1cff9700 1 -- 172.21.15.89:0/2807292078 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf2c024400 con 0x5651f91c9500 2023-10-26T06:47:57.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:47:57.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:47:57.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 20 client.25598 populate_metadata read hostname 'smithi089' 2023-10-26T06:47:57.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f9231c10 0x5651f9234030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:47:57.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 4 client.25598 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:47:57.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.346+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5651f910ee50 con 0x5651f9231c10 2023-10-26T06:47:57.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf3939b700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f9231c10 0x5651f9234030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:47:57.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf1cff9700 10 client.25598 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf3939b700 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f9231c10 0x5651f9234030 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c0278347c68fa server_cookie=b7cb1f6b91cde14a in_seq=0 out_seq=0 2023-10-26T06:47:57.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf1cff9700 1 -- 172.21.15.89:0/2807292078 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5651f910ee50 con 0x5651f9231c10 2023-10-26T06:47:57.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf1cff9700 10 client.25598 handle_command_reply: tid=0 2023-10-26T06:47:57.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 client.25598 shutdown 2023-10-26T06:47:57.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 msgr2=0x5651f91d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91d1c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009c10 tx=0x7fbf2c004ad0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 shutdown_connections 2023-10-26T06:47:57.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f91c35d0 0x5651f91d36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5651f9231c10 0x5651f9234030 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651f91c9500 0x5651f91d1c10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 --2- 172.21.15.89:0/2807292078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5651f91c8be0 0x5651f91d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf1cff9700 0 client.25598 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:47:57.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.347+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 >> 172.21.15.89:0/2807292078 conn(0x5651f9137720 msgr2=0x5651f91c6640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 shutdown_connections 2023-10-26T06:47:57.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf4ef14b80 1 -- 172.21.15.89:0/2807292078 wait complete. 2023-10-26T06:47:57.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf4ef14b80 20 client.25598 trim_cache size 0 max 16384 2023-10-26T06:47:57.373 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:47:57.373 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:47:57.373 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:47:57.374 INFO:teuthology.orchestra.run.smithi089.stdout: "d122659a-a8bf-4397-85f5-7c9f71d6e4f5": { 2023-10-26T06:47:57.374 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:47:57.374 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "d122659a-a8bf-4397-85f5-7c9f71d6e4f5", 2023-10-26T06:47:57.374 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:47:57.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:47:57.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:47:57.375 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:47:57.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf200a15f0 msgr2=0x7fbf200a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf200a15f0 0x7fbf200a3ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009ec0 tx=0x7fbf2c009040 comp rx=0 tx=0).stop 2023-10-26T06:47:57.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 msgr2=0x7fbf3419d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:47:57.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf3419d300 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005950 tx=0x7fbf3000fa00 comp rx=0 tx=0).stop 2023-10-26T06:47:57.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 shutdown_connections 2023-10-26T06:47:57.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf200a15f0 0x7fbf200a3ab0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34072b90 0x7fbf3419d300 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf34111730 0x7fbf341a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 --2- 172.21.15.89:0/953937781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf340734b0 0x7fbf3419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:47:57.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.348+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 >> 172.21.15.89:0/953937781 conn(0x7fbf340fc1b0 msgr2=0x7fbf340fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:47:57.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.349+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 shutdown_connections 2023-10-26T06:47:57.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:47:57.349+0000 7fbf3b5ff700 1 -- 172.21.15.89:0/953937781 wait complete. 2023-10-26T06:47:57.379 INFO:tasks.cephfs.filesystem:scrub status for tag:d122659a-a8bf-4397-85f5-7c9f71d6e4f5 - {'path': '/', 'tag': 'd122659a-a8bf-4397-85f5-7c9f71d6e4f5', 'options': 'recursive,force'} 2023-10-26T06:47:57.386 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2023-10-26T06:47:57.387 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2023-10-26T06:47:57.388 DEBUG:teuthology.orchestra.run.smithi089:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=384993681abceb8cb8affea10538d607052db4b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2023-10-26T06:47:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:57 smithi135 ceph-mon[131403]: pgmap v341: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:57.484 INFO:tasks.workunit.client.0.smithi089.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2023-10-26T06:47:57.486 INFO:tasks.workunit.client.0.smithi089.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2023-10-26T06:47:57.486 INFO:tasks.workunit.client.0.smithi089.stderr:+ trap -- retry EXIT 2023-10-26T06:47:57.487 INFO:tasks.workunit.client.0.smithi089.stderr:+ rm -rf ffsb 2023-10-26T06:47:57.488 INFO:tasks.workunit.client.0.smithi089.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2023-10-26T06:47:57.495 INFO:tasks.workunit.client.0.smithi089.stderr:Cloning into 'ffsb'... 2023-10-26T06:47:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:57 smithi089 ceph-mon[95031]: pgmap v341: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:57 smithi192 ceph-mon[131661]: pgmap v341: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:47:58.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:58 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:47:58.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:58 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:58.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:58 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:47:58.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:58 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:58.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:58 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:47:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:58 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:58 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:47:58.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:58 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:58.555 INFO:tasks.workunit.client.0.smithi089.stderr:+ trap - EXIT 2023-10-26T06:47:58.555 INFO:tasks.workunit.client.0.smithi089.stderr:+ cd ffsb 2023-10-26T06:47:58.556 INFO:tasks.workunit.client.0.smithi089.stderr:+ ./configure 2023-10-26T06:47:58.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:58 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:47:58.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:58 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:58.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:58 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:47:58.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:58 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:47:58.838 INFO:tasks.workunit.client.0.smithi089.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-26T06:47:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:59 smithi135 ceph-mon[131403]: pgmap v342: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:47:59.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:47:59 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:47:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:59 smithi089 ceph-mon[95031]: pgmap v342: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:47:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:47:59 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:47:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:59 smithi192 ceph-mon[131661]: pgmap v342: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:47:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:47:59 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:47:59.850 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether build environment is sane... yes 2023-10-26T06:47:59.870 INFO:tasks.workunit.client.0.smithi089.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2023-10-26T06:47:59.871 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gawk... gawk 2023-10-26T06:47:59.908 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether make sets $(MAKE)... yes 2023-10-26T06:47:59.986 INFO:tasks.workunit.client.0.smithi089.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-10-26T06:47:59.987 INFO:tasks.workunit.client.0.smithi089.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-10-26T06:48:00.004 INFO:tasks.workunit.client.0.smithi089.stdout:checking for style of include used by make... GNU 2023-10-26T06:48:00.009 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc... gcc 2023-10-26T06:48:00.111 INFO:tasks.workunit.client.0.smithi089.stdout:checking for C compiler default output file name... a.out 2023-10-26T06:48:00.117 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether the C compiler works... yes 2023-10-26T06:48:00.120 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are cross compiling... no 2023-10-26T06:48:00.186 INFO:tasks.workunit.client.0.smithi089.stdout:checking for suffix of executables... 2023-10-26T06:48:00.247 INFO:tasks.workunit.client.0.smithi089.stdout:checking for suffix of object files... o 2023-10-26T06:48:00.303 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are using the GNU C compiler... yes 2023-10-26T06:48:00.359 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether gcc accepts -g... yes 2023-10-26T06:48:00.432 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc option to accept ISO C89... none needed 2023-10-26T06:48:00.567 INFO:tasks.workunit.client.0.smithi089.stdout:checking dependency style of gcc... gcc3 2023-10-26T06:48:00.661 INFO:tasks.workunit.client.0.smithi089.stdout:checking how to run the C preprocessor... gcc -E 2023-10-26T06:48:00.732 INFO:tasks.workunit.client.0.smithi089.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-10-26T06:48:00.735 INFO:tasks.workunit.client.0.smithi089.stdout:checking for egrep... /usr/bin/grep -E 2023-10-26T06:48:00.752 INFO:tasks.workunit.client.0.smithi089.stdout:checking for AIX... no 2023-10-26T06:48:00.772 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc... (cached) gcc 2023-10-26T06:48:00.793 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-10-26T06:48:00.795 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether gcc accepts -g... (cached) yes 2023-10-26T06:48:00.795 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-10-26T06:48:00.796 INFO:tasks.workunit.client.0.smithi089.stdout:checking dependency style of gcc... (cached) gcc3 2023-10-26T06:48:00.898 INFO:tasks.workunit.client.0.smithi089.stdout:checking for main in -lm... yes 2023-10-26T06:48:00.983 INFO:tasks.workunit.client.0.smithi089.stdout:checking for main in -lpthread... yes 2023-10-26T06:48:01.229 INFO:tasks.workunit.client.0.smithi089.stdout:checking for ANSI C header files... yes 2023-10-26T06:48:01.304 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2023-10-26T06:48:01.380 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/types.h... yes 2023-10-26T06:48:01.463 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/stat.h... yes 2023-10-26T06:48:01.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:01 smithi135 ceph-mon[131403]: pgmap v343: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:48:01.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:01 smithi089 ceph-mon[95031]: pgmap v343: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:48:01.545 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdlib.h... yes 2023-10-26T06:48:01.628 INFO:tasks.workunit.client.0.smithi089.stdout:checking for string.h... yes 2023-10-26T06:48:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:01 smithi192 ceph-mon[131661]: pgmap v343: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:48:01.713 INFO:tasks.workunit.client.0.smithi089.stdout:checking for memory.h... yes 2023-10-26T06:48:01.797 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strings.h... yes 2023-10-26T06:48:01.882 INFO:tasks.workunit.client.0.smithi089.stdout:checking for inttypes.h... yes 2023-10-26T06:48:01.968 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdint.h... yes 2023-10-26T06:48:02.055 INFO:tasks.workunit.client.0.smithi089.stdout:checking for unistd.h... yes 2023-10-26T06:48:02.151 INFO:tasks.workunit.client.0.smithi089.stdout:checking pthread.h usability... yes 2023-10-26T06:48:02.186 INFO:tasks.workunit.client.0.smithi089.stdout:checking pthread.h presence... yes 2023-10-26T06:48:02.187 INFO:tasks.workunit.client.0.smithi089.stdout:checking for pthread.h... yes 2023-10-26T06:48:02.275 INFO:tasks.workunit.client.0.smithi089.stdout:checking fcntl.h usability... yes 2023-10-26T06:48:02.307 INFO:tasks.workunit.client.0.smithi089.stdout:checking fcntl.h presence... yes 2023-10-26T06:48:02.308 INFO:tasks.workunit.client.0.smithi089.stdout:checking for fcntl.h... yes 2023-10-26T06:48:02.397 INFO:tasks.workunit.client.0.smithi089.stdout:checking limits.h usability... yes 2023-10-26T06:48:02.429 INFO:tasks.workunit.client.0.smithi089.stdout:checking limits.h presence... yes 2023-10-26T06:48:02.430 INFO:tasks.workunit.client.0.smithi089.stdout:checking for limits.h... yes 2023-10-26T06:48:02.456 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdint.h... (cached) yes 2023-10-26T06:48:02.538 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/time.h usability... yes 2023-10-26T06:48:02.572 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/time.h presence... yes 2023-10-26T06:48:02.573 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/time.h... yes 2023-10-26T06:48:02.599 INFO:tasks.workunit.client.0.smithi089.stdout:checking for unistd.h... (cached) yes 2023-10-26T06:48:02.679 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/vfs.h usability... yes 2023-10-26T06:48:02.710 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/vfs.h presence... yes 2023-10-26T06:48:02.711 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/vfs.h... yes 2023-10-26T06:48:02.845 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/limits.h usability... no 2023-10-26T06:48:02.878 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/limits.h presence... no 2023-10-26T06:48:02.879 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/limits.h... no 2023-10-26T06:48:02.938 INFO:tasks.workunit.client.0.smithi089.stdout:checking for an ANSI C-conforming const... yes 2023-10-26T06:48:02.996 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-10-26T06:48:02.998 INFO:tasks.workunit.client.0.smithi089.stdout:checking for function prototypes... yes 2023-10-26T06:48:03.046 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether setvbuf arguments are reversed... no 2023-10-26T06:48:03.168 INFO:tasks.workunit.client.0.smithi089.stdout:checking for system... yes 2023-10-26T06:48:03.294 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gettimeofday... yes 2023-10-26T06:48:03.401 INFO:tasks.workunit.client.0.smithi089.stdout:checking for mkdir... yes 2023-10-26T06:48:03.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:03 smithi135 ceph-mon[131403]: pgmap v344: 97 pgs: 97 active+clean; 1.3 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2023-10-26T06:48:03.509 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strerror... yes 2023-10-26T06:48:03.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:03 smithi089 ceph-mon[95031]: pgmap v344: 97 pgs: 97 active+clean; 1.3 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2023-10-26T06:48:03.617 INFO:tasks.workunit.client.0.smithi089.stdout:checking for open64... yes 2023-10-26T06:48:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:03 smithi192 ceph-mon[131661]: pgmap v344: 97 pgs: 97 active+clean; 1.3 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2023-10-26T06:48:03.724 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stat64... yes 2023-10-26T06:48:03.833 INFO:tasks.workunit.client.0.smithi089.stdout:checking for fseeko64... yes 2023-10-26T06:48:03.943 INFO:tasks.workunit.client.0.smithi089.stdout:checking for lrand48_r... yes 2023-10-26T06:48:04.055 INFO:tasks.workunit.client.0.smithi089.stdout:checking for srand48_r... yes 2023-10-26T06:48:04.101 INFO:tasks.workunit.client.0.smithi089.stdout:configure: creating ./config.status 2023-10-26T06:48:04.324 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating Makefile 2023-10-26T06:48:04.354 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating config.h 2023-10-26T06:48:04.385 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: executing depfiles commands 2023-10-26T06:48:04.469 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: executing default commands 2023-10-26T06:48:04.542 INFO:tasks.workunit.client.0.smithi089.stderr:+ make 2023-10-26T06:48:04.553 INFO:tasks.workunit.client.0.smithi089.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2023-10-26T06:48:04.558 INFO:tasks.workunit.client.0.smithi089.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2023-10-26T06:48:04.560 INFO:tasks.workunit.client.0.smithi089.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2023-10-26T06:48:04.560 INFO:tasks.workunit.client.0.smithi089.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2023-10-26T06:48:04.560 INFO:tasks.workunit.client.0.smithi089.stderr: to install the `Automake' and `Perl' packages. Grab them from 2023-10-26T06:48:04.560 INFO:tasks.workunit.client.0.smithi089.stderr: any GNU archive site. 2023-10-26T06:48:04.564 INFO:tasks.workunit.client.0.smithi089.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2023-10-26T06:48:04.567 INFO:tasks.workunit.client.0.smithi089.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2023-10-26T06:48:04.569 INFO:tasks.workunit.client.0.smithi089.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2023-10-26T06:48:04.570 INFO:tasks.workunit.client.0.smithi089.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2023-10-26T06:48:04.570 INFO:tasks.workunit.client.0.smithi089.stderr: You might want to install the `Automake' and `Perl' packages. 2023-10-26T06:48:04.570 INFO:tasks.workunit.client.0.smithi089.stderr: Grab them from any GNU archive site. 2023-10-26T06:48:04.626 INFO:tasks.workunit.client.0.smithi089.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2023-10-26T06:48:04.947 INFO:tasks.workunit.client.0.smithi089.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2023-10-26T06:48:04.948 INFO:tasks.workunit.client.0.smithi089.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2023-10-26T06:48:04.948 INFO:tasks.workunit.client.0.smithi089.stderr:If you have problems, you may need to regenerate the build system entirely. 2023-10-26T06:48:04.948 INFO:tasks.workunit.client.0.smithi089.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2023-10-26T06:48:05.106 INFO:tasks.workunit.client.0.smithi089.stdout:/bin/sh ./config.status --recheck 2023-10-26T06:48:05.232 INFO:tasks.workunit.client.0.smithi089.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2023-10-26T06:48:05.425 INFO:tasks.workunit.client.0.smithi089.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-26T06:48:05.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:05 smithi135 ceph-mon[131403]: pgmap v345: 97 pgs: 97 active+clean; 3.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-10-26T06:48:05.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:05 smithi089 ceph-mon[95031]: pgmap v345: 97 pgs: 97 active+clean; 3.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-10-26T06:48:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:05 smithi192 ceph-mon[131661]: pgmap v345: 97 pgs: 97 active+clean; 3.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-10-26T06:48:06.437 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether build environment is sane... yes 2023-10-26T06:48:06.449 INFO:tasks.workunit.client.0.smithi089.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2023-10-26T06:48:06.449 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gawk... gawk 2023-10-26T06:48:06.479 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether make sets $(MAKE)... yes 2023-10-26T06:48:06.555 INFO:tasks.workunit.client.0.smithi089.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-10-26T06:48:06.556 INFO:tasks.workunit.client.0.smithi089.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-10-26T06:48:06.573 INFO:tasks.workunit.client.0.smithi089.stdout:checking for style of include used by make... GNU 2023-10-26T06:48:06.580 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc... gcc 2023-10-26T06:48:06.713 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether the C compiler works... yes 2023-10-26T06:48:06.713 INFO:tasks.workunit.client.0.smithi089.stdout:checking for C compiler default output file name... a.out 2023-10-26T06:48:06.776 INFO:tasks.workunit.client.0.smithi089.stdout:checking for suffix of executables... 2023-10-26T06:48:06.855 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are cross compiling... no 2023-10-26T06:48:06.916 INFO:tasks.workunit.client.0.smithi089.stdout:checking for suffix of object files... o 2023-10-26T06:48:06.956 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are using the GNU C compiler... yes 2023-10-26T06:48:06.996 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether gcc accepts -g... yes 2023-10-26T06:48:07.051 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc option to accept ISO C89... none needed 2023-10-26T06:48:07.196 INFO:tasks.workunit.client.0.smithi089.stdout:checking dependency style of gcc... gcc3 2023-10-26T06:48:07.258 INFO:tasks.workunit.client.0.smithi089.stdout:checking how to run the C preprocessor... gcc -E 2023-10-26T06:48:07.324 INFO:tasks.workunit.client.0.smithi089.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-10-26T06:48:07.327 INFO:tasks.workunit.client.0.smithi089.stdout:checking for egrep... /usr/bin/grep -E 2023-10-26T06:48:07.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:07 smithi135 ceph-mon[131403]: pgmap v346: 97 pgs: 97 active+clean; 4.3 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 297 KiB/s wr, 37 op/s 2023-10-26T06:48:07.519 INFO:tasks.workunit.client.0.smithi089.stdout:checking for ANSI C header files... yes 2023-10-26T06:48:07.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:07 smithi089 ceph-mon[95031]: pgmap v346: 97 pgs: 97 active+clean; 4.3 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 297 KiB/s wr, 37 op/s 2023-10-26T06:48:07.575 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/types.h... yes 2023-10-26T06:48:07.642 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/stat.h... yes 2023-10-26T06:48:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:07 smithi192 ceph-mon[131661]: pgmap v346: 97 pgs: 97 active+clean; 4.3 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 297 KiB/s wr, 37 op/s 2023-10-26T06:48:07.709 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdlib.h... yes 2023-10-26T06:48:07.775 INFO:tasks.workunit.client.0.smithi089.stdout:checking for string.h... yes 2023-10-26T06:48:07.856 INFO:tasks.workunit.client.0.smithi089.stdout:checking for memory.h... yes 2023-10-26T06:48:07.927 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strings.h... yes 2023-10-26T06:48:08.000 INFO:tasks.workunit.client.0.smithi089.stdout:checking for inttypes.h... yes 2023-10-26T06:48:08.075 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdint.h... yes 2023-10-26T06:48:08.150 INFO:tasks.workunit.client.0.smithi089.stdout:checking for unistd.h... yes 2023-10-26T06:48:08.223 INFO:tasks.workunit.client.0.smithi089.stdout:checking minix/config.h usability... no 2023-10-26T06:48:08.258 INFO:tasks.workunit.client.0.smithi089.stdout:checking minix/config.h presence... no 2023-10-26T06:48:08.258 INFO:tasks.workunit.client.0.smithi089.stdout:checking for minix/config.h... no 2023-10-26T06:48:08.315 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2023-10-26T06:48:08.315 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc... (cached) gcc 2023-10-26T06:48:08.366 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-10-26T06:48:08.366 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether gcc accepts -g... (cached) yes 2023-10-26T06:48:08.366 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-10-26T06:48:08.367 INFO:tasks.workunit.client.0.smithi089.stdout:checking dependency style of gcc... (cached) gcc3 2023-10-26T06:48:08.433 INFO:tasks.workunit.client.0.smithi089.stdout:checking for main in -lm... yes 2023-10-26T06:48:08.504 INFO:tasks.workunit.client.0.smithi089.stdout:checking for main in -lpthread... yes 2023-10-26T06:48:08.507 INFO:tasks.workunit.client.0.smithi089.stdout:checking for ANSI C header files... (cached) yes 2023-10-26T06:48:08.557 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2023-10-26T06:48:08.651 INFO:tasks.workunit.client.0.smithi089.stdout:checking pthread.h usability... yes 2023-10-26T06:48:08.674 INFO:tasks.workunit.client.0.smithi089.stdout:checking pthread.h presence... yes 2023-10-26T06:48:08.675 INFO:tasks.workunit.client.0.smithi089.stdout:checking for pthread.h... yes 2023-10-26T06:48:08.758 INFO:tasks.workunit.client.0.smithi089.stdout:checking fcntl.h usability... yes 2023-10-26T06:48:08.779 INFO:tasks.workunit.client.0.smithi089.stdout:checking fcntl.h presence... yes 2023-10-26T06:48:08.779 INFO:tasks.workunit.client.0.smithi089.stdout:checking for fcntl.h... yes 2023-10-26T06:48:08.863 INFO:tasks.workunit.client.0.smithi089.stdout:checking limits.h usability... yes 2023-10-26T06:48:08.884 INFO:tasks.workunit.client.0.smithi089.stdout:checking limits.h presence... yes 2023-10-26T06:48:08.884 INFO:tasks.workunit.client.0.smithi089.stdout:checking for limits.h... yes 2023-10-26T06:48:08.915 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdint.h... (cached) yes 2023-10-26T06:48:08.981 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/time.h usability... yes 2023-10-26T06:48:09.003 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/time.h presence... yes 2023-10-26T06:48:09.003 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/time.h... yes 2023-10-26T06:48:09.032 INFO:tasks.workunit.client.0.smithi089.stdout:checking for unistd.h... (cached) yes 2023-10-26T06:48:09.097 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/vfs.h usability... yes 2023-10-26T06:48:09.118 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/vfs.h presence... yes 2023-10-26T06:48:09.118 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/vfs.h... yes 2023-10-26T06:48:09.204 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/limits.h usability... no 2023-10-26T06:48:09.240 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/limits.h presence... no 2023-10-26T06:48:09.241 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/limits.h... no 2023-10-26T06:48:09.283 INFO:tasks.workunit.client.0.smithi089.stdout:checking for an ANSI C-conforming const... yes 2023-10-26T06:48:09.329 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-10-26T06:48:09.426 INFO:tasks.workunit.client.0.smithi089.stdout:checking for system... yes 2023-10-26T06:48:09.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:09 smithi135 ceph-mon[131403]: pgmap v347: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 466 KiB/s wr, 55 op/s 2023-10-26T06:48:09.523 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gettimeofday... yes 2023-10-26T06:48:09.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:09 smithi089 ceph-mon[95031]: pgmap v347: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 466 KiB/s wr, 55 op/s 2023-10-26T06:48:09.622 INFO:tasks.workunit.client.0.smithi089.stdout:checking for mkdir... yes 2023-10-26T06:48:09.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:09 smithi192 ceph-mon[131661]: pgmap v347: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 466 KiB/s wr, 55 op/s 2023-10-26T06:48:09.720 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strerror... yes 2023-10-26T06:48:09.818 INFO:tasks.workunit.client.0.smithi089.stdout:checking for open64... yes 2023-10-26T06:48:09.918 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stat64... yes 2023-10-26T06:48:10.015 INFO:tasks.workunit.client.0.smithi089.stdout:checking for fseeko64... yes 2023-10-26T06:48:10.114 INFO:tasks.workunit.client.0.smithi089.stdout:checking for lrand48_r... yes 2023-10-26T06:48:10.212 INFO:tasks.workunit.client.0.smithi089.stdout:checking for srand48_r... yes 2023-10-26T06:48:10.260 INFO:tasks.workunit.client.0.smithi089.stdout:configure: creating ./config.status 2023-10-26T06:48:10.386 INFO:tasks.workunit.client.0.smithi089.stdout: /bin/sh ./config.status 2023-10-26T06:48:10.484 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating Makefile 2023-10-26T06:48:10.510 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating config.h 2023-10-26T06:48:10.542 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: executing depfiles commands 2023-10-26T06:48:10.574 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: executing default commands 2023-10-26T06:48:10.613 INFO:tasks.workunit.client.0.smithi089.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2023-10-26T06:48:10.807 INFO:tasks.workunit.client.0.smithi089.stdout:rm -f stamp-h1 2023-10-26T06:48:10.810 INFO:tasks.workunit.client.0.smithi089.stdout:touch config.h.in 2023-10-26T06:48:10.813 INFO:tasks.workunit.client.0.smithi089.stdout:cd . && /bin/sh ./config.status config.h 2023-10-26T06:48:10.894 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating config.h 2023-10-26T06:48:10.936 INFO:tasks.workunit.client.0.smithi089.stdout:make all-am 2023-10-26T06:48:10.950 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2023-10-26T06:48:10.951 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2023-10-26T06:48:11.126 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2023-10-26T06:48:11.131 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2023-10-26T06:48:11.227 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2023-10-26T06:48:11.231 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2023-10-26T06:48:11.346 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/main.Tpo .deps/main.Po 2023-10-26T06:48:11.350 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2023-10-26T06:48:11.466 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2023-10-26T06:48:11.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:11 smithi135 ceph-mon[131403]: pgmap v348: 97 pgs: 97 active+clean; 7.6 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 610 KiB/s wr, 60 op/s 2023-10-26T06:48:11.471 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2023-10-26T06:48:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:11 smithi089 ceph-mon[95031]: pgmap v348: 97 pgs: 97 active+clean; 7.6 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 610 KiB/s wr, 60 op/s 2023-10-26T06:48:11.611 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2023-10-26T06:48:11.616 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2023-10-26T06:48:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:11 smithi192 ceph-mon[131661]: pgmap v348: 97 pgs: 97 active+clean; 7.6 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 610 KiB/s wr, 60 op/s 2023-10-26T06:48:11.685 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2023-10-26T06:48:11.689 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2023-10-26T06:48:11.748 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2023-10-26T06:48:11.753 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2023-10-26T06:48:11.818 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2023-10-26T06:48:11.822 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2023-10-26T06:48:12.002 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2023-10-26T06:48:12.006 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2023-10-26T06:48:12.141 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2023-10-26T06:48:12.146 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2023-10-26T06:48:12.307 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2023-10-26T06:48:12.313 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2023-10-26T06:48:12.397 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2023-10-26T06:48:12.402 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2023-10-26T06:48:12.490 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2023-10-26T06:48:12.495 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2023-10-26T06:48:12.617 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/util.Tpo .deps/util.Po 2023-10-26T06:48:12.622 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2023-10-26T06:48:12.660 INFO:tasks.workunit.client.0.smithi089.stderr:In file included from parser.h:22, 2023-10-26T06:48:12.661 INFO:tasks.workunit.client.0.smithi089.stderr: from parser.c:25: 2023-10-26T06:48:12.661 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c: In function ‘init_filesys’: 2023-10-26T06:48:12.661 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.661 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.661 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.662 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.662 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.662 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.662 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.663 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.663 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.663 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.663 INFO:tasks.workunit.client.0.smithi089.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-26T06:48:12.663 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.664 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.664 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2023-10-26T06:48:12.664 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.664 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.664 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.664 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.665 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.665 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.665 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.665 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.665 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.666 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.666 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.666 INFO:tasks.workunit.client.0.smithi089.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-26T06:48:12.666 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.666 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.667 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2023-10-26T06:48:12.667 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.667 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.667 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.667 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.668 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.668 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.668 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.668 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.669 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.669 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.670 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.670 INFO:tasks.workunit.client.0.smithi089.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-26T06:48:12.670 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.671 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.671 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2023-10-26T06:48:12.672 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.672 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.672 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.673 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.673 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.673 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.674 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.674 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.674 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.675 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.675 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.676 INFO:tasks.workunit.client.0.smithi089.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-26T06:48:12.676 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.676 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.677 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2023-10-26T06:48:12.677 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.677 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c: In function ‘init_tg_stats’: 2023-10-26T06:48:12.678 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.678 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.679 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.679 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.679 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.680 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.680 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.681 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.681 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.681 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.682 INFO:tasks.workunit.client.0.smithi089.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-26T06:48:12.682 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.682 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.683 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, 2023-10-26T06:48:12.683 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.683 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.684 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.684 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.685 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.685 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.685 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.686 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.686 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.686 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.687 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.687 INFO:tasks.workunit.client.0.smithi089.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-26T06:48:12.687 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.688 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.688 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, 2023-10-26T06:48:12.688 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.689 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.689 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.690 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.690 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.690 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.691 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.691 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.691 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.692 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.692 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.692 INFO:tasks.workunit.client.0.smithi089.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-10-26T06:48:12.692 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.693 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.693 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, 2023-10-26T06:48:12.693 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.694 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-10-26T06:48:12.694 INFO:tasks.workunit.client.0.smithi089.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-10-26T06:48:12.694 INFO:tasks.workunit.client.0.smithi089.stderr: ^ 2023-10-26T06:48:12.694 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-10-26T06:48:12.695 INFO:tasks.workunit.client.0.smithi089.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-10-26T06:48:12.695 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~ 2023-10-26T06:48:12.695 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-10-26T06:48:12.695 INFO:tasks.workunit.client.0.smithi089.stderr: container_of(ptr, type, member) 2023-10-26T06:48:12.695 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~ 2023-10-26T06:48:12.695 INFO:tasks.workunit.client.0.smithi089.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-10-26T06:48:12.696 INFO:tasks.workunit.client.0.smithi089.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-10-26T06:48:12.696 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~ 2023-10-26T06:48:12.696 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2023-10-26T06:48:12.696 INFO:tasks.workunit.client.0.smithi089.stderr: list_for_each_entry(tmp_list, 2023-10-26T06:48:12.696 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.775 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c: In function ‘set_weight’: 2023-10-26T06:48:12.775 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:623:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2023-10-26T06:48:12.776 INFO:tasks.workunit.client.0.smithi089.stderr: strncpy (op, tmp_config->name, len - 7); 2023-10-26T06:48:12.776 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.776 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:620:10: note: length computed here 2023-10-26T06:48:12.776 INFO:tasks.workunit.client.0.smithi089.stderr: len = strlen(tmp_config->name); 2023-10-26T06:48:12.776 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.780 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c: In function ‘get_weight_total’: 2023-10-26T06:48:12.780 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:645:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2023-10-26T06:48:12.780 INFO:tasks.workunit.client.0.smithi089.stderr: strncpy (op, tmp_config->name, len - 7); 2023-10-26T06:48:12.780 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.780 INFO:tasks.workunit.client.0.smithi089.stderr:parser.c:642:10: note: length computed here 2023-10-26T06:48:12.781 INFO:tasks.workunit.client.0.smithi089.stderr: len = strlen(tmp_config->name); 2023-10-26T06:48:12.781 INFO:tasks.workunit.client.0.smithi089.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-10-26T06:48:12.917 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2023-10-26T06:48:12.922 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2023-10-26T06:48:12.997 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2023-10-26T06:48:13.002 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2023-10-26T06:48:13.109 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2023-10-26T06:48:13.114 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2023-10-26T06:48:13.159 INFO:tasks.workunit.client.0.smithi089.stdout:mv -f .deps/list.Tpo .deps/list.Po 2023-10-26T06:48:13.165 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2023-10-26T06:48:13.214 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2023-10-26T06:48:13.215 INFO:tasks.workunit.client.0.smithi089.stderr:+ cd .. 2023-10-26T06:48:13.215 INFO:tasks.workunit.client.0.smithi089.stderr:+ mkdir tmp 2023-10-26T06:48:13.220 INFO:tasks.workunit.client.0.smithi089.stderr:+ cd tmp 2023-10-26T06:48:13.220 INFO:tasks.workunit.client.0.smithi089.stderr:+ for f in $mydir/*.ffsb 2023-10-26T06:48:13.220 INFO:tasks.workunit.client.0.smithi089.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2023-10-26T06:48:13.221 INFO:tasks.workunit.client.0.smithi089.stdout:FFSB version 6.0-RC2 started 2023-10-26T06:48:13.221 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.228 INFO:tasks.workunit.client.0.smithi089.stdout:benchmark time = 300 2023-10-26T06:48:13.228 INFO:tasks.workunit.client.0.smithi089.stdout:ThreadGroup 0 2023-10-26T06:48:13.228 INFO:tasks.workunit.client.0.smithi089.stdout:================ 2023-10-26T06:48:13.228 INFO:tasks.workunit.client.0.smithi089.stdout: num_threads = 32 2023-10-26T06:48:13.228 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.229 INFO:tasks.workunit.client.0.smithi089.stdout: read_random = off 2023-10-26T06:48:13.229 INFO:tasks.workunit.client.0.smithi089.stdout: read_size = 0 (0B) 2023-10-26T06:48:13.229 INFO:tasks.workunit.client.0.smithi089.stdout: read_blocksize = 0 (0B) 2023-10-26T06:48:13.229 INFO:tasks.workunit.client.0.smithi089.stdout: read_skip = off 2023-10-26T06:48:13.230 INFO:tasks.workunit.client.0.smithi089.stdout: read_skipsize = 0 (0B) 2023-10-26T06:48:13.230 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.230 INFO:tasks.workunit.client.0.smithi089.stdout: write_random = on 2023-10-26T06:48:13.230 INFO:tasks.workunit.client.0.smithi089.stdout: write_size = 5242880 (5MB) 2023-10-26T06:48:13.230 INFO:tasks.workunit.client.0.smithi089.stdout: fsync_file = 0 2023-10-26T06:48:13.231 INFO:tasks.workunit.client.0.smithi089.stdout: write_blocksize = 4096 (4KB) 2023-10-26T06:48:13.231 INFO:tasks.workunit.client.0.smithi089.stdout: wait time = 0 2023-10-26T06:48:13.231 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.231 INFO:tasks.workunit.client.0.smithi089.stdout: op weights 2023-10-26T06:48:13.231 INFO:tasks.workunit.client.0.smithi089.stdout: read = 0 (0.00%) 2023-10-26T06:48:13.232 INFO:tasks.workunit.client.0.smithi089.stdout: readall = 0 (0.00%) 2023-10-26T06:48:13.232 INFO:tasks.workunit.client.0.smithi089.stdout: write = 1 (100.00%) 2023-10-26T06:48:13.232 INFO:tasks.workunit.client.0.smithi089.stdout: create = 0 (0.00%) 2023-10-26T06:48:13.232 INFO:tasks.workunit.client.0.smithi089.stdout: append = 0 (0.00%) 2023-10-26T06:48:13.232 INFO:tasks.workunit.client.0.smithi089.stdout: delete = 0 (0.00%) 2023-10-26T06:48:13.233 INFO:tasks.workunit.client.0.smithi089.stdout: metaop = 0 (0.00%) 2023-10-26T06:48:13.233 INFO:tasks.workunit.client.0.smithi089.stdout: createdir = 0 (0.00%) 2023-10-26T06:48:13.233 INFO:tasks.workunit.client.0.smithi089.stdout: stat = 0 (0.00%) 2023-10-26T06:48:13.233 INFO:tasks.workunit.client.0.smithi089.stdout: writeall = 0 (0.00%) 2023-10-26T06:48:13.233 INFO:tasks.workunit.client.0.smithi089.stdout: writeall_fsync = 0 (0.00%) 2023-10-26T06:48:13.233 INFO:tasks.workunit.client.0.smithi089.stdout: open_close = 0 (0.00%) 2023-10-26T06:48:13.234 INFO:tasks.workunit.client.0.smithi089.stdout: write_fsync = 0 (0.00%) 2023-10-26T06:48:13.234 INFO:tasks.workunit.client.0.smithi089.stdout: create_fsync = 0 (0.00%) 2023-10-26T06:48:13.234 INFO:tasks.workunit.client.0.smithi089.stdout: append_fsync = 0 (0.00%) 2023-10-26T06:48:13.234 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.234 INFO:tasks.workunit.client.0.smithi089.stdout:FileSystem . 2023-10-26T06:48:13.235 INFO:tasks.workunit.client.0.smithi089.stdout:========== 2023-10-26T06:48:13.235 INFO:tasks.workunit.client.0.smithi089.stdout: num_dirs = 0 2023-10-26T06:48:13.235 INFO:tasks.workunit.client.0.smithi089.stdout: starting files = 128 2023-10-26T06:48:13.235 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.235 INFO:tasks.workunit.client.0.smithi089.stdout: min file size = 104857600 (100MB) 2023-10-26T06:48:13.235 INFO:tasks.workunit.client.0.smithi089.stdout: max file size = 104857600 (100MB) 2023-10-26T06:48:13.236 INFO:tasks.workunit.client.0.smithi089.stdout: directio = off 2023-10-26T06:48:13.236 INFO:tasks.workunit.client.0.smithi089.stdout: alignedio = on 2023-10-26T06:48:13.236 INFO:tasks.workunit.client.0.smithi089.stdout: bufferedio = off 2023-10-26T06:48:13.236 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.236 INFO:tasks.workunit.client.0.smithi089.stdout: aging is off 2023-10-26T06:48:13.237 INFO:tasks.workunit.client.0.smithi089.stdout: current utilization = 0.00% 2023-10-26T06:48:13.237 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:48:13.237 INFO:tasks.workunit.client.0.smithi089.stdout:checking existing fs: . 2023-10-26T06:48:13.238 INFO:tasks.workunit.client.0.smithi089.stdout:recreating new fileset 2023-10-26T06:48:13.238 INFO:tasks.workunit.client.0.smithi089.stderr:opendir: No such file or directory 2023-10-26T06:48:13.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:13 smithi135 ceph-mon[131403]: pgmap v349: 97 pgs: 97 active+clean; 7.6 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 613 KiB/s wr, 61 op/s 2023-10-26T06:48:13.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:13 smithi089 ceph-mon[95031]: pgmap v349: 97 pgs: 97 active+clean; 7.6 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 613 KiB/s wr, 61 op/s 2023-10-26T06:48:13.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:13 smithi192 ceph-mon[131661]: pgmap v349: 97 pgs: 97 active+clean; 7.6 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 613 KiB/s wr, 61 op/s 2023-10-26T06:48:15.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:15 smithi135 ceph-mon[131403]: pgmap v350: 97 pgs: 97 active+clean; 7.6 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 614 KiB/s wr, 62 op/s 2023-10-26T06:48:15.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:15 smithi089 ceph-mon[95031]: pgmap v350: 97 pgs: 97 active+clean; 7.6 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 614 KiB/s wr, 62 op/s 2023-10-26T06:48:15.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:15 smithi192 ceph-mon[131661]: pgmap v350: 97 pgs: 97 active+clean; 7.6 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 614 KiB/s wr, 62 op/s 2023-10-26T06:48:17.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:17 smithi135 ceph-mon[131403]: pgmap v351: 97 pgs: 97 active+clean; 8.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 490 KiB/s wr, 59 op/s 2023-10-26T06:48:17.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:17 smithi089 ceph-mon[95031]: pgmap v351: 97 pgs: 97 active+clean; 8.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 490 KiB/s wr, 59 op/s 2023-10-26T06:48:17.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:17 smithi192 ceph-mon[131661]: pgmap v351: 97 pgs: 97 active+clean; 8.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 490 KiB/s wr, 59 op/s 2023-10-26T06:48:18.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:48:18.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:18 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:48:18.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:18 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:48:18.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:18.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:18 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:19 smithi135 ceph-mon[131403]: pgmap v352: 97 pgs: 97 active+clean; 8.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 407 KiB/s wr, 34 op/s 2023-10-26T06:48:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:19 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:48:19.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:19 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:48:19.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:19 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:19.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:19 smithi089 ceph-mon[95031]: pgmap v352: 97 pgs: 97 active+clean; 8.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 407 KiB/s wr, 34 op/s 2023-10-26T06:48:19.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:19 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:48:19.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:19 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:48:19.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:19 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:19.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:19 smithi192 ceph-mon[131661]: pgmap v352: 97 pgs: 97 active+clean; 8.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 407 KiB/s wr, 34 op/s 2023-10-26T06:48:19.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:19 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:48:19.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:19 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:48:19.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:19 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:48:21.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:21 smithi135 ceph-mon[131403]: pgmap v353: 97 pgs: 97 active+clean; 27 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.8 MiB/s wr, 39 op/s 2023-10-26T06:48:21.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:21 smithi089 ceph-mon[95031]: pgmap v353: 97 pgs: 97 active+clean; 27 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.8 MiB/s wr, 39 op/s 2023-10-26T06:48:21.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:21 smithi192 ceph-mon[131661]: pgmap v353: 97 pgs: 97 active+clean; 27 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.8 MiB/s wr, 39 op/s 2023-10-26T06:48:23.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:23 smithi089 ceph-mon[95031]: pgmap v354: 97 pgs: 97 active+clean; 51 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 3.7 MiB/s wr, 43 op/s 2023-10-26T06:48:23.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:23 smithi192 ceph-mon[131661]: pgmap v354: 97 pgs: 97 active+clean; 51 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 3.7 MiB/s wr, 43 op/s 2023-10-26T06:48:23.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:23 smithi135 ceph-mon[131403]: pgmap v354: 97 pgs: 97 active+clean; 51 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 3.7 MiB/s wr, 43 op/s 2023-10-26T06:48:25.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:25 smithi089 ceph-mon[95031]: pgmap v355: 97 pgs: 97 active+clean; 292 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 24 MiB/s wr, 64 op/s 2023-10-26T06:48:25.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:25 smithi192 ceph-mon[131661]: pgmap v355: 97 pgs: 97 active+clean; 292 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 24 MiB/s wr, 64 op/s 2023-10-26T06:48:25.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:25 smithi135 ceph-mon[131403]: pgmap v355: 97 pgs: 97 active+clean; 292 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 24 MiB/s wr, 64 op/s 2023-10-26T06:48:27.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:48:27.551 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:27 smithi089 ceph-mon[95031]: pgmap v356: 97 pgs: 97 active+clean; 916 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 76 MiB/s wr, 93 op/s 2023-10-26T06:48:27.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:27 smithi192 ceph-mon[131661]: pgmap v356: 97 pgs: 97 active+clean; 916 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 76 MiB/s wr, 93 op/s 2023-10-26T06:48:27.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:27 smithi135 ceph-mon[131403]: pgmap v356: 97 pgs: 97 active+clean; 916 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 76 MiB/s wr, 93 op/s 2023-10-26T06:48:27.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.769+0000 7f77bbbff700 1 -- 172.21.15.89:0/2148832688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b410c0e0 msgr2=0x7f77b4113c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.769+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2148832688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b410c0e0 0x7f77b4113c70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b0a0 tx=0x7f77ac00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.769+0000 7f77bbbff700 1 -- 172.21.15.89:0/2148832688 shutdown_connections 2023-10-26T06:48:27.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.769+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2148832688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77b41078a0 0x7f77b4107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.769+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2148832688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b410c0e0 0x7f77b4113c70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.769+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2148832688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77b41081c0 0x7f77b410ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.769+0000 7f77bbbff700 1 -- 172.21.15.89:0/2148832688 >> 172.21.15.89:0/2148832688 conn(0x7f77b40765b0 msgr2=0x7f77b40789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:27.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.770+0000 7f77bbbff700 1 -- 172.21.15.89:0/2148832688 shutdown_connections 2023-10-26T06:48:27.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.771+0000 7f77bbbff700 1 -- 172.21.15.89:0/2148832688 wait complete. 2023-10-26T06:48:27.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.772+0000 7f77bbbff700 1 Processor -- start 2023-10-26T06:48:27.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.772+0000 7f77bbbff700 1 -- start start 2023-10-26T06:48:27.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77bbbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77b41078a0 0x7f77b419d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77bbbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b41081c0 0x7f77b419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77bbbff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77b410c0e0 0x7f77b41a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77bbbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41a1fe0 con 0x7f77b41078a0 2023-10-26T06:48:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77bbbff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41a2150 con 0x7f77b410c0e0 2023-10-26T06:48:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77bbbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41a2310 con 0x7f77b41081c0 2023-10-26T06:48:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77b999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77b41078a0 0x7f77b419d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77b999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77b41078a0 0x7f77b419d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54838/0 (socket says 172.21.15.89:54838) 2023-10-26T06:48:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77b999b700 1 -- 172.21.15.89:0/2727781100 learned_addr learned my addr 172.21.15.89:0/2727781100 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:27.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77b919a700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b41081c0 0x7f77b419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.773+0000 7f77ba19c700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77b410c0e0 0x7f77b41a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77b919a700 1 -- 172.21.15.89:0/2727781100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77b410c0e0 msgr2=0x7f77b41a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77b919a700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77b410c0e0 0x7f77b41a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77b919a700 1 -- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77b41078a0 msgr2=0x7f77b419d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77b919a700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77b41078a0 0x7f77b419d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77b919a700 1 -- 172.21.15.89:0/2727781100 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac009d00 con 0x7f77b41081c0 2023-10-26T06:48:27.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77ba19c700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77b410c0e0 0x7f77b41a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:48:27.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77b919a700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b41081c0 0x7f77b419d600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77b000e9f0 tx=0x7f77b000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:27.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77a6ffd700 1 -- 172.21.15.89:0/2727781100 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b000eee0 con 0x7f77b41081c0 2023-10-26T06:48:27.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77a6ffd700 1 -- 172.21.15.89:0/2727781100 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77b000cbd0 con 0x7f77b41081c0 2023-10-26T06:48:27.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b41a25f0 con 0x7f77b41081c0 2023-10-26T06:48:27.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.775+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b41a9e30 con 0x7f77b41081c0 2023-10-26T06:48:27.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.774+0000 7f77a6ffd700 1 -- 172.21.15.89:0/2727781100 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0018680 con 0x7f77b41081c0 2023-10-26T06:48:27.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.779+0000 7f77a6ffd700 1 -- 172.21.15.89:0/2727781100 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f77b000cd40 con 0x7f77b41081c0 2023-10-26T06:48:27.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.779+0000 7f77cf501b80 1 Processor -- start 2023-10-26T06:48:27.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.780+0000 7f77cf501b80 1 -- start start 2023-10-26T06:48:27.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.780+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d7be0 0x557b746d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.780+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d8500 0x557b746d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.780+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.780+0000 7f77cf501b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b746456e0 con 0x557b746d8500 2023-10-26T06:48:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.780+0000 7f77cf501b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b7462bc60 con 0x557b746d7be0 2023-10-26T06:48:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.780+0000 7f77cf501b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b74629580 con 0x557b746d25d0 2023-10-26T06:48:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77b919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77b919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33342/0 (socket says 172.21.15.89:33342) 2023-10-26T06:48:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77b919a700 1 -- 172.21.15.89:0/3759442321 learned_addr learned my addr 172.21.15.89:0/3759442321 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77ba19c700 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d7be0 0x557b746d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77b999b700 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d8500 0x557b746d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77ba19c700 1 -- 172.21.15.89:0/3759442321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 msgr2=0x557b746d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77ba19c700 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77ba19c700 1 -- 172.21.15.89:0/3759442321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d8500 msgr2=0x557b746d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77ba19c700 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d8500 0x557b746d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.781+0000 7f77ba19c700 1 -- 172.21.15.89:0/3759442321 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b745e23f0 con 0x557b746d7be0 2023-10-26T06:48:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.782+0000 7f77ba19c700 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d7be0 0x557b746d7fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b780 tx=0x7f77ac009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22fda2c6679d066c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.782+0000 7f77a6ffd700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77a00a1670 0x7f77a00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.782+0000 7f779ffff700 1 -- 172.21.15.89:0/3759442321 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac00e050 con 0x557b746d7be0 2023-10-26T06:48:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.782+0000 7f779ffff700 1 -- 172.21.15.89:0/3759442321 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac004a50 con 0x557b746d7be0 2023-10-26T06:48:27.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.782+0000 7f779ffff700 1 -- 172.21.15.89:0/3759442321 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac007940 con 0x557b746d7be0 2023-10-26T06:48:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.782+0000 7f77a6ffd700 1 -- 172.21.15.89:0/2727781100 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f77b0014070 con 0x7f77b41081c0 2023-10-26T06:48:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77b999b700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77a00a1670 0x7f77a00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77cf501b80 1 -- 172.21.15.89:0/3759442321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d7be0 msgr2=0x557b746d7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77cf501b80 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d7be0 0x557b746d7fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b780 tx=0x7f77ac009360 comp rx=0 tx=0).stop 2023-10-26T06:48:27.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77cf501b80 1 -- 172.21.15.89:0/3759442321 shutdown_connections 2023-10-26T06:48:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77cf501b80 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77cf501b80 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d7be0 0x557b746d7fc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77cf501b80 1 --2- 172.21.15.89:0/3759442321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d8500 0x557b746d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.783+0000 7f77cf501b80 1 -- 172.21.15.89:0/3759442321 >> 172.21.15.89:0/3759442321 conn(0x557b74646720 msgr2=0x557b746ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.784+0000 7f77b999b700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77a00a1670 0x7f77a00a3b30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x557b746cda70 tx=0x7f77a800b4a0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.784+0000 7f77cf501b80 1 -- 172.21.15.89:0/3759442321 shutdown_connections 2023-10-26T06:48:27.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.784+0000 7f77cf501b80 1 -- 172.21.15.89:0/3759442321 wait complete. 2023-10-26T06:48:27.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.785+0000 7f77cf501b80 1 -- start start 2023-10-26T06:48:27.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.786+0000 7f77cf501b80 1 Processor -- start 2023-10-26T06:48:27.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77cf501b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b74624e70 con 0x557b746d7be0 2023-10-26T06:48:27.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77cf501b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b745c1710 con 0x557b746d8500 2023-10-26T06:48:27.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77cf501b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b745c13b0 con 0x557b746d25d0 2023-10-26T06:48:27.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77cf501b80 10 client.0 _ll_register_callbacks cb 0x557b7450ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:27.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77b999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.787+0000 7f77b999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746db890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33350/0 (socket says 172.21.15.89:33350) 2023-10-26T06:48:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b999b700 1 -- 172.21.15.89:0/4125724086 learned_addr learned my addr 172.21.15.89:0/4125724086 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77ba19c700 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b919a700 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b999b700 1 -- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 msgr2=0x557b746dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b999b700 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b999b700 1 -- 172.21.15.89:0/4125724086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 msgr2=0x557b746dbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b999b700 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746dbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b999b700 1 -- 172.21.15.89:0/4125724086 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b41144b0 con 0x557b746d25d0 2023-10-26T06:48:27.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f77b999b700 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746db890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77a801f4c0 tx=0x7f77a80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a88e7e3da3d62d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:27.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f779d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:48:27.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.788+0000 7f779d7fa700 1 -- 172.21.15.89:0/4125724086 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8016070 con 0x557b746d25d0 2023-10-26T06:48:27.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b74629ef0 con 0x557b746d25d0 2023-10-26T06:48:27.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f779d7fa700 1 -- 172.21.15.89:0/4125724086 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f77a8027410 con 0x557b746d25d0 2023-10-26T06:48:27.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f779d7fa700 1 -- 172.21.15.89:0/4125724086 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77a80275d0 con 0x557b746d25d0 2023-10-26T06:48:27.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 10 client.25612 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:27.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557b7444fde0 con 0x557b746d25d0 2023-10-26T06:48:27.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f779d7fa700 1 -- 172.21.15.89:0/4125724086 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b74629ef0 con 0x557b746d25d0 2023-10-26T06:48:27.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f779d7fa700 1 -- 172.21.15.89:0/4125724086 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77a8010570 con 0x557b746d25d0 2023-10-26T06:48:27.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:27.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:27.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 20 client.25612 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:27.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b746ebf40 0x557b746ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 4 client.25612 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:48:27.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.789+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b746a18b0 con 0x557b746ebf40 2023-10-26T06:48:27.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.790+0000 7f77b919a700 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b746ebf40 0x557b746ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.790+0000 7f779d7fa700 10 client.25612 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:27.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.790+0000 7f77b919a700 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b746ebf40 0x557b746ee360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc810b0085f13e31 server_cookie=97e123ab1c5737c3 in_seq=0 out_seq=0 2023-10-26T06:48:27.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f779d7fa700 1 -- 172.21.15.89:0/4125724086 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b746a18b0 con 0x557b746ebf40 2023-10-26T06:48:27.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f779d7fa700 10 client.25612 handle_command_reply: tid=0 2023-10-26T06:48:27.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 client.25612 shutdown 2023-10-26T06:48:27.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 msgr2=0x557b746db890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746db890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77a801f4c0 tx=0x7f77a80047c0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 shutdown_connections 2023-10-26T06:48:27.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b746ebf40 0x557b746ee360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746db890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 --2- 172.21.15.89:0/4125724086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746dbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 >> 172.21.15.89:0/4125724086 conn(0x557b74646720 msgr2=0x557b746d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:27.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f779d7fa700 0 client.25612 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:27.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.791+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 shutdown_connections 2023-10-26T06:48:27.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.792+0000 7f77cf501b80 1 -- 172.21.15.89:0/4125724086 wait complete. 2023-10-26T06:48:27.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.792+0000 7f77cf501b80 20 client.25612 trim_cache size 0 max 16384 2023-10-26T06:48:27.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 Processor -- start 2023-10-26T06:48:27.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 -- start start 2023-10-26T06:48:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b744a5920 con 0x557b746d7be0 2023-10-26T06:48:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b746e9620 con 0x557b746d8500 2023-10-26T06:48:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.810+0000 7f77cf501b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b746dce90 con 0x557b746d25d0 2023-10-26T06:48:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33364/0 (socket says 172.21.15.89:33364) 2023-10-26T06:48:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 -- 172.21.15.89:0/526125767 learned_addr learned my addr 172.21.15.89:0/526125767 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77ba19c700 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b919a700 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 -- 172.21.15.89:0/526125767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 msgr2=0x557b746dc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 -- 172.21.15.89:0/526125767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 msgr2=0x557b746e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 -- 172.21.15.89:0/526125767 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b74438c60 con 0x557b746d25d0 2023-10-26T06:48:27.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.811+0000 7f77b999b700 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d7060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f77a80108d0 tx=0x7f77a8010f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8431c485ae2b4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:27.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.812+0000 7f77a4bff700 1 -- 172.21.15.89:0/526125767 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a800bc30 con 0x557b746d25d0 2023-10-26T06:48:27.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.812+0000 7f77a4bff700 1 -- 172.21.15.89:0/526125767 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a80044e0 con 0x557b746d25d0 2023-10-26T06:48:27.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.812+0000 7f77a4bff700 1 -- 172.21.15.89:0/526125767 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8004cc0 con 0x557b746d25d0 2023-10-26T06:48:27.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.812+0000 7f77cf501b80 1 -- 172.21.15.89:0/526125767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 msgr2=0x557b746d7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.812+0000 7f77cf501b80 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d7060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f77a80108d0 tx=0x7f77a8010f70 comp rx=0 tx=0).stop 2023-10-26T06:48:27.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 -- 172.21.15.89:0/526125767 shutdown_connections 2023-10-26T06:48:27.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746dc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d25d0 0x557b746d7060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 --2- 172.21.15.89:0/526125767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d7be0 0x557b746e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 -- 172.21.15.89:0/526125767 >> 172.21.15.89:0/526125767 conn(0x557b74646720 msgr2=0x557b746d6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:27.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 -- 172.21.15.89:0/526125767 shutdown_connections 2023-10-26T06:48:27.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 -- 172.21.15.89:0/526125767 wait complete. 2023-10-26T06:48:27.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 -- start start 2023-10-26T06:48:27.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.813+0000 7f77cf501b80 1 Processor -- start 2023-10-26T06:48:27.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d25d0 0x557b746e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d7be0 0x557b746e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77cf501b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77cf501b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b746e2200 con 0x557b746d25d0 2023-10-26T06:48:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77cf501b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b746e2370 con 0x557b746d8500 2023-10-26T06:48:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77cf501b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b7473fd90 con 0x557b746d7be0 2023-10-26T06:48:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77cf501b80 10 client.0 _ll_register_callbacks cb 0x557b7450ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77b999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d25d0 0x557b746e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77ba19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77b999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d25d0 0x557b746e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54884/0 (socket says 172.21.15.89:54884) 2023-10-26T06:48:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77b999b700 1 -- 172.21.15.89:0/2433293881 learned_addr learned my addr 172.21.15.89:0/2433293881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.814+0000 7f77b919a700 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d7be0 0x557b746e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77ba19c700 1 -- 172.21.15.89:0/2433293881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d7be0 msgr2=0x557b746e1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77ba19c700 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d7be0 0x557b746e1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77ba19c700 1 -- 172.21.15.89:0/2433293881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d25d0 msgr2=0x557b746e2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77ba19c700 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d25d0 0x557b746e2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77ba19c700 1 -- 172.21.15.89:0/2433293881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b7444fde0 con 0x557b746d8500 2023-10-26T06:48:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77b999b700 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d25d0 0x557b746e2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:48:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77ba19c700 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746e1af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b070 tx=0x7f77ac0076d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edee06da14534f35 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f779d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:48:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f779d7fa700 1 -- 172.21.15.89:0/2433293881 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02f630 con 0x557b746d8500 2023-10-26T06:48:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f779d7fa700 1 -- 172.21.15.89:0/2433293881 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f77ac02f790 con 0x557b746d8500 2023-10-26T06:48:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.815+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b746d3040 con 0x557b746d8500 2023-10-26T06:48:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f779d7fa700 1 -- 172.21.15.89:0/2433293881 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77ac00e020 con 0x557b746d8500 2023-10-26T06:48:27.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 10 client.25616 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:27.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557b74476250 con 0x557b746d8500 2023-10-26T06:48:27.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f779d7fa700 1 -- 172.21.15.89:0/2433293881 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac00be70 con 0x557b746d8500 2023-10-26T06:48:27.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f779d7fa700 1 -- 172.21.15.89:0/2433293881 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77ac00b0d0 con 0x557b746d8500 2023-10-26T06:48:27.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 10 client.25616 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 20 client.25616 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b747410e0 0x557b747434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 4 client.25616 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:48:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.816+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b746d3040 con 0x557b747410e0 2023-10-26T06:48:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.817+0000 7f77b999b700 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b747410e0 0x557b747434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.817+0000 7f779d7fa700 10 client.25616 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.817+0000 7f77b999b700 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b747410e0 0x557b747434d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb85e57db94b8b6 server_cookie=6650c8a3ea079c9a in_seq=0 out_seq=0 2023-10-26T06:48:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.817+0000 7f779d7fa700 1 -- 172.21.15.89:0/2433293881 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557b746d3040 con 0x557b747410e0 2023-10-26T06:48:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.817+0000 7f779d7fa700 10 client.25616 handle_command_reply: tid=0 2023-10-26T06:48:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.817+0000 7f77cf501b80 1 client.25616 shutdown 2023-10-26T06:48:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 msgr2=0x557b746e1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746e1af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b070 tx=0x7f77ac0076d0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 shutdown_connections 2023-10-26T06:48:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557b746d25d0 0x557b746e2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557b747410e0 0x557b747434d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557b746d8500 0x557b746e1af0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 --2- 172.21.15.89:0/2433293881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b746d7be0 0x557b746e1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f779d7fa700 0 client.25616 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 >> 172.21.15.89:0/2433293881 conn(0x557b74646720 msgr2=0x557b746c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 shutdown_connections 2023-10-26T06:48:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 1 -- 172.21.15.89:0/2433293881 wait complete. 2023-10-26T06:48:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.818+0000 7f77cf501b80 20 client.25616 trim_cache size 0 max 16384 2023-10-26T06:48:27.833 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:48:27.833 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:48:27.833 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:48:27.833 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:48:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77a00a1670 msgr2=0x7f77a00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77a00a1670 0x7f77a00a3b30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x557b746cda70 tx=0x7f77a800b4a0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b41081c0 msgr2=0x7f77b419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b41081c0 0x7f77b419d600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77b000e9f0 tx=0x7f77b000ed00 comp rx=0 tx=0).stop 2023-10-26T06:48:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 shutdown_connections 2023-10-26T06:48:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77a00a1670 0x7f77a00a3b30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77b41078a0 0x7f77b419d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77b410c0e0 0x7f77b41a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 --2- 172.21.15.89:0/2727781100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77b41081c0 0x7f77b419d600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 >> 172.21.15.89:0/2727781100 conn(0x7f77b40765b0 msgr2=0x7f77b410d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 shutdown_connections 2023-10-26T06:48:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:27.819+0000 7f77bbbff700 1 -- 172.21.15.89:0/2727781100 wait complete. 2023-10-26T06:48:27.838 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:48:27.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:48:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.230+0000 7fd92099d700 1 -- 172.21.15.89:0/1197870917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10e3c0 msgr2=0x7fd91c11bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:28.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.234+0000 7fd92099d700 1 --2- 172.21.15.89:0/1197870917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10e3c0 0x7fd91c11bfe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b0a0 tx=0x7fd91400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.237+0000 7fd92099d700 1 -- 172.21.15.89:0/1197870917 shutdown_connections 2023-10-26T06:48:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.237+0000 7fd92099d700 1 --2- 172.21.15.89:0/1197870917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10e3c0 0x7fd91c11bfe0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.237+0000 7fd92099d700 1 --2- 172.21.15.89:0/1197870917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91c10ed30 0x7fd91c10f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.237+0000 7fd92099d700 1 --2- 172.21.15.89:0/1197870917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd91c10d970 0x7fd91c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.237+0000 7fd92099d700 1 -- 172.21.15.89:0/1197870917 >> 172.21.15.89:0/1197870917 conn(0x7fd91c06b580 msgr2=0x7fd91c06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:28.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.239+0000 7fd92099d700 1 -- 172.21.15.89:0/1197870917 shutdown_connections 2023-10-26T06:48:28.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.243+0000 7fd92099d700 1 -- 172.21.15.89:0/1197870917 wait complete. 2023-10-26T06:48:28.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.246+0000 7fd92099d700 1 Processor -- start 2023-10-26T06:48:28.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.247+0000 7fd92099d700 1 -- start start 2023-10-26T06:48:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.250+0000 7fd92099d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10d970 0x7fd91c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.250+0000 7fd92099d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd91c10e3c0 0x7fd91c1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.250+0000 7fd92099d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91c10ed30 0x7fd91c1a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.250+0000 7fd92099d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1aa380 con 0x7fd91c10d970 2023-10-26T06:48:28.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.250+0000 7fd92099d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1aa4f0 con 0x7fd91c10e3c0 2023-10-26T06:48:28.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.250+0000 7fd92099d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1aa6b0 con 0x7fd91c10ed30 2023-10-26T06:48:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10d970 0x7fd91c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91c10ed30 0x7fd91c1a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91c10ed30 0x7fd91c1a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33388/0 (socket says 172.21.15.89:33388) 2023-10-26T06:48:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91bfff700 1 -- 172.21.15.89:0/4077228881 learned_addr learned my addr 172.21.15.89:0/4077228881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:28.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91b7fe700 1 -- 172.21.15.89:0/4077228881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91c10ed30 msgr2=0x7fd91c1a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91b7fe700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91c10ed30 0x7fd91c1a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91b7fe700 1 -- 172.21.15.89:0/4077228881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd91c10e3c0 msgr2=0x7fd91c1a59a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-26T06:48:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91b7fe700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd91c10e3c0 0x7fd91c1a59a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.252+0000 7fd91b7fe700 1 -- 172.21.15.89:0/4077228881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914009d00 con 0x7fd91c10d970 2023-10-26T06:48:28.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.253+0000 7fd91b7fe700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10d970 0x7fd91c1a5460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c970 tx=0x7fd91000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.253+0000 7fd918ff9700 1 -- 172.21.15.89:0/4077228881 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910004440 con 0x7fd91c10d970 2023-10-26T06:48:28.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.253+0000 7fd918ff9700 1 -- 172.21.15.89:0/4077228881 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd910007bb0 con 0x7fd91c10d970 2023-10-26T06:48:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.253+0000 7fd918ff9700 1 -- 172.21.15.89:0/4077228881 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910003d80 con 0x7fd91c10d970 2023-10-26T06:48:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.253+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c1aa990 con 0x7fd91c10d970 2023-10-26T06:48:28.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.254+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c1b21d0 con 0x7fd91c10d970 2023-10-26T06:48:28.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.260+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c06c790 con 0x7fd91c10d970 2023-10-26T06:48:28.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.263+0000 7fd918ff9700 1 -- 172.21.15.89:0/4077228881 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd910007d20 con 0x7fd91c10d970 2023-10-26T06:48:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.267+0000 7fd918ff9700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9040a1670 0x7fd9040a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.267+0000 7fd918ff9700 1 -- 172.21.15.89:0/4077228881 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd9100cd0c0 con 0x7fd91c10d970 2023-10-26T06:48:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.272+0000 7fd91affd700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9040a1670 0x7fd9040a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.273+0000 7fd91affd700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9040a1670 0x7fd9040a3b30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd91c1a69b0 tx=0x7fd90c006c60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:28.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.282+0000 7fd918ff9700 1 -- 172.21.15.89:0/4077228881 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd910096420 con 0x7fd91c10d970 2023-10-26T06:48:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.802+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd91c0673e0 con 0x7fd91c10d970 2023-10-26T06:48:28.809 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:48:28.809 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:48:28.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.804+0000 7fd918ff9700 1 -- 172.21.15.89:0/4077228881 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fd91c0673e0 con 0x7fd91c10d970 2023-10-26T06:48:28.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.809+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9040a1670 msgr2=0x7fd9040a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:28.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.809+0000 7fd92099d700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9040a1670 0x7fd9040a3b30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd91c1a69b0 tx=0x7fd90c006c60 comp rx=0 tx=0).stop 2023-10-26T06:48:28.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.809+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10d970 msgr2=0x7fd91c1a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:28.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.809+0000 7fd92099d700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10d970 0x7fd91c1a5460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c970 tx=0x7fd91000cd30 comp rx=0 tx=0).stop 2023-10-26T06:48:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.810+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 shutdown_connections 2023-10-26T06:48:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.810+0000 7fd92099d700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd9040a1670 0x7fd9040a3b30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.810+0000 7fd92099d700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd91c10d970 0x7fd91c1a5460 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.810+0000 7fd92099d700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd91c10ed30 0x7fd91c1a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.810+0000 7fd92099d700 1 --2- 172.21.15.89:0/4077228881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd91c10e3c0 0x7fd91c1a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.810+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 >> 172.21.15.89:0/4077228881 conn(0x7fd91c06b580 msgr2=0x7fd91c119fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.812+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 shutdown_connections 2023-10-26T06:48:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:28.812+0000 7fd92099d700 1 -- 172.21.15.89:0/4077228881 wait complete. 2023-10-26T06:48:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:48:28.849 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:48:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.250+0000 7f641dd9d700 1 -- 172.21.15.89:0/2075840579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 msgr2=0x7f6418072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.250+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2075840579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 0x7f6418072f70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f640c009990 tx=0x7f640c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.250+0000 7f641dd9d700 1 -- 172.21.15.89:0/2075840579 shutdown_connections 2023-10-26T06:48:29.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.250+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2075840579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64181115c0 0x7f64181139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.250+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2075840579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 0x7f6418072f70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.250+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2075840579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64180734b0 0x7f6418111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.250+0000 7f641dd9d700 1 -- 172.21.15.89:0/2075840579 >> 172.21.15.89:0/2075840579 conn(0x7f64180fc0c0 msgr2=0x7f64180fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.251+0000 7f641dd9d700 1 -- 172.21.15.89:0/2075840579 shutdown_connections 2023-10-26T06:48:29.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.251+0000 7f641dd9d700 1 -- 172.21.15.89:0/2075840579 wait complete. 2023-10-26T06:48:29.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.252+0000 7f641dd9d700 1 Processor -- start 2023-10-26T06:48:29.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.252+0000 7f641dd9d700 1 -- start start 2023-10-26T06:48:29.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641dd9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 0x7f641819d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641dd9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64180734b0 0x7f641819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641dd9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64181115c0 0x7f64181a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641dd9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64181a2040 con 0x7f64181115c0 2023-10-26T06:48:29.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641dd9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64181a21b0 con 0x7f6418072b90 2023-10-26T06:48:29.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641dd9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64181a2370 con 0x7f64180734b0 2023-10-26T06:48:29.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641cd9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 0x7f641819d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641cd9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 0x7f641819d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:35424/0 (socket says 172.21.15.89:35424) 2023-10-26T06:48:29.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.253+0000 7f641cd9b700 1 -- 172.21.15.89:0/2071819432 learned_addr learned my addr 172.21.15.89:0/2071819432 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:29.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.254+0000 7f6417fff700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64180734b0 0x7f641819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.254+0000 7f641d59c700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64181115c0 0x7f64181a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.254+0000 7f641d59c700 1 -- 172.21.15.89:0/2071819432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64180734b0 msgr2=0x7f641819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.254+0000 7f641d59c700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64180734b0 0x7f641819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.254+0000 7f641d59c700 1 -- 172.21.15.89:0/2071819432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 msgr2=0x7f641819d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.254+0000 7f641d59c700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 0x7f641819d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.255+0000 7f641d59c700 1 -- 172.21.15.89:0/2071819432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0095f0 con 0x7f64181115c0 2023-10-26T06:48:29.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.255+0000 7f6417fff700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64180734b0 0x7f641819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:48:29.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.255+0000 7f641d59c700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64181115c0 0x7f64181a1a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f641000bdf0 tx=0x7f6410009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:29.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.256+0000 7f6415ffb700 1 -- 172.21.15.89:0/2071819432 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641000cb30 con 0x7f64181115c0 2023-10-26T06:48:29.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.256+0000 7f6415ffb700 1 -- 172.21.15.89:0/2071819432 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641000cc90 con 0x7f64181115c0 2023-10-26T06:48:29.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.256+0000 7f6415ffb700 1 -- 172.21.15.89:0/2071819432 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64100126b0 con 0x7f64181115c0 2023-10-26T06:48:29.260 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:29 smithi089 ceph-mon[95031]: pgmap v357: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 155 MiB/s wr, 106 op/s 2023-10-26T06:48:29.261 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:29 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4077228881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:48:29.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.266+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64181a2650 con 0x7f64181115c0 2023-10-26T06:48:29.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.266+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64181a9e90 con 0x7f64181115c0 2023-10-26T06:48:29.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.273+0000 7f64328a8b80 1 Processor -- start 2023-10-26T06:48:29.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.274+0000 7f6415ffb700 1 -- 172.21.15.89:0/2071819432 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6410014440 con 0x7f64181115c0 2023-10-26T06:48:29.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.274+0000 7f64328a8b80 1 -- start start 2023-10-26T06:48:29.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e9036140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9036680 0x5596e90302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e9030c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f64328a8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e8fa36e0 con 0x5596e9030820 2023-10-26T06:48:29.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f64328a8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e8f89c60 con 0x5596e9036680 2023-10-26T06:48:29.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f64328a8b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e8f87580 con 0x5596e9035d60 2023-10-26T06:48:29.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f6417fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e9036140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f6417fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e9036140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33412/0 (socket says 172.21.15.89:33412) 2023-10-26T06:48:29.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f6417fff700 1 -- 172.21.15.89:0/1378404324 learned_addr learned my addr 172.21.15.89:0/1378404324 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:29.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.275+0000 7f6417fff700 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e9030c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.276+0000 7f6417fff700 1 -- 172.21.15.89:0/1378404324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9036680 msgr2=0x5596e90302e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:48:29.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.276+0000 7f6417fff700 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9036680 0x5596e90302e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.276+0000 7f6417fff700 1 -- 172.21.15.89:0/1378404324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 msgr2=0x5596e9030c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:29.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.276+0000 7f6417fff700 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e9030c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.276+0000 7f6417fff700 1 -- 172.21.15.89:0/1378404324 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596e8f403f0 con 0x5596e9035d60 2023-10-26T06:48:29.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.277+0000 7f6417fff700 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e9036140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f64040118a0 tx=0x7f6404009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91c17d7aa52655f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:29.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.277+0000 7f6415ffb700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64080a16f0 0x7f64080a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.277+0000 7f6415ffb700 1 -- 172.21.15.89:0/2071819432 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f6410006e40 con 0x7f64181115c0 2023-10-26T06:48:29.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.277+0000 7f6402ffd700 1 -- 172.21.15.89:0/1378404324 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400f700 con 0x5596e9035d60 2023-10-26T06:48:29.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.277+0000 7f6402ffd700 1 -- 172.21.15.89:0/1378404324 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6404017070 con 0x5596e9035d60 2023-10-26T06:48:29.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f6402ffd700 1 -- 172.21.15.89:0/1378404324 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640402ec80 con 0x5596e9035d60 2023-10-26T06:48:29.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f64328a8b80 1 -- 172.21.15.89:0/1378404324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 msgr2=0x5596e9036140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f64328a8b80 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e9036140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f64040118a0 tx=0x7f6404009c30 comp rx=0 tx=0).stop 2023-10-26T06:48:29.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f641cd9b700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64080a16f0 0x7f64080a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f64328a8b80 1 -- 172.21.15.89:0/1378404324 shutdown_connections 2023-10-26T06:48:29.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f64328a8b80 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e9030c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f64328a8b80 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e9036140 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f64328a8b80 1 --2- 172.21.15.89:0/1378404324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9036680 0x5596e90302e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.278+0000 7f64328a8b80 1 -- 172.21.15.89:0/1378404324 >> 172.21.15.89:0/1378404324 conn(0x5596e8fa4720 msgr2=0x5596e902acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:29.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.279+0000 7f64328a8b80 1 -- 172.21.15.89:0/1378404324 shutdown_connections 2023-10-26T06:48:29.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.279+0000 7f64328a8b80 1 -- 172.21.15.89:0/1378404324 wait complete. 2023-10-26T06:48:29.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.279+0000 7f64328a8b80 1 -- start start 2023-10-26T06:48:29.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.280+0000 7f64328a8b80 1 Processor -- start 2023-10-26T06:48:29.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.280+0000 7f641cd9b700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64080a16f0 0x7f64080a3bb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5596e9036ae0 tx=0x7f640c00b280 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:29.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.281+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e903c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.281+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e9039920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.281+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9036680 0x5596e9039fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.281+0000 7f64328a8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e8f82e70 con 0x5596e9030820 2023-10-26T06:48:29.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.281+0000 7f64328a8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e8f1f710 con 0x5596e9035d60 2023-10-26T06:48:29.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.281+0000 7f64328a8b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e8f1f3b0 con 0x5596e9036680 2023-10-26T06:48:29.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f64328a8b80 10 client.0 _ll_register_callbacks cb 0x5596e8e69a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:29.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e9039920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e9039920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:35440/0 (socket says 172.21.15.89:35440) 2023-10-26T06:48:29.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 -- 172.21.15.89:0/3426184859 learned_addr learned my addr 172.21.15.89:0/3426184859 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:29.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f641cd9b700 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e903c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 -- 172.21.15.89:0/3426184859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9036680 msgr2=0x5596e9039fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:48:29.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9036680 0x5596e9039fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 -- 172.21.15.89:0/3426184859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 msgr2=0x5596e903c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:29.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e903c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.282+0000 7f6417fff700 1 -- 172.21.15.89:0/3426184859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6404019040 con 0x5596e9035d60 2023-10-26T06:48:29.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.283+0000 7f641cd9b700 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e903c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:48:29.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.283+0000 7f6417fff700 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e9039920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f640402e800 tx=0x7f640402e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce5781186c87882 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:29.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.284+0000 7f63f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:48:29.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.285+0000 7f63f7fff700 1 -- 172.21.15.89:0/3426184859 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640402e9c0 con 0x5596e9035d60 2023-10-26T06:48:29.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.285+0000 7f63f7fff700 1 -- 172.21.15.89:0/3426184859 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f640402eb20 con 0x5596e9035d60 2023-10-26T06:48:29.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.285+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596e8f87ef0 con 0x5596e9035d60 2023-10-26T06:48:29.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.285+0000 7f63f7fff700 1 -- 172.21.15.89:0/3426184859 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f640402ece0 con 0x5596e9035d60 2023-10-26T06:48:29.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.285+0000 7f64328a8b80 10 client.25625 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:29.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.285+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5596e8dadde0 con 0x5596e9035d60 2023-10-26T06:48:29.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f63f7fff700 1 -- 172.21.15.89:0/3426184859 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596e8f87ef0 con 0x5596e9035d60 2023-10-26T06:48:29.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f63f7fff700 1 -- 172.21.15.89:0/3426184859 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6404036070 con 0x5596e9035d60 2023-10-26T06:48:29.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f64328a8b80 10 client.25625 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:29.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f64328a8b80 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:29.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f64328a8b80 20 client.25625 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:29.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f64328a8b80 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e904a1d0 0x5596e904c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f64328a8b80 4 client.25625 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:48:29.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.286+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596e8fff8b0 con 0x5596e904a1d0 2023-10-26T06:48:29.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.289+0000 7f641cd9b700 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e904a1d0 0x5596e904c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.293+0000 7f641cd9b700 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e904a1d0 0x5596e904c5f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec42e2ad85581322 server_cookie=6d9edb457a21ab47 in_seq=0 out_seq=0 2023-10-26T06:48:29.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.293+0000 7f63f7fff700 10 client.25625 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:29.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.294+0000 7f63f7fff700 1 -- 172.21.15.89:0/3426184859 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596e8fff8b0 con 0x5596e904a1d0 2023-10-26T06:48:29.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.294+0000 7f63f7fff700 10 client.25625 handle_command_reply: tid=0 2023-10-26T06:48:29.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.294+0000 7f64328a8b80 1 client.25625 shutdown 2023-10-26T06:48:29.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 msgr2=0x5596e9039920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e9039920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f640402e800 tx=0x7f640402e6d0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 shutdown_connections 2023-10-26T06:48:29.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9030820 0x5596e903c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e904a1d0 0x5596e904c5f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9036680 0x5596e9039fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 --2- 172.21.15.89:0/3426184859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e9039920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f63f7fff700 0 client.25625 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:29.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 >> 172.21.15.89:0/3426184859 conn(0x5596e8fa4720 msgr2=0x5596e902f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:29.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 shutdown_connections 2023-10-26T06:48:29.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.304+0000 7f64328a8b80 1 -- 172.21.15.89:0/3426184859 wait complete. 2023-10-26T06:48:29.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.305+0000 7f64328a8b80 20 client.25625 trim_cache size 0 max 16384 2023-10-26T06:48:29.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.343+0000 7f64328a8b80 1 Processor -- start 2023-10-26T06:48:29.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.344+0000 7f64328a8b80 1 -- start start 2023-10-26T06:48:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.345+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9030820 0x5596e9041150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.345+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e90381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.345+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e9038730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.345+0000 7f64328a8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e8e03920 con 0x5596e9036680 2023-10-26T06:48:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.345+0000 7f64328a8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e9041690 con 0x5596e9030820 2023-10-26T06:48:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.345+0000 7f64328a8b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e90273f0 con 0x5596e9035d60 2023-10-26T06:48:29.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.346+0000 7f641d59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e9038730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.346+0000 7f641d59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e9038730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54940/0 (socket says 172.21.15.89:54940) 2023-10-26T06:48:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.346+0000 7f641d59c700 1 -- 172.21.15.89:0/4284664183 learned_addr learned my addr 172.21.15.89:0/4284664183 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.347+0000 7f641d59c700 1 -- 172.21.15.89:0/4284664183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 msgr2=0x5596e90381f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:48:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.347+0000 7f641d59c700 1 --2- 172.21.15.89:0/4284664183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e90381f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.347+0000 7f641d59c700 1 -- 172.21.15.89:0/4284664183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9030820 msgr2=0x5596e9041150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:48:29.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.347+0000 7f641d59c700 1 --2- 172.21.15.89:0/4284664183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9030820 0x5596e9041150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.347+0000 7f641d59c700 1 -- 172.21.15.89:0/4284664183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596e8d96c60 con 0x5596e9036680 2023-10-26T06:48:29.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.348+0000 7f641d59c700 1 --2- 172.21.15.89:0/4284664183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e9038730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6410095f70 tx=0x7f64100993d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d073126123c4ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:29.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.348+0000 7f64037fe700 1 -- 172.21.15.89:0/4284664183 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410099a70 con 0x5596e9036680 2023-10-26T06:48:29.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.349+0000 7f64037fe700 1 -- 172.21.15.89:0/4284664183 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641009dbc0 con 0x5596e9036680 2023-10-26T06:48:29.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.349+0000 7f64037fe700 1 -- 172.21.15.89:0/4284664183 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410099a70 con 0x5596e9036680 2023-10-26T06:48:29.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.350+0000 7f64328a8b80 1 -- 172.21.15.89:0/4284664183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 msgr2=0x5596e9038730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.350+0000 7f64328a8b80 1 --2- 172.21.15.89:0/4284664183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e9038730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6410095f70 tx=0x7f64100993d0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.350+0000 7f64328a8b80 1 -- 172.21.15.89:0/4284664183 shutdown_connections 2023-10-26T06:48:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.350+0000 7f64328a8b80 1 --2- 172.21.15.89:0/4284664183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e9038730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.351+0000 7f64328a8b80 1 --2- 172.21.15.89:0/4284664183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9030820 0x5596e9041150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.351+0000 7f64328a8b80 1 --2- 172.21.15.89:0/4284664183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9035d60 0x5596e90381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.351+0000 7f64328a8b80 1 -- 172.21.15.89:0/4284664183 >> 172.21.15.89:0/4284664183 conn(0x5596e8fa4720 msgr2=0x5596e902c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.351+0000 7f64328a8b80 1 -- 172.21.15.89:0/4284664183 shutdown_connections 2023-10-26T06:48:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.351+0000 7f64328a8b80 1 -- 172.21.15.89:0/4284664183 wait complete. 2023-10-26T06:48:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.352+0000 7f64328a8b80 1 -- start start 2023-10-26T06:48:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.352+0000 7f64328a8b80 1 Processor -- start 2023-10-26T06:48:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.353+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9030820 0x5596e9040e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e903b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f64328a8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e903b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f641cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9030820 0x5596e9040e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f641cd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9030820 0x5596e9040e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33428/0 (socket says 172.21.15.89:33428) 2023-10-26T06:48:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e903bd00 con 0x5596e9036680 2023-10-26T06:48:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f641cd9b700 1 -- 172.21.15.89:0/866614137 learned_addr learned my addr 172.21.15.89:0/866614137 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e903be70 con 0x5596e9035d60 2023-10-26T06:48:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e903c030 con 0x5596e9030820 2023-10-26T06:48:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.354+0000 7f64328a8b80 10 client.0 _ll_register_callbacks cb 0x5596e8e69a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:29.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.355+0000 7f641d59c700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e903b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.355+0000 7f6417fff700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e903b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.355+0000 7f641d59c700 1 -- 172.21.15.89:0/866614137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9030820 msgr2=0x5596e9040e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:29.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.355+0000 7f641d59c700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9030820 0x5596e9040e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.355+0000 7f641d59c700 1 -- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 msgr2=0x5596e903b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:29.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.355+0000 7f641d59c700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e903b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.355+0000 7f641d59c700 1 -- 172.21.15.89:0/866614137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64181a9e90 con 0x5596e9036680 2023-10-26T06:48:29.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.356+0000 7f641cd9b700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9030820 0x5596e9040e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:48:29.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.356+0000 7f641d59c700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e903b700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5596e903aed0 tx=0x7f64100ca560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13efdd0d81ac8f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:29.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.356+0000 7f63f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:48:29.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.356+0000 7f63f7fff700 1 -- 172.21.15.89:0/866614137 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410099ed0 con 0x5596e9036680 2023-10-26T06:48:29.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.356+0000 7f63f7fff700 1 -- 172.21.15.89:0/866614137 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f6410099a70 con 0x5596e9036680 2023-10-26T06:48:29.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.356+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6404017070 con 0x5596e9036680 2023-10-26T06:48:29.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.357+0000 7f63f7fff700 1 -- 172.21.15.89:0/866614137 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6410098f60 con 0x5596e9036680 2023-10-26T06:48:29.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.357+0000 7f64328a8b80 10 client.15633 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:29.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.357+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5596e8dd4250 con 0x5596e9036680 2023-10-26T06:48:29.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.357+0000 7f63f7fff700 1 -- 172.21.15.89:0/866614137 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6404017070 con 0x5596e9036680 2023-10-26T06:48:29.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.358+0000 7f63f7fff700 1 -- 172.21.15.89:0/866614137 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6410095860 con 0x5596e9036680 2023-10-26T06:48:29.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.358+0000 7f64328a8b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:29.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.358+0000 7f64328a8b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:29.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.358+0000 7f64328a8b80 20 client.15633 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:29.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.358+0000 7f64328a8b80 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e909f2c0 0x5596e90a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:29.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.358+0000 7f64328a8b80 4 client.15633 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:48:29.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.358+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596e8f7be50 con 0x5596e909f2c0 2023-10-26T06:48:29.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.360+0000 7f641cd9b700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e909f2c0 0x5596e90a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:29.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.365+0000 7f63f7fff700 10 client.15633 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:29.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.365+0000 7f641cd9b700 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e909f2c0 0x5596e90a16e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957c9e687b17a896 server_cookie=7a48b1a00be9fded in_seq=0 out_seq=0 2023-10-26T06:48:29.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.372+0000 7f63f7fff700 1 -- 172.21.15.89:0/866614137 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596e8f7be50 con 0x5596e909f2c0 2023-10-26T06:48:29.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.372+0000 7f63f7fff700 10 client.15633 handle_command_reply: tid=0 2023-10-26T06:48:29.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.374+0000 7f64328a8b80 1 client.15633 shutdown 2023-10-26T06:48:29.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 msgr2=0x5596e903b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e903b700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x5596e903aed0 tx=0x7f64100ca560 comp rx=0 tx=0).stop 2023-10-26T06:48:29.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 shutdown_connections 2023-10-26T06:48:29.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5596e9036680 0x5596e903b700 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5596e909f2c0 0x5596e90a16e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596e9030820 0x5596e9040e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 --2- 172.21.15.89:0/866614137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e9035d60 0x5596e903b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 >> 172.21.15.89:0/866614137 conn(0x5596e8fa4720 msgr2=0x5596e9038130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:29.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.375+0000 7f63f7fff700 0 client.15633 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:29.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.376+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 shutdown_connections 2023-10-26T06:48:29.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.376+0000 7f64328a8b80 1 -- 172.21.15.89:0/866614137 wait complete. 2023-10-26T06:48:29.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.377+0000 7f64328a8b80 20 client.15633 trim_cache size 0 max 16384 2023-10-26T06:48:29.380 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:48:29.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64080a16f0 msgr2=0x7f64080a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64080a16f0 0x7f64080a3bb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5596e9036ae0 tx=0x7f640c00b280 comp rx=0 tx=0).stop 2023-10-26T06:48:29.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64181115c0 msgr2=0x7f64181a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:29.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64181115c0 0x7f64181a1a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f641000bdf0 tx=0x7f6410009ed0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 shutdown_connections 2023-10-26T06:48:29.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64080a16f0 0x7f64080a3bb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64181115c0 0x7f64181a1a00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6418072b90 0x7f641819d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 --2- 172.21.15.89:0/2071819432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64180734b0 0x7f641819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:29.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.382+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 >> 172.21.15.89:0/2071819432 conn(0x7f64180fc0c0 msgr2=0x7f64180fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:29.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.383+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 shutdown_connections 2023-10-26T06:48:29.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:29.383+0000 7f641dd9d700 1 -- 172.21.15.89:0/2071819432 wait complete. 2023-10-26T06:48:29.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:29 smithi192 ceph-mon[131661]: pgmap v357: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 155 MiB/s wr, 106 op/s 2023-10-26T06:48:29.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:29 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4077228881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:48:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:29 smithi135 ceph-mon[131403]: pgmap v357: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 155 MiB/s wr, 106 op/s 2023-10-26T06:48:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:29 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4077228881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:48:30.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:48:30.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:48:30.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 -- 172.21.15.89:0/1473858190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef900ff4e0 msgr2=0x7fef900ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 --2- 172.21.15.89:0/1473858190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef900ff4e0 0x7fef900ff960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fef800099c0 tx=0x7fef80009cd0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 -- 172.21.15.89:0/1473858190 shutdown_connections 2023-10-26T06:48:30.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 --2- 172.21.15.89:0/1473858190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ffea0 0x7fef9010c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 --2- 172.21.15.89:0/1473858190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef901055a0 0x7fef90105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 --2- 172.21.15.89:0/1473858190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef900ff4e0 0x7fef900ff960 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 -- 172.21.15.89:0/1473858190 >> 172.21.15.89:0/1473858190 conn(0x7fef900fafb0 msgr2=0x7fef900fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 -- 172.21.15.89:0/1473858190 shutdown_connections 2023-10-26T06:48:30.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.633+0000 7fef9679d700 1 -- 172.21.15.89:0/1473858190 wait complete. 2023-10-26T06:48:30.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.634+0000 7fef9679d700 1 Processor -- start 2023-10-26T06:48:30.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.634+0000 7fef9679d700 1 -- start start 2023-10-26T06:48:30.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9679d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ff4e0 0x7fef9010cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9679d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef900ffea0 0x7fef9010d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9679d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef901055a0 0x7fef9010d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9679d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901a4e10 con 0x7fef901055a0 2023-10-26T06:48:30.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9679d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901a4f80 con 0x7fef900ffea0 2023-10-26T06:48:30.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9679d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901a5140 con 0x7fef900ff4e0 2023-10-26T06:48:30.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ff4e0 0x7fef9010cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ff4e0 0x7fef9010cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:56862/0 (socket says 172.21.15.89:56862) 2023-10-26T06:48:30.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 -- 172.21.15.89:0/2080235806 learned_addr learned my addr 172.21.15.89:0/2080235806 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef94f9a700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef900ffea0 0x7fef9010d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef95f9c700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef901055a0 0x7fef9010d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 -- 172.21.15.89:0/2080235806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef900ffea0 msgr2=0x7fef9010d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef900ffea0 0x7fef9010d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 -- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef901055a0 msgr2=0x7fef9010d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef901055a0 0x7fef9010d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 -- 172.21.15.89:0/2080235806 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800095f0 con 0x7fef900ff4e0 2023-10-26T06:48:30.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.635+0000 7fef9579b700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ff4e0 0x7fef9010cd30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9a0 tx=0x7fef8c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.636+0000 7fef867fc700 1 -- 172.21.15.89:0/2080235806 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c009790 con 0x7fef900ff4e0 2023-10-26T06:48:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.636+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef901a5420 con 0x7fef900ff4e0 2023-10-26T06:48:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.636+0000 7fef867fc700 1 -- 172.21.15.89:0/2080235806 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef8c00cbd0 con 0x7fef900ff4e0 2023-10-26T06:48:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.636+0000 7fef867fc700 1 -- 172.21.15.89:0/2080235806 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c018680 con 0x7fef900ff4e0 2023-10-26T06:48:30.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.636+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef901a5900 con 0x7fef900ff4e0 2023-10-26T06:48:30.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.639+0000 7fef867fc700 1 -- 172.21.15.89:0/2080235806 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fef8c00cd40 con 0x7fef900ff4e0 2023-10-26T06:48:30.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.640+0000 7fef867fc700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fef7c0a15f0 0x7fef7c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.640+0000 7fefab381b80 1 Processor -- start 2023-10-26T06:48:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.640+0000 7fef867fc700 1 -- 172.21.15.89:0/2080235806 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fef8c014070 con 0x7fef900ff4e0 2023-10-26T06:48:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.640+0000 7fefab381b80 1 -- start start 2023-10-26T06:48:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef94f9a700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fef7c0a15f0 0x7fef7c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fefab381b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fefab381b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b8c700 0x55d861b86360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fefab381b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b868a0 0x55d861b86d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fefab381b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861af9760 con 0x55d861b868a0 2023-10-26T06:48:30.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fefab381b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861adfce0 con 0x55d861b8c700 2023-10-26T06:48:30.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fefab381b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861add600 con 0x55d861b8bde0 2023-10-26T06:48:30.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef95f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b868a0 0x55d861b86d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef9579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef95f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b868a0 0x55d861b86d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45652/0 (socket says 172.21.15.89:45652) 2023-10-26T06:48:30.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef95f9c700 1 -- 172.21.15.89:0/3648260846 learned_addr learned my addr 172.21.15.89:0/3648260846 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef94f9a700 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b8c700 0x55d861b86360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef94f9a700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fef7c0a15f0 0x7fef7c0a3ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef80009fd0 tx=0x7fef80019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef9579b700 1 -- 172.21.15.89:0/3648260846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b8c700 msgr2=0x55d861b86360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef9579b700 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b8c700 0x55d861b86360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef9579b700 1 -- 172.21.15.89:0/3648260846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b868a0 msgr2=0x55d861b86d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef9579b700 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b868a0 0x55d861b86d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.641+0000 7fef9579b700 1 -- 172.21.15.89:0/3648260846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d861a96470 con 0x55d861b8bde0 2023-10-26T06:48:30.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.642+0000 7fef9579b700 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8c1c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0caa70 tx=0x7fef8c09a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b12eefdc3c5cf51 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.642+0000 7fef7b7fe700 1 -- 172.21.15.89:0/3648260846 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c09a300 con 0x55d861b8bde0 2023-10-26T06:48:30.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.642+0000 7fef7b7fe700 1 -- 172.21.15.89:0/3648260846 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef8c09e450 con 0x55d861b8bde0 2023-10-26T06:48:30.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.642+0000 7fef7b7fe700 1 -- 172.21.15.89:0/3648260846 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c09a7c0 con 0x55d861b8bde0 2023-10-26T06:48:30.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 -- 172.21.15.89:0/3648260846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 msgr2=0x55d861b8c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8c1c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0caa70 tx=0x7fef8c09a200 comp rx=0 tx=0).stop 2023-10-26T06:48:30.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 -- 172.21.15.89:0/3648260846 shutdown_connections 2023-10-26T06:48:30.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b868a0 0x55d861b86d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8c1c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 --2- 172.21.15.89:0/3648260846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b8c700 0x55d861b86360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 -- 172.21.15.89:0/3648260846 >> 172.21.15.89:0/3648260846 conn(0x55d861afa7a0 msgr2=0x55d861b80d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 -- 172.21.15.89:0/3648260846 shutdown_connections 2023-10-26T06:48:30.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 -- 172.21.15.89:0/3648260846 wait complete. 2023-10-26T06:48:30.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.643+0000 7fefab381b80 1 -- start start 2023-10-26T06:48:30.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 1 Processor -- start 2023-10-26T06:48:30.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b92720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8c700 0x55d861b901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861ad8ef0 con 0x55d861b8c700 2023-10-26T06:48:30.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861a75790 con 0x55d861b868a0 2023-10-26T06:48:30.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861a75430 con 0x55d861b8bde0 2023-10-26T06:48:30.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.644+0000 7fefab381b80 10 client.0 _ll_register_callbacks cb 0x55d8619bfb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:30.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b92720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef95f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8c700 0x55d861b901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b92720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:33394/0 (socket says 172.21.15.89:33394) 2023-10-26T06:48:30.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 -- 172.21.15.89:0/1560229630 learned_addr learned my addr 172.21.15.89:0/1560229630 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef94f9a700 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 -- 172.21.15.89:0/1560229630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 msgr2=0x55d861b8fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 -- 172.21.15.89:0/1560229630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8c700 msgr2=0x55d861b901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8c700 0x55d861b901a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 -- 172.21.15.89:0/1560229630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef901a5900 con 0x55d861b868a0 2023-10-26T06:48:30.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef9579b700 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b92720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0caff0 tx=0x7fef8c010b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91392b56bbb66ca8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:48:30.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.645+0000 7fef78ff9700 1 -- 172.21.15.89:0/1560229630 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c09a380 con 0x55d861b868a0 2023-10-26T06:48:30.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d861addf70 con 0x55d861b868a0 2023-10-26T06:48:30.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fef78ff9700 1 -- 172.21.15.89:0/1560229630 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fef8c09a560 con 0x55d861b868a0 2023-10-26T06:48:30.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fef78ff9700 1 -- 172.21.15.89:0/1560229630 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8c09a720 con 0x55d861b868a0 2023-10-26T06:48:30.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 10 client.25640 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:30.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d861903e60 con 0x55d861b868a0 2023-10-26T06:48:30.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fef78ff9700 1 -- 172.21.15.89:0/1560229630 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d861addf70 con 0x55d861b868a0 2023-10-26T06:48:30.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fef78ff9700 1 -- 172.21.15.89:0/1560229630 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef8c07c050 con 0x55d861b868a0 2023-10-26T06:48:30.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:30.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:30.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 20 client.25640 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:30.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861b9fd50 0x55d861ba2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 4 client.25640 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:48:30.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.646+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d861b55930 con 0x55d861b9fd50 2023-10-26T06:48:30.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.647+0000 7fef94f9a700 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861b9fd50 0x55d861ba2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.647+0000 7fef94f9a700 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861b9fd50 0x55d861ba2170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31374aff055ce07e server_cookie=6873f0798595704 in_seq=0 out_seq=0 2023-10-26T06:48:30.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.647+0000 7fef78ff9700 10 client.25640 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fef78ff9700 1 -- 172.21.15.89:0/1560229630 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d861b55930 con 0x55d861b9fd50 2023-10-26T06:48:30.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fef78ff9700 10 client.25640 handle_command_reply: tid=0 2023-10-26T06:48:30.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 client.25640 shutdown 2023-10-26T06:48:30.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 msgr2=0x55d861b92720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b92720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0caff0 tx=0x7fef8c010b70 comp rx=0 tx=0).stop 2023-10-26T06:48:30.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 shutdown_connections 2023-10-26T06:48:30.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8c700 0x55d861b901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861b9fd50 0x55d861ba2170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b92720 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fef78ff9700 0 client.25640 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 --2- 172.21.15.89:0/1560229630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8bde0 0x55d861b8fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 >> 172.21.15.89:0/1560229630 conn(0x55d861afa7a0 msgr2=0x55d861b85350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 shutdown_connections 2023-10-26T06:48:30.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 1 -- 172.21.15.89:0/1560229630 wait complete. 2023-10-26T06:48:30.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.648+0000 7fefab381b80 20 client.25640 trim_cache size 0 max 16384 2023-10-26T06:48:30.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.665+0000 7fefab381b80 1 Processor -- start 2023-10-26T06:48:30.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fefab381b80 1 -- start start 2023-10-26T06:48:30.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fefab381b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fefab381b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861b898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fefab381b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b89e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fefab381b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8619599a0 con 0x55d861b8bde0 2023-10-26T06:48:30.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fefab381b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861b792a0 con 0x55d861b868a0 2023-10-26T06:48:30.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fefab381b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861b8daf0 con 0x55d861b8c700 2023-10-26T06:48:30.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fef95f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b89e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fef9579b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fef95f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b89e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:56874/0 (socket says 172.21.15.89:56874) 2023-10-26T06:48:30.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fef95f9c700 1 -- 172.21.15.89:0/1494461722 learned_addr learned my addr 172.21.15.89:0/1494461722 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.666+0000 7fef94f9a700 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861b898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef95f9c700 1 -- 172.21.15.89:0/1494461722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 msgr2=0x55d861b970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef95f9c700 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef95f9c700 1 -- 172.21.15.89:0/1494461722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 msgr2=0x55d861b898e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef95f9c700 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861b898e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef95f9c700 1 -- 172.21.15.89:0/1494461722 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8618ecce0 con 0x55d861b8c700 2023-10-26T06:48:30.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef94f9a700 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861b898e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:48:30.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef9579b700 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:48:30.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef95f9c700 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b89e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef88009520 tx=0x7fef8800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4f29d2e74b7ad8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef7bfff700 1 -- 172.21.15.89:0/1494461722 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800e040 con 0x55d861b8c700 2023-10-26T06:48:30.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef7bfff700 1 -- 172.21.15.89:0/1494461722 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef8800f040 con 0x55d861b8c700 2023-10-26T06:48:30.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.667+0000 7fef7bfff700 1 -- 172.21.15.89:0/1494461722 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88017410 con 0x55d861b8c700 2023-10-26T06:48:30.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 -- 172.21.15.89:0/1494461722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 msgr2=0x55d861b89e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b89e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef88009520 tx=0x7fef8800b040 comp rx=0 tx=0).stop 2023-10-26T06:48:30.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 -- 172.21.15.89:0/1494461722 shutdown_connections 2023-10-26T06:48:30.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b89e20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861b970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 --2- 172.21.15.89:0/1494461722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861b898e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 -- 172.21.15.89:0/1494461722 >> 172.21.15.89:0/1494461722 conn(0x55d861afa7a0 msgr2=0x55d861b8e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 -- 172.21.15.89:0/1494461722 shutdown_connections 2023-10-26T06:48:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 -- 172.21.15.89:0/1494461722 wait complete. 2023-10-26T06:48:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.668+0000 7fefab381b80 1 -- start start 2023-10-26T06:48:30.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 1 Processor -- start 2023-10-26T06:48:30.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861bed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861bed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b96390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861b96960 con 0x55d861b8bde0 2023-10-26T06:48:30.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861b96ad0 con 0x55d861b868a0 2023-10-26T06:48:30.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d861b96c90 con 0x55d861b8c700 2023-10-26T06:48:30.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fefab381b80 10 client.0 _ll_register_callbacks cb 0x7fef88009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:30.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fef9579b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861bed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fef9579b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861bed070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:33410/0 (socket says 172.21.15.89:33410) 2023-10-26T06:48:30.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fef94f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861bed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.669+0000 7fef9579b700 1 -- 172.21.15.89:0/1106825962 learned_addr learned my addr 172.21.15.89:0/1106825962 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef95f9c700 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b96390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef9579b700 1 -- 172.21.15.89:0/1106825962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 msgr2=0x55d861b96390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef9579b700 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b96390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef9579b700 1 -- 172.21.15.89:0/1106825962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 msgr2=0x55d861bed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef9579b700 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861bed5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef9579b700 1 -- 172.21.15.89:0/1106825962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d861903e60 con 0x55d861b868a0 2023-10-26T06:48:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef95f9c700 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b96390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:48:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef9579b700 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861bed070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef8c09a290 tx=0x7fef8c096760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f179c7711e46437e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:48:30.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef78ff9700 1 -- 172.21.15.89:0/1106825962 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c099c60 con 0x55d861b868a0 2023-10-26T06:48:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef78ff9700 1 -- 172.21.15.89:0/1106825962 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fef8c09a300 con 0x55d861b868a0 2023-10-26T06:48:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef8c09e450 con 0x55d861b868a0 2023-10-26T06:48:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fef78ff9700 1 -- 172.21.15.89:0/1106825962 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8c0a2200 con 0x55d861b868a0 2023-10-26T06:48:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fefab381b80 10 client.25646 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.670+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fef880091d0 con 0x55d861b868a0 2023-10-26T06:48:30.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fef78ff9700 1 -- 172.21.15.89:0/1106825962 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef8c0a6080 con 0x55d861b868a0 2023-10-26T06:48:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fef78ff9700 1 -- 172.21.15.89:0/1106825962 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef8c07c020 con 0x55d861b868a0 2023-10-26T06:48:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fefab381b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fefab381b80 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fefab381b80 20 client.25646 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:30.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fefab381b80 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861bf5c80 0x55d861bf8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fefab381b80 4 client.25646 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:48:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d861ad1ed0 con 0x55d861bf5c80 2023-10-26T06:48:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.671+0000 7fef94f9a700 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861bf5c80 0x55d861bf8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.672+0000 7fef78ff9700 10 client.25646 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.672+0000 7fef94f9a700 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861bf5c80 0x55d861bf8070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee8a12039b60c81 server_cookie=32bf1cacf6eb885c in_seq=0 out_seq=0 2023-10-26T06:48:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.672+0000 7fef78ff9700 1 -- 172.21.15.89:0/1106825962 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d861ad1ed0 con 0x55d861bf5c80 2023-10-26T06:48:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.672+0000 7fef78ff9700 10 client.25646 handle_command_reply: tid=0 2023-10-26T06:48:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.672+0000 7fefab381b80 1 client.25646 shutdown 2023-10-26T06:48:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 msgr2=0x55d861bed070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861bed070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef8c09a290 tx=0x7fef8c096760 comp rx=0 tx=0).stop 2023-10-26T06:48:30.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 shutdown_connections 2023-10-26T06:48:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d861bf5c80 0x55d861bf8070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d861b8c700 0x55d861b96390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d861b868a0 0x55d861bed070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 --2- 172.21.15.89:0/1106825962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d861b8bde0 0x55d861bed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 >> 172.21.15.89:0/1106825962 conn(0x55d861afa7a0 msgr2=0x55d861b898e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fef78ff9700 0 client.25646 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 shutdown_connections 2023-10-26T06:48:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 1 -- 172.21.15.89:0/1106825962 wait complete. 2023-10-26T06:48:30.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.673+0000 7fefab381b80 20 client.25646 trim_cache size 0 max 16384 2023-10-26T06:48:30.691 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:48:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:48:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "ef3564d6-89cc-4366-bdf1-3efcb4245a96", 2023-10-26T06:48:30.691 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:48:30.691 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:48:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fef7c0a15f0 msgr2=0x7fef7c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fef7c0a15f0 0x7fef7c0a3ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef80009fd0 tx=0x7fef80019040 comp rx=0 tx=0).stop 2023-10-26T06:48:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ff4e0 msgr2=0x7fef9010cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ff4e0 0x7fef9010cd30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9a0 tx=0x7fef8c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 shutdown_connections 2023-10-26T06:48:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fef7c0a15f0 0x7fef7c0a3ab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef901055a0 0x7fef9010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef900ff4e0 0x7fef9010cd30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 --2- 172.21.15.89:0/2080235806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef900ffea0 0x7fef9010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 >> 172.21.15.89:0/2080235806 conn(0x7fef900fafb0 msgr2=0x7fef900fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 shutdown_connections 2023-10-26T06:48:30.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.674+0000 7fef9679d700 1 -- 172.21.15.89:0/2080235806 wait complete. 2023-10-26T06:48:30.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:48:30.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 -- 172.21.15.89:0/2415513203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc106720 msgr2=0x7f1bdc106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 --2- 172.21.15.89:0/2415513203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc106720 0x7f1bdc106b00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009990 tx=0x7f1bcc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 -- 172.21.15.89:0/2415513203 shutdown_connections 2023-10-26T06:48:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 --2- 172.21.15.89:0/2415513203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bdc1010d0 0x7f1bdc10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 --2- 172.21.15.89:0/2415513203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc106720 0x7f1bdc106b00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 --2- 172.21.15.89:0/2415513203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bdc100710 0x7f1bdc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 -- 172.21.15.89:0/2415513203 >> 172.21.15.89:0/2415513203 conn(0x7f1bdc0fc200 msgr2=0x7f1bdc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 -- 172.21.15.89:0/2415513203 shutdown_connections 2023-10-26T06:48:30.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.901+0000 7f1be21ff700 1 -- 172.21.15.89:0/2415513203 wait complete. 2023-10-26T06:48:30.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.902+0000 7f1be21ff700 1 Processor -- start 2023-10-26T06:48:30.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.902+0000 7f1be21ff700 1 -- start start 2023-10-26T06:48:30.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1be21ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc100710 0x7f1bdc19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1be21ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bdc1010d0 0x7f1bdc19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1be21ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bdc106720 0x7f1bdc196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1be21ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc197060 con 0x7f1bdc1010d0 2023-10-26T06:48:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1be21ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc1971d0 con 0x7f1bdc100710 2023-10-26T06:48:30.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1be21ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc197390 con 0x7f1bdc106720 2023-10-26T06:48:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc100710 0x7f1bdc19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc100710 0x7f1bdc19c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:33422/0 (socket says 172.21.15.89:33422) 2023-10-26T06:48:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/3190172374 learned_addr learned my addr 172.21.15.89:0/3190172374 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bdc1010d0 0x7f1bdc19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.903+0000 7f1bdbfff700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bdc106720 0x7f1bdc196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/3190172374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bdc106720 msgr2=0x7f1bdc196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bdc106720 0x7f1bdc196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bdc1010d0 msgr2=0x7f1bdc19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bdc1010d0 0x7f1bdc19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/3190172374 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0095f0 con 0x7f1bdc100710 2023-10-26T06:48:30.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bdc1010d0 0x7f1bdc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:48:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc100710 0x7f1bdc19c8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc004730 tx=0x7f1bcc004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bd8ff9700 1 -- 172.21.15.89:0/3190172374 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc01d070 con 0x7f1bdc100710 2023-10-26T06:48:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bd8ff9700 1 -- 172.21.15.89:0/3190172374 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bcc022470 con 0x7f1bdc100710 2023-10-26T06:48:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc197610 con 0x7f1bdc100710 2023-10-26T06:48:30.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1bd8ff9700 1 -- 172.21.15.89:0/3190172374 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc017650 con 0x7f1bdc100710 2023-10-26T06:48:30.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.904+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc1a38c0 con 0x7f1bdc100710 2023-10-26T06:48:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.907+0000 7f1bd8ff9700 1 -- 172.21.15.89:0/3190172374 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1bcc0225e0 con 0x7f1bdc100710 2023-10-26T06:48:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.909+0000 7f1bd8ff9700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1bc40a1670 0x7f1bc40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.909+0000 7f1bd8ff9700 1 -- 172.21.15.89:0/3190172374 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1bcc0ce3b0 con 0x7f1bdc100710 2023-10-26T06:48:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.909+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1bc40a1670 0x7f1bc40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.910+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1bc40a1670 0x7f1bc40a3b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc197bc0 tx=0x7f1bd0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 Processor -- start 2023-10-26T06:48:30.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 -- start start 2023-10-26T06:48:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c538ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c538cfb0 0x5649c5390bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c5391100 0x5649c5391560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c52fb6e0 con 0x5649c538c690 2023-10-26T06:48:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c52e1c60 con 0x5649c538cfb0 2023-10-26T06:48:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.911+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c52df580 con 0x5649c5391100 2023-10-26T06:48:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c538ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c538ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45714/0 (socket says 172.21.15.89:45714) 2023-10-26T06:48:30.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/1770007391 learned_addr learned my addr 172.21.15.89:0/1770007391 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdbfff700 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c5391100 0x5649c5391560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c538cfb0 0x5649c5390bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/1770007391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c5391100 msgr2=0x5649c5391560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c5391100 0x5649c5391560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/1770007391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c538cfb0 msgr2=0x5649c5390bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c538cfb0 0x5649c5390bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/1770007391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649c52983f0 con 0x5649c538c690 2023-10-26T06:48:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.912+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c538ca70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0a31d0 tx=0x7f1bcc09b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e27f74dcb1c1fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.913+0000 7f1bc1ffb700 1 -- 172.21.15.89:0/1770007391 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc09bb00 con 0x5649c538c690 2023-10-26T06:48:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.913+0000 7f1bc1ffb700 1 -- 172.21.15.89:0/1770007391 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bcc09b400 con 0x5649c538c690 2023-10-26T06:48:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.913+0000 7f1bc1ffb700 1 -- 172.21.15.89:0/1770007391 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc09f380 con 0x5649c538c690 2023-10-26T06:48:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1770007391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 msgr2=0x5649c538ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c538ca70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0a31d0 tx=0x7f1bcc09b250 comp rx=0 tx=0).stop 2023-10-26T06:48:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1770007391 shutdown_connections 2023-10-26T06:48:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c538ca70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c5391100 0x5649c5391560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1770007391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c538cfb0 0x5649c5390bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1770007391 >> 172.21.15.89:0/1770007391 conn(0x5649c52fc720 msgr2=0x5649c5382b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1770007391 shutdown_connections 2023-10-26T06:48:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1770007391 wait complete. 2023-10-26T06:48:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.914+0000 7f1bf5a8fb80 1 -- start start 2023-10-26T06:48:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 1 Processor -- start 2023-10-26T06:48:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538c690 0x5649c5398890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538cfb0 0x5649c5395c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c52dae70 con 0x5649c538cfb0 2023-10-26T06:48:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c5277710 con 0x5649c5391100 2023-10-26T06:48:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c52773b0 con 0x5649c538c690 2023-10-26T06:48:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bf5a8fb80 10 client.0 _ll_register_callbacks cb 0x5649c51c1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bdaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538cfb0 0x5649c5395c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538c690 0x5649c5398890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.915+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538cfb0 0x5649c5395c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45726/0 (socket says 172.21.15.89:45726) 2023-10-26T06:48:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdaffd700 1 -- 172.21.15.89:0/3381253517 learned_addr learned my addr 172.21.15.89:0/3381253517 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/3381253517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 msgr2=0x5649c53962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/3381253517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538cfb0 msgr2=0x5649c5395c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538cfb0 0x5649c5395c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/3381253517 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc1a38c0 con 0x5649c538c690 2023-10-26T06:48:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538c690 0x5649c5398890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc097e70 tx=0x7f1bcc09f350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8423ac9ef3a7960c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:48:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bbaffd700 1 -- 172.21.15.89:0/3381253517 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc09bb00 con 0x5649c538c690 2023-10-26T06:48:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bbaffd700 1 -- 172.21.15.89:0/3381253517 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1bcc0a3700 con 0x5649c538c690 2023-10-26T06:48:30.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649c52dfef0 con 0x5649c538c690 2023-10-26T06:48:30.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.916+0000 7f1bbaffd700 1 -- 172.21.15.89:0/3381253517 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bcc09f550 con 0x5649c538c690 2023-10-26T06:48:30.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 10 client.25657 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:30.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5649c5105de0 con 0x5649c538c690 2023-10-26T06:48:30.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bbaffd700 1 -- 172.21.15.89:0/3381253517 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649c52dfef0 con 0x5649c538c690 2023-10-26T06:48:30.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bbaffd700 1 -- 172.21.15.89:0/3381253517 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1bcc0a3300 con 0x5649c538c690 2023-10-26T06:48:30.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 10 client.25657 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:30.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:30.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 20 client.25657 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:30.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53a5a40 0x5649c53a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 4 client.25657 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:48:30.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649c53578b0 con 0x5649c53a5a40 2023-10-26T06:48:30.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.917+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53a5a40 0x5649c53a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.918+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53a5a40 0x5649c53a7e30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf4e99c6856c19f3 server_cookie=d1fe5ebbde04fe3c in_seq=0 out_seq=0 2023-10-26T06:48:30.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.918+0000 7f1bbaffd700 10 client.25657 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.918+0000 7f1bbaffd700 1 -- 172.21.15.89:0/3381253517 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649c53578b0 con 0x5649c53a5a40 2023-10-26T06:48:30.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.918+0000 7f1bbaffd700 10 client.25657 handle_command_reply: tid=0 2023-10-26T06:48:30.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 client.25657 shutdown 2023-10-26T06:48:30.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538c690 msgr2=0x5649c5398890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538c690 0x5649c5398890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc097e70 tx=0x7f1bcc09f350 comp rx=0 tx=0).stop 2023-10-26T06:48:30.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 shutdown_connections 2023-10-26T06:48:30.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53a5a40 0x5649c53a7e30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538c690 0x5649c5398890 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bbaffd700 0 client.25657 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/3381253517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538cfb0 0x5649c5395c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 >> 172.21.15.89:0/3381253517 conn(0x5649c52fc720 msgr2=0x5649c5387080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 shutdown_connections 2023-10-26T06:48:30.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/3381253517 wait complete. 2023-10-26T06:48:30.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.919+0000 7f1bf5a8fb80 20 client.25657 trim_cache size 0 max 16384 2023-10-26T06:48:30.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.937+0000 7f1bf5a8fb80 1 Processor -- start 2023-10-26T06:48:30.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.937+0000 7f1bf5a8fb80 1 -- start start 2023-10-26T06:48:30.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c539d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c53924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c538f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c515b920 con 0x5649c538c690 2023-10-26T06:48:30.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c539d550 con 0x5649c5391100 2023-10-26T06:48:30.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c538d390 con 0x5649c538cfb0 2023-10-26T06:48:30.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c539d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bdaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c53924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c538f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c539d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45732/0 (socket says 172.21.15.89:45732) 2023-10-26T06:48:30.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.938+0000 7f1bdb7fe700 1 -- 172.21.15.89:0/1197006851 learned_addr learned my addr 172.21.15.89:0/1197006851 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bdaffd700 1 -- 172.21.15.89:0/1197006851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 msgr2=0x5649c538f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/1197006851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c538f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bdaffd700 1 -- 172.21.15.89:0/1197006851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 msgr2=0x5649c539d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/1197006851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c539d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bdaffd700 1 -- 172.21.15.89:0/1197006851 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649c50eec60 con 0x5649c538cfb0 2023-10-26T06:48:30.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/1197006851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c53924f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5649c53a86d0 tx=0x7f1bd0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d0474e5ab1ce27f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bc27fc700 1 -- 172.21.15.89:0/1197006851 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0027040 con 0x5649c538cfb0 2023-10-26T06:48:30.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bc27fc700 1 -- 172.21.15.89:0/1197006851 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd00292c0 con 0x5649c538cfb0 2023-10-26T06:48:30.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.939+0000 7f1bc27fc700 1 -- 172.21.15.89:0/1197006851 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0029b10 con 0x5649c538cfb0 2023-10-26T06:48:30.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1197006851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 msgr2=0x5649c53924f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1197006851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c53924f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5649c53a86d0 tx=0x7f1bd0017000 comp rx=0 tx=0).stop 2023-10-26T06:48:30.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1197006851 shutdown_connections 2023-10-26T06:48:30.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1197006851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c539d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1197006851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c538f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/1197006851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c53924f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1197006851 >> 172.21.15.89:0/1197006851 conn(0x5649c52fc720 msgr2=0x5649c5382810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1197006851 shutdown_connections 2023-10-26T06:48:30.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/1197006851 wait complete. 2023-10-26T06:48:30.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.940+0000 7f1bf5a8fb80 1 -- start start 2023-10-26T06:48:30.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 1 Processor -- start 2023-10-26T06:48:30.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c5399e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c539a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c53f3d80 con 0x5649c538c690 2023-10-26T06:48:30.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c53f3ef0 con 0x5649c5391100 2023-10-26T06:48:30.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649c53f40b0 con 0x5649c538cfb0 2023-10-26T06:48:30.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bf5a8fb80 10 client.0 _ll_register_callbacks cb 0x5649c51c1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:48:30.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bdaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c539a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bdb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c5399e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bdaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c539a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:56934/0 (socket says 172.21.15.89:56934) 2023-10-26T06:48:30.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:33444/0 (socket says 172.21.15.89:33444) 2023-10-26T06:48:30.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.941+0000 7f1bdaffd700 1 -- 172.21.15.89:0/4239361750 learned_addr learned my addr 172.21.15.89:0/4239361750 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:48:30.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bdaffd700 1 -- 172.21.15.89:0/4239361750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 msgr2=0x5649c53f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bdaffd700 1 -- 172.21.15.89:0/4239361750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 msgr2=0x5649c5399e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:48:30.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c5399e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bdaffd700 1 -- 172.21.15.89:0/4239361750 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd00041d0 con 0x5649c538cfb0 2023-10-26T06:48:30.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bdaffd700 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c539a3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0011fd0 tx=0x7f1bd0011d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d6da003fabbe60 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:48:30.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:48:30.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bbaffd700 1 -- 172.21.15.89:0/4239361750 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0027070 con 0x5649c538cfb0 2023-10-26T06:48:30.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bcc09b400 con 0x5649c538cfb0 2023-10-26T06:48:30.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.942+0000 7f1bbaffd700 1 -- 172.21.15.89:0/4239361750 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1bd0018070 con 0x5649c538cfb0 2023-10-26T06:48:30.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bbaffd700 1 -- 172.21.15.89:0/4239361750 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bd0003e70 con 0x5649c538cfb0 2023-10-26T06:48:30.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 10 client.25663 fetch_fsmap learned FSMap version 20 2023-10-26T06:48:30.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5649c512c250 con 0x5649c538cfb0 2023-10-26T06:48:30.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bbaffd700 1 -- 172.21.15.89:0/4239361750 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd0040020 con 0x5649c538cfb0 2023-10-26T06:48:30.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bbaffd700 1 -- 172.21.15.89:0/4239361750 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1bd0029b60 con 0x5649c538cfb0 2023-10-26T06:48:30.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 10 client.25663 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:48:30.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 10 client.25663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:48:30.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 20 client.25663 populate_metadata read hostname 'smithi089' 2023-10-26T06:48:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53fb8c0 0x5649c53fdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:48:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 4 client.25663 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:48:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5649c52d3e50 con 0x5649c53fb8c0 2023-10-26T06:48:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.943+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53fb8c0 0x5649c53fdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:48:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bbaffd700 10 client.25663 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bdb7fe700 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53fb8c0 0x5649c53fdce0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d2f834ee023ac1 server_cookie=e2eb19e1663758ef in_seq=0 out_seq=0 2023-10-26T06:48:30.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bbaffd700 1 -- 172.21.15.89:0/4239361750 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5649c52d3e50 con 0x5649c53fb8c0 2023-10-26T06:48:30.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bbaffd700 10 client.25663 handle_command_reply: tid=0 2023-10-26T06:48:30.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 client.25663 shutdown 2023-10-26T06:48:30.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 msgr2=0x5649c539a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c539a3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0011fd0 tx=0x7f1bd0011d80 comp rx=0 tx=0).stop 2023-10-26T06:48:30.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 shutdown_connections 2023-10-26T06:48:30.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649c538c690 0x5649c5399e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5649c53fb8c0 0x5649c53fdce0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649c5391100 0x5649c53f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 --2- 172.21.15.89:0/4239361750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649c538cfb0 0x5649c539a3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bbaffd700 0 client.25663 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:48:30.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 >> 172.21.15.89:0/4239361750 conn(0x5649c52fc720 msgr2=0x5649c53934f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 shutdown_connections 2023-10-26T06:48:30.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 1 -- 172.21.15.89:0/4239361750 wait complete. 2023-10-26T06:48:30.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.944+0000 7f1bf5a8fb80 20 client.25663 trim_cache size 0 max 16384 2023-10-26T06:48:30.960 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:48:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:48:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:48:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "ef3564d6-89cc-4366-bdf1-3efcb4245a96": { 2023-10-26T06:48:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:48:30.960 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "ef3564d6-89cc-4366-bdf1-3efcb4245a96", 2023-10-26T06:48:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:48:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:48:30.961 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:48:30.961 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:48:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1bc40a1670 msgr2=0x7f1bc40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1bc40a1670 0x7f1bc40a3b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc197bc0 tx=0x7f1bd0009260 comp rx=0 tx=0).stop 2023-10-26T06:48:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc100710 msgr2=0x7f1bdc19c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:48:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc100710 0x7f1bdc19c8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc004730 tx=0x7f1bcc004760 comp rx=0 tx=0).stop 2023-10-26T06:48:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 shutdown_connections 2023-10-26T06:48:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1bc40a1670 0x7f1bc40a3b30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bdc106720 0x7f1bdc196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1bdc100710 0x7f1bdc19c8d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 --2- 172.21.15.89:0/3190172374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bdc1010d0 0x7f1bdc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:48:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 >> 172.21.15.89:0/3190172374 conn(0x7f1bdc0fc200 msgr2=0x7f1bdc0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:48:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 shutdown_connections 2023-10-26T06:48:30.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:48:30.945+0000 7f1be21ff700 1 -- 172.21.15.89:0/3190172374 wait complete. 2023-10-26T06:48:30.965 INFO:tasks.cephfs.filesystem:scrub status for tag:ef3564d6-89cc-4366-bdf1-3efcb4245a96 - {'path': '/', 'tag': 'ef3564d6-89cc-4366-bdf1-3efcb4245a96', 'options': 'recursive,force'} 2023-10-26T06:48:31.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:31 smithi089 ceph-mon[95031]: pgmap v358: 97 pgs: 97 active+clean; 2.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 204 MiB/s wr, 126 op/s 2023-10-26T06:48:31.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:31 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:48:31.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:31 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:48:31.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:31 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:48:31.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:31 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:48:31.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:31 smithi192 ceph-mon[131661]: pgmap v358: 97 pgs: 97 active+clean; 2.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 204 MiB/s wr, 126 op/s 2023-10-26T06:48:31.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:31 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:48:31.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:31 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:48:31.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:31 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:48:31.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:31 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:48:31.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:31 smithi135 ceph-mon[131403]: pgmap v358: 97 pgs: 97 active+clean; 2.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 204 MiB/s wr, 126 op/s 2023-10-26T06:48:31.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:31 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:48:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:31 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:48:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:31 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:48:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:31 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:48:33.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:33 smithi089 ceph-mon[95031]: pgmap v359: 97 pgs: 97 active+clean; 2.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 223 MiB/s wr, 108 op/s 2023-10-26T06:48:33.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:33 smithi192 ceph-mon[131661]: pgmap v359: 97 pgs: 97 active+clean; 2.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 223 MiB/s wr, 108 op/s 2023-10-26T06:48:33.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:33 smithi135 ceph-mon[131403]: pgmap v359: 97 pgs: 97 active+clean; 2.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 223 MiB/s wr, 108 op/s 2023-10-26T06:48:34.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:34 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:48:34.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:34 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:48:34.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:34 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:48:35.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:35 smithi192 ceph-mon[131661]: pgmap v360: 97 pgs: 97 active+clean; 3.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 257 MiB/s wr, 108 op/s 2023-10-26T06:48:35.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:35 smithi135 ceph-mon[131403]: pgmap v360: 97 pgs: 97 active+clean; 3.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 257 MiB/s wr, 108 op/s 2023-10-26T06:48:35.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:35 smithi089 ceph-mon[95031]: pgmap v360: 97 pgs: 97 active+clean; 3.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 257 MiB/s wr, 108 op/s 2023-10-26T06:48:36.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:36 smithi135 ceph-mon[131403]: pgmap v361: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 269 MiB/s wr, 106 op/s 2023-10-26T06:48:36.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:36 smithi089 ceph-mon[95031]: pgmap v361: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 269 MiB/s wr, 106 op/s 2023-10-26T06:48:36.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:36 smithi192 ceph-mon[131661]: pgmap v361: 97 pgs: 97 active+clean; 3.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 269 MiB/s wr, 106 op/s 2023-10-26T06:48:39.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:38 smithi192 ceph-mon[131661]: pgmap v362: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 275 MiB/s wr, 90 op/s 2023-10-26T06:48:39.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:38 smithi135 ceph-mon[131403]: pgmap v362: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 275 MiB/s wr, 90 op/s 2023-10-26T06:48:39.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:38 smithi089 ceph-mon[95031]: pgmap v362: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 275 MiB/s wr, 90 op/s 2023-10-26T06:48:41.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:41 smithi192 ceph-mon[131661]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 242 MiB/s wr, 91 op/s 2023-10-26T06:48:41.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:41 smithi135 ceph-mon[131403]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 242 MiB/s wr, 91 op/s 2023-10-26T06:48:41.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:41 smithi089 ceph-mon[95031]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 242 MiB/s wr, 91 op/s 2023-10-26T06:48:43.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:43 smithi192 ceph-mon[131661]: pgmap v364: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 215 MiB/s wr, 76 op/s 2023-10-26T06:48:43.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:43 smithi135 ceph-mon[131403]: pgmap v364: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 215 MiB/s wr, 76 op/s 2023-10-26T06:48:43.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:43 smithi089 ceph-mon[95031]: pgmap v364: 97 pgs: 97 active+clean; 4.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 215 MiB/s wr, 76 op/s 2023-10-26T06:48:45.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:45 smithi135 ceph-mon[131403]: pgmap v365: 97 pgs: 97 active+clean; 5.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 250 MiB/s wr, 85 op/s 2023-10-26T06:48:45.543 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:45 smithi089 ceph-mon[95031]: pgmap v365: 97 pgs: 97 active+clean; 5.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 250 MiB/s wr, 85 op/s 2023-10-26T06:48:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:45 smithi192 ceph-mon[131661]: pgmap v365: 97 pgs: 97 active+clean; 5.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 250 MiB/s wr, 85 op/s 2023-10-26T06:48:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:47 smithi135 ceph-mon[131403]: pgmap v366: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 266 MiB/s wr, 99 op/s 2023-10-26T06:48:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:47 smithi089 ceph-mon[95031]: pgmap v366: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 266 MiB/s wr, 99 op/s 2023-10-26T06:48:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:47 smithi192 ceph-mon[131661]: pgmap v366: 97 pgs: 97 active+clean; 6.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 266 MiB/s wr, 99 op/s 2023-10-26T06:48:49.468 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:49 smithi135 ceph-mon[131403]: pgmap v367: 97 pgs: 97 active+clean; 6.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-10-26T06:48:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:49 smithi089 ceph-mon[95031]: pgmap v367: 97 pgs: 97 active+clean; 6.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-10-26T06:48:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:49 smithi192 ceph-mon[131661]: pgmap v367: 97 pgs: 97 active+clean; 6.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-10-26T06:48:51.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:51 smithi135 ceph-mon[131403]: pgmap v368: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 270 MiB/s wr, 98 op/s 2023-10-26T06:48:51.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:51 smithi089 ceph-mon[95031]: pgmap v368: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 270 MiB/s wr, 98 op/s 2023-10-26T06:48:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:51 smithi192 ceph-mon[131661]: pgmap v368: 97 pgs: 97 active+clean; 7.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 270 MiB/s wr, 98 op/s 2023-10-26T06:48:53.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:53 smithi135 ceph-mon[131403]: pgmap v369: 97 pgs: 97 active+clean; 7.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 249 MiB/s wr, 83 op/s 2023-10-26T06:48:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:53 smithi089 ceph-mon[95031]: pgmap v369: 97 pgs: 97 active+clean; 7.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 249 MiB/s wr, 83 op/s 2023-10-26T06:48:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:53 smithi192 ceph-mon[131661]: pgmap v369: 97 pgs: 97 active+clean; 7.6 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 249 MiB/s wr, 83 op/s 2023-10-26T06:48:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:55 smithi135 ceph-mon[131403]: pgmap v370: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 295 MiB/s wr, 95 op/s 2023-10-26T06:48:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:55 smithi089 ceph-mon[95031]: pgmap v370: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 295 MiB/s wr, 95 op/s 2023-10-26T06:48:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:55 smithi192 ceph-mon[131661]: pgmap v370: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 295 MiB/s wr, 95 op/s 2023-10-26T06:48:56.282 INFO:tasks.workunit.client.0.smithi089.stdout:fs setup took 43 secs 2023-10-26T06:48:57.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:57 smithi135 ceph-mon[131403]: pgmap v371: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 295 MiB/s wr, 106 op/s 2023-10-26T06:48:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:57 smithi089 ceph-mon[95031]: pgmap v371: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 295 MiB/s wr, 106 op/s 2023-10-26T06:48:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:57 smithi192 ceph-mon[131661]: pgmap v371: 97 pgs: 97 active+clean; 9.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 295 MiB/s wr, 106 op/s 2023-10-26T06:48:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:48:59 smithi089 ceph-mon[95031]: pgmap v372: 97 pgs: 97 active+clean; 9.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 292 MiB/s wr, 96 op/s 2023-10-26T06:48:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:48:59 smithi192 ceph-mon[131661]: pgmap v372: 97 pgs: 97 active+clean; 9.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 292 MiB/s wr, 96 op/s 2023-10-26T06:48:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:48:59 smithi135 ceph-mon[131403]: pgmap v372: 97 pgs: 97 active+clean; 9.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 292 MiB/s wr, 96 op/s 2023-10-26T06:49:00.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:49:01.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.355+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/2095248455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd8100720 msgr2=0x7fbfd8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.355+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/2095248455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd8100720 0x7fbfd8100ba0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0098f0 tx=0x7fbfcc009c00 comp rx=0 tx=0).stop 2023-10-26T06:49:01.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.356+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/2095248455 shutdown_connections 2023-10-26T06:49:01.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.356+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/2095248455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfd81010e0 0x7fbfd810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.356+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/2095248455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfd8106730 0x7fbfd8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.356+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/2095248455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd8100720 0x7fbfd8100ba0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.356+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/2095248455 >> 172.21.15.89:0/2095248455 conn(0x7fbfd80fc1b0 msgr2=0x7fbfd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:01.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.356+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/2095248455 shutdown_connections 2023-10-26T06:49:01.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.356+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/2095248455 wait complete. 2023-10-26T06:49:01.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.357+0000 7fbfdd9ff700 1 Processor -- start 2023-10-26T06:49:01.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.357+0000 7fbfdd9ff700 1 -- start start 2023-10-26T06:49:01.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfdd9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfd8100720 0x7fbfd810c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfdd9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd81010e0 0x7fbfd810cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfdd9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfd8106730 0x7fbfd81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfdd9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd81a20f0 con 0x7fbfd81010e0 2023-10-26T06:49:01.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfdd9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd81a2260 con 0x7fbfd8100720 2023-10-26T06:49:01.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfdd9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd81a2420 con 0x7fbfd8106730 2023-10-26T06:49:01.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfd67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd81010e0 0x7fbfd810cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfd67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd81010e0 0x7fbfd810cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55042/0 (socket says 172.21.15.89:55042) 2023-10-26T06:49:01.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfd67fc700 1 -- 172.21.15.89:0/1352234653 learned_addr learned my addr 172.21.15.89:0/1352234653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:01.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.358+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfd8100720 0x7fbfd810c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.359+0000 7fbfd77fe700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfd8106730 0x7fbfd81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.359+0000 7fbfd67fc700 1 -- 172.21.15.89:0/1352234653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfd8106730 msgr2=0x7fbfd81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.359+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfd8106730 0x7fbfd81a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.359+0000 7fbfd67fc700 1 -- 172.21.15.89:0/1352234653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfd8100720 msgr2=0x7fbfd810c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.359+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfd8100720 0x7fbfd810c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.359+0000 7fbfd67fc700 1 -- 172.21.15.89:0/1352234653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc009520 con 0x7fbfd81010e0 2023-10-26T06:49:01.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.359+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd81010e0 0x7fbfd810cd50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009230 tx=0x7fbfcc00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.360+0000 7fbfdc9fd700 1 -- 172.21.15.89:0/1352234653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc01c070 con 0x7fbfd81010e0 2023-10-26T06:49:01.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.360+0000 7fbfdc9fd700 1 -- 172.21.15.89:0/1352234653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfcc00bbd0 con 0x7fbfd81010e0 2023-10-26T06:49:01.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.360+0000 7fbfdc9fd700 1 -- 172.21.15.89:0/1352234653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0038e0 con 0x7fbfd81010e0 2023-10-26T06:49:01.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.360+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd81a26a0 con 0x7fbfd81010e0 2023-10-26T06:49:01.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.361+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd81a9f40 con 0x7fbfd81010e0 2023-10-26T06:49:01.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.363+0000 7fbfdc9fd700 1 -- 172.21.15.89:0/1352234653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbfcc020030 con 0x7fbfd81010e0 2023-10-26T06:49:01.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.365+0000 7fbfdc9fd700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbfc00a16b0 0x7fbfc00a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.365+0000 7fbfdc9fd700 1 -- 172.21.15.89:0/1352234653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbfcc0cdec0 con 0x7fbfd81010e0 2023-10-26T06:49:01.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbfc00a16b0 0x7fbfc00a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 Processor -- start 2023-10-26T06:49:01.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 -- start start 2023-10-26T06:49:01.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 0x56160241d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241dbb0 0x5616024217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561602421d00 0x561602422160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160237f6e0 con 0x561602421d00 2023-10-26T06:49:01.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x561602365c60 con 0x56160241d290 2023-10-26T06:49:01.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.366+0000 7fbff1237b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561602363580 con 0x56160241dbb0 2023-10-26T06:49:01.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561602421d00 0x561602422160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561602421d00 0x561602422160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55052/0 (socket says 172.21.15.89:55052) 2023-10-26T06:49:01.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 -- 172.21.15.89:0/3452948181 learned_addr learned my addr 172.21.15.89:0/3452948181 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:01.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241dbb0 0x5616024217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 -- 172.21.15.89:0/3452948181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241dbb0 msgr2=0x5616024217c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:01.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241dbb0 0x5616024217c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 -- 172.21.15.89:0/3452948181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 msgr2=0x56160241d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:01.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 0x56160241d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.367+0000 7fbfd77fe700 1 -- 172.21.15.89:0/3452948181 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160231c3f0 con 0x561602421d00 2023-10-26T06:49:01.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.368+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbfc00a16b0 0x7fbfc00a3b70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8005fd0 tx=0x7fbfc8011720 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.368+0000 7fbfd77fe700 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561602421d00 0x561602422160 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009440 tx=0x7fbfd0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec7b9fdb9eb3c3f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.368+0000 7fbfbdffb700 1 -- 172.21.15.89:0/3452948181 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000e070 con 0x561602421d00 2023-10-26T06:49:01.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.368+0000 7fbfbdffb700 1 -- 172.21.15.89:0/3452948181 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd000f040 con 0x561602421d00 2023-10-26T06:49:01.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.368+0000 7fbfbdffb700 1 -- 172.21.15.89:0/3452948181 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0015b00 con 0x561602421d00 2023-10-26T06:49:01.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 -- 172.21.15.89:0/3452948181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561602421d00 msgr2=0x561602422160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561602421d00 0x561602422160 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009440 tx=0x7fbfd0009920 comp rx=0 tx=0).stop 2023-10-26T06:49:01.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 -- 172.21.15.89:0/3452948181 shutdown_connections 2023-10-26T06:49:01.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561602421d00 0x561602422160 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 0x56160241d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3452948181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241dbb0 0x5616024217c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 -- 172.21.15.89:0/3452948181 >> 172.21.15.89:0/3452948181 conn(0x561602380720 msgr2=0x561602406b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:01.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 -- 172.21.15.89:0/3452948181 shutdown_connections 2023-10-26T06:49:01.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.369+0000 7fbff1237b80 1 -- 172.21.15.89:0/3452948181 wait complete. 2023-10-26T06:49:01.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.370+0000 7fbff1237b80 1 -- start start 2023-10-26T06:49:01.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.370+0000 7fbff1237b80 1 Processor -- start 2023-10-26T06:49:01.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.374+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 0x561602429440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.374+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602426780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.374+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 0x561602426db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.374+0000 7fbff1237b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160235ee70 con 0x56160241dbb0 2023-10-26T06:49:01.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.374+0000 7fbff1237b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616022fb710 con 0x56160241d290 2023-10-26T06:49:01.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.374+0000 7fbff1237b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616022fb3b0 con 0x561602421d00 2023-10-26T06:49:01.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.374+0000 7fbff1237b80 10 client.0 _ll_register_callbacks cb 0x561602245a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:01.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.376+0000 7fbfd67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602426780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.377+0000 7fbfd6ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 0x561602429440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.378+0000 7fbfd67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602426780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55054/0 (socket says 172.21.15.89:55054) 2023-10-26T06:49:01.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.378+0000 7fbfd67fc700 1 -- 172.21.15.89:0/3351025716 learned_addr learned my addr 172.21.15.89:0/3351025716 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:01.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.379+0000 7fbfd67fc700 1 -- 172.21.15.89:0/3351025716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 msgr2=0x561602426db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:49:01.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.379+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 0x561602426db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.379+0000 7fbfd67fc700 1 -- 172.21.15.89:0/3351025716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 msgr2=0x561602429440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:01.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.379+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 0x561602429440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.379+0000 7fbfd67fc700 1 -- 172.21.15.89:0/3351025716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd00090f0 con 0x56160241dbb0 2023-10-26T06:49:01.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.379+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602426780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0cb6b0 tx=0x7fbfcc09a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07d3f651ba7c46b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.379+0000 7fbfb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:01.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.380+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/3351025716 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0b2070 con 0x56160241dbb0 2023-10-26T06:49:01.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.380+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/3351025716 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbfcc0a3270 con 0x56160241dbb0 2023-10-26T06:49:01.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.380+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561602363ef0 con 0x56160241dbb0 2023-10-26T06:49:01.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.381+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/3351025716 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfcc0cb8d0 con 0x56160241dbb0 2023-10-26T06:49:01.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.381+0000 7fbff1237b80 10 client.15681 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:01.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.381+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x561602189de0 con 0x56160241dbb0 2023-10-26T06:49:01.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.381+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/3351025716 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561602363ef0 con 0x56160241dbb0 2023-10-26T06:49:01.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.382+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/3351025716 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbfcc0051d0 con 0x56160241dbb0 2023-10-26T06:49:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.382+0000 7fbff1237b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.382+0000 7fbff1237b80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.382+0000 7fbff1237b80 20 client.15681 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.382+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x561602438fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.382+0000 7fbff1237b80 4 client.15681 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:01.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.382+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616023db8b0 con 0x561602436b80 2023-10-26T06:49:01.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.385+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x561602438fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.388+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x561602438fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a962e0a4902260 server_cookie=6deb445929fd05b8 in_seq=0 out_seq=0 2023-10-26T06:49:01.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.388+0000 7fbfb6ffd700 10 client.15681 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:01.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.389+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/3351025716 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5616023db8b0 con 0x561602436b80 2023-10-26T06:49:01.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.389+0000 7fbfb6ffd700 10 client.15681 handle_command_reply: tid=0 2023-10-26T06:49:01.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.389+0000 7fbff1237b80 1 client.15681 shutdown 2023-10-26T06:49:01.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.389+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 msgr2=0x561602426780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.389+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602426780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0cb6b0 tx=0x7fbfcc09a980 comp rx=0 tx=0).stop 2023-10-26T06:49:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 shutdown_connections 2023-10-26T06:49:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x561602438fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 0x561602426db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbfb6ffd700 0 client.15681 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:01.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241d290 0x561602429440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbff1237b80 1 --2- 172.21.15.89:0/3351025716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602426780 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 >> 172.21.15.89:0/3351025716 conn(0x561602380720 msgr2=0x56160240b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:01.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.390+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 shutdown_connections 2023-10-26T06:49:01.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.391+0000 7fbff1237b80 1 -- 172.21.15.89:0/3351025716 wait complete. 2023-10-26T06:49:01.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.391+0000 7fbff1237b80 20 client.15681 trim_cache size 0 max 16384 2023-10-26T06:49:01.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 Processor -- start 2023-10-26T06:49:01.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 -- start start 2023-10-26T06:49:01.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241d290 0x56160242def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241dbb0 0x5616024230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 0x56160241fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616021df920 con 0x56160241d290 2023-10-26T06:49:01.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616024078a0 con 0x56160241dbb0 2023-10-26T06:49:01.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.418+0000 7fbff1237b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616023fefd0 con 0x561602421d00 2023-10-26T06:49:01.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241d290 0x56160242def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241d290 0x56160242def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55060/0 (socket says 172.21.15.89:55060) 2023-10-26T06:49:01.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 -- 172.21.15.89:0/470186805 learned_addr learned my addr 172.21.15.89:0/470186805 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:01.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd77fe700 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 0x56160241fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241dbb0 0x5616024230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 -- 172.21.15.89:0/470186805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 msgr2=0x56160241fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:01.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 0x56160241fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 -- 172.21.15.89:0/470186805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241dbb0 msgr2=0x5616024230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:01.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241dbb0 0x5616024230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.419+0000 7fbfd6ffd700 1 -- 172.21.15.89:0/470186805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561602172c60 con 0x56160241d290 2023-10-26T06:49:01.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.420+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241dbb0 0x5616024230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:01.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.421+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241d290 0x56160242def0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009350 tx=0x7fbfc8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2e4fcb6c3aa11f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.422+0000 7fbfbe7fc700 1 -- 172.21.15.89:0/470186805 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc801e320 con 0x56160241d290 2023-10-26T06:49:01.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.422+0000 7fbfbe7fc700 1 -- 172.21.15.89:0/470186805 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc8024040 con 0x56160241d290 2023-10-26T06:49:01.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.422+0000 7fbfbe7fc700 1 -- 172.21.15.89:0/470186805 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8025c10 con 0x56160241d290 2023-10-26T06:49:01.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.423+0000 7fbff1237b80 1 -- 172.21.15.89:0/470186805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241d290 msgr2=0x56160242def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.423+0000 7fbff1237b80 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241d290 0x56160242def0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009350 tx=0x7fbfc8014000 comp rx=0 tx=0).stop 2023-10-26T06:49:01.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.423+0000 7fbff1237b80 1 -- 172.21.15.89:0/470186805 shutdown_connections 2023-10-26T06:49:01.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.424+0000 7fbff1237b80 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241d290 0x56160242def0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.424+0000 7fbff1237b80 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561602421d00 0x56160241fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.424+0000 7fbff1237b80 1 --2- 172.21.15.89:0/470186805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56160241dbb0 0x5616024230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.424+0000 7fbff1237b80 1 -- 172.21.15.89:0/470186805 >> 172.21.15.89:0/470186805 conn(0x561602380720 msgr2=0x56160240b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:01.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.424+0000 7fbff1237b80 1 -- 172.21.15.89:0/470186805 shutdown_connections 2023-10-26T06:49:01.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.424+0000 7fbff1237b80 1 -- 172.21.15.89:0/470186805 wait complete. 2023-10-26T06:49:01.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.425+0000 7fbff1237b80 1 -- start start 2023-10-26T06:49:01.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.425+0000 7fbff1237b80 1 Processor -- start 2023-10-26T06:49:01.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.426+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241d290 0x5616024220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.426+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602422620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.426+0000 7fbff1237b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561602421d00 0x561602422b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.426+0000 7fbff1237b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561602423140 con 0x56160241dbb0 2023-10-26T06:49:01.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.426+0000 7fbff1237b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616024232b0 con 0x561602421d00 2023-10-26T06:49:01.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.426+0000 7fbff1237b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561602423470 con 0x56160241d290 2023-10-26T06:49:01.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.426+0000 7fbff1237b80 10 client.0 _ll_register_callbacks cb 0x561602245a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:01.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.427+0000 7fbfd67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602422620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.427+0000 7fbfd6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241d290 0x5616024220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.427+0000 7fbfd67fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602422620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55076/0 (socket says 172.21.15.89:55076) 2023-10-26T06:49:01.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.427+0000 7fbfd77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561602421d00 0x561602422b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.427+0000 7fbfd6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241d290 0x5616024220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:36902/0 (socket says 172.21.15.89:36902) 2023-10-26T06:49:01.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.427+0000 7fbfd6ffd700 1 -- 172.21.15.89:0/1562992369 learned_addr learned my addr 172.21.15.89:0/1562992369 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:01.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfd67fc700 1 -- 172.21.15.89:0/1562992369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241d290 msgr2=0x5616024220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:01.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241d290 0x5616024220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfd67fc700 1 -- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561602421d00 msgr2=0x561602422b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:01.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561602421d00 0x561602422b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfd67fc700 1 -- 172.21.15.89:0/1562992369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc8017070 con 0x56160241dbb0 2023-10-26T06:49:01.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfd77fe700 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561602421d00 0x561602422b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:49:01.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfd67fc700 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602422620 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0979e0 tx=0x7fbfcc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f3cd6c1899591e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.428+0000 7fbfb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:01.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.429+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/1562992369 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0b2070 con 0x56160241dbb0 2023-10-26T06:49:01.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.429+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/1562992369 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbfcc0a3270 con 0x56160241dbb0 2023-10-26T06:49:01.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.429+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfd000f040 con 0x56160241dbb0 2023-10-26T06:49:01.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.429+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/1562992369 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfcc003b10 con 0x56160241dbb0 2023-10-26T06:49:01.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 10 client.15687 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:01.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5616021b0250 con 0x56160241dbb0 2023-10-26T06:49:01.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/1562992369 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd000f040 con 0x56160241dbb0 2023-10-26T06:49:01.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/1562992369 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbfcc09a460 con 0x56160241dbb0 2023-10-26T06:49:01.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:01.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:01.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 20 client.15687 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:01.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x56160249e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 4 client.15687 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:49:01.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.431+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561602357e50 con 0x561602436b80 2023-10-26T06:49:01.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.432+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x56160249e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.433+0000 7fbfb6ffd700 10 client.15687 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:01.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.433+0000 7fbfd6ffd700 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x56160249e860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a139b609237e9c server_cookie=fed11997288dc3cf in_seq=0 out_seq=0 2023-10-26T06:49:01.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbfb6ffd700 1 -- 172.21.15.89:0/1562992369 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561602357e50 con 0x561602436b80 2023-10-26T06:49:01.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbfb6ffd700 10 client.15687 handle_command_reply: tid=0 2023-10-26T06:49:01.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 client.15687 shutdown 2023-10-26T06:49:01.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 msgr2=0x561602422620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602422620 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0979e0 tx=0x7fbfcc009c00 comp rx=0 tx=0).stop 2023-10-26T06:49:01.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 shutdown_connections 2023-10-26T06:49:01.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561602436b80 0x56160249e860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561602421d00 0x561602422b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56160241d290 0x5616024220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 --2- 172.21.15.89:0/1562992369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56160241dbb0 0x561602422620 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbfb6ffd700 0 client.15687 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:01.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.434+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 >> 172.21.15.89:0/1562992369 conn(0x561602380720 msgr2=0x56160241fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:01.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.435+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 shutdown_connections 2023-10-26T06:49:01.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.435+0000 7fbff1237b80 1 -- 172.21.15.89:0/1562992369 wait complete. 2023-10-26T06:49:01.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.435+0000 7fbff1237b80 20 client.15687 trim_cache size 0 max 16384 2023-10-26T06:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:49:01.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.436+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbfc00a16b0 msgr2=0x7fbfc00a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.436+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbfc00a16b0 0x7fbfc00a3b70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8005fd0 tx=0x7fbfc8011720 comp rx=0 tx=0).stop 2023-10-26T06:49:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.436+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd81010e0 msgr2=0x7fbfd810cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.436+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd81010e0 0x7fbfd810cd50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009230 tx=0x7fbfcc00bfb0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 shutdown_connections 2023-10-26T06:49:01.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbfc00a16b0 0x7fbfc00a3b70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfd8106730 0x7fbfd81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfd8100720 0x7fbfd810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 --2- 172.21.15.89:0/1352234653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfd81010e0 0x7fbfd810cd50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 >> 172.21.15.89:0/1352234653 conn(0x7fbfd80fc1b0 msgr2=0x7fbfd80fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:01.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 shutdown_connections 2023-10-26T06:49:01.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.437+0000 7fbfdd9ff700 1 -- 172.21.15.89:0/1352234653 wait complete. 2023-10-26T06:49:01.457 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:01 smithi089 ceph-mon[95031]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 285 MiB/s wr, 99 op/s 2023-10-26T06:49:01.459 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:49:01.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:49:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:01 smithi192 ceph-mon[131661]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 285 MiB/s wr, 99 op/s 2023-10-26T06:49:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:01 smithi135 ceph-mon[131403]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 285 MiB/s wr, 99 op/s 2023-10-26T06:49:01.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 -- 172.21.15.89:0/1340496013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c067830 msgr2=0x7f883c067cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 --2- 172.21.15.89:0/1340496013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c067830 0x7f883c067cb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8824009990 tx=0x7f8824009ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 -- 172.21.15.89:0/1340496013 shutdown_connections 2023-10-26T06:49:01.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 --2- 172.21.15.89:0/1340496013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883c0681f0 0x7f883c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 --2- 172.21.15.89:0/1340496013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f883c1066d0 0x7f883c106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 --2- 172.21.15.89:0/1340496013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c067830 0x7f883c067cb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 -- 172.21.15.89:0/1340496013 >> 172.21.15.89:0/1340496013 conn(0x7f883c076400 msgr2=0x7f883c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:01.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.835+0000 7f88427ff700 1 -- 172.21.15.89:0/1340496013 shutdown_connections 2023-10-26T06:49:01.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.836+0000 7f88427ff700 1 -- 172.21.15.89:0/1340496013 wait complete. 2023-10-26T06:49:01.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.837+0000 7f88427ff700 1 Processor -- start 2023-10-26T06:49:01.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.840+0000 7f88427ff700 1 -- start start 2023-10-26T06:49:01.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.841+0000 7f88427ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883c067830 0x7f883c10dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.841+0000 7f88427ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f883c0681f0 0x7f883c10e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.841+0000 7f88427ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c1066d0 0x7f883c10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.841+0000 7f88427ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1a94b0 con 0x7f883c1066d0 2023-10-26T06:49:01.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.841+0000 7f88427ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1a95f0 con 0x7f883c0681f0 2023-10-26T06:49:01.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.841+0000 7f88427ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1a9760 con 0x7f883c067830 2023-10-26T06:49:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.842+0000 7f8840d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c1066d0 0x7f883c10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.842+0000 7f8840d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c1066d0 0x7f883c10e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55102/0 (socket says 172.21.15.89:55102) 2023-10-26T06:49:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.842+0000 7f8840d9c700 1 -- 172.21.15.89:0/55825651 learned_addr learned my addr 172.21.15.89:0/55825651 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.845+0000 7f883b7fe700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f883c0681f0 0x7f883c10e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.845+0000 7f883bfff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883c067830 0x7f883c10dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.846+0000 7f8840d9c700 1 -- 172.21.15.89:0/55825651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883c067830 msgr2=0x7f883c10dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.846+0000 7f8840d9c700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883c067830 0x7f883c10dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.846+0000 7f8840d9c700 1 -- 172.21.15.89:0/55825651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f883c0681f0 msgr2=0x7f883c10e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.846+0000 7f8840d9c700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f883c0681f0 0x7f883c10e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.846+0000 7f8840d9c700 1 -- 172.21.15.89:0/55825651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88240095f0 con 0x7f883c1066d0 2023-10-26T06:49:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.846+0000 7f883b7fe700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f883c0681f0 0x7f883c10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.846+0000 7f8840d9c700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c1066d0 0x7f883c10e6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b550 tx=0x7f882c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.847+0000 7f88397fa700 1 -- 172.21.15.89:0/55825651 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c011830 con 0x7f883c1066d0 2023-10-26T06:49:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.848+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c1a9a40 con 0x7f883c1066d0 2023-10-26T06:49:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.848+0000 7f88397fa700 1 -- 172.21.15.89:0/55825651 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882c011e50 con 0x7f883c1066d0 2023-10-26T06:49:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.848+0000 7f88397fa700 1 -- 172.21.15.89:0/55825651 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c010e70 con 0x7f883c1066d0 2023-10-26T06:49:01.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.848+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c1a9f50 con 0x7f883c1066d0 2023-10-26T06:49:01.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.854+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883c108c00 con 0x7f883c1066d0 2023-10-26T06:49:01.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.856+0000 7f88397fa700 1 -- 172.21.15.89:0/55825651 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f882c010510 con 0x7f883c1066d0 2023-10-26T06:49:01.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.858+0000 7f88397fa700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f88280a1730 0x7f88280a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:01.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.859+0000 7f88397fa700 1 -- 172.21.15.89:0/55825651 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f882c0ccab0 con 0x7f883c1066d0 2023-10-26T06:49:01.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.860+0000 7f883bfff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f88280a1730 0x7f88280a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:01.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.863+0000 7f883bfff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f88280a1730 0x7f88280a3bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8834009f20 tx=0x7f88340094f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:01.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:01.871+0000 7f88397fa700 1 -- 172.21.15.89:0/55825651 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f882c095ec0 con 0x7f883c1066d0 2023-10-26T06:49:02.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.341+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f883c067530 con 0x7f883c1066d0 2023-10-26T06:49:02.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.342+0000 7f88397fa700 1 -- 172.21.15.89:0/55825651 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f882c015090 con 0x7f883c1066d0 2023-10-26T06:49:02.344 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:49:02.345 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:49:02.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f88280a1730 msgr2=0x7f88280a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f88280a1730 0x7f88280a3bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8834009f20 tx=0x7f88340094f0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c1066d0 msgr2=0x7f883c10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c1066d0 0x7f883c10e6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b550 tx=0x7f882c00b910 comp rx=0 tx=0).stop 2023-10-26T06:49:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 shutdown_connections 2023-10-26T06:49:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f88280a1730 0x7f88280a3bf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f883c1066d0 0x7f883c10e6f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883c067830 0x7f883c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 --2- 172.21.15.89:0/55825651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f883c0681f0 0x7f883c10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 >> 172.21.15.89:0/55825651 conn(0x7f883c076400 msgr2=0x7f883c0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:02.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 shutdown_connections 2023-10-26T06:49:02.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.345+0000 7f88427ff700 1 -- 172.21.15.89:0/55825651 wait complete. 2023-10-26T06:49:02.350 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:49:02.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:49:02.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.679+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3108291949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe14101790 msgr2=0x7fbe14101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.679+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3108291949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe14101790 0x7fbe14101b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009990 tx=0x7fbe04009ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.682+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3108291949 shutdown_connections 2023-10-26T06:49:02.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.682+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3108291949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe1410abb0 0x7fbe1410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.682+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3108291949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe14101790 0x7fbe14101b70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.682+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3108291949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe141020b0 0x7fbe1410a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.682+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3108291949 >> 172.21.15.89:0/3108291949 conn(0x7fbe140fafb0 msgr2=0x7fbe140fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:02.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.682+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3108291949 shutdown_connections 2023-10-26T06:49:02.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.682+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3108291949 wait complete. 2023-10-26T06:49:02.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.684+0000 7fbe18d9d700 1 Processor -- start 2023-10-26T06:49:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.684+0000 7fbe18d9d700 1 -- start start 2023-10-26T06:49:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.684+0000 7fbe18d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe14101790 0x7fbe1419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.684+0000 7fbe18d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe141020b0 0x7fbe1419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe18d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1410abb0 0x7fbe141969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe18d9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe14197000 con 0x7fbe1410abb0 2023-10-26T06:49:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe18d9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe14197170 con 0x7fbe14101790 2023-10-26T06:49:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe18d9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe14197330 con 0x7fbe141020b0 2023-10-26T06:49:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe14101790 0x7fbe1419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe14101790 0x7fbe1419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52896/0 (socket says 172.21.15.89:52896) 2023-10-26T06:49:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 -- 172.21.15.89:0/3388625048 learned_addr learned my addr 172.21.15.89:0/3388625048 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe13fff700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1410abb0 0x7fbe141969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe141020b0 0x7fbe1419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 -- 172.21.15.89:0/3388625048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe141020b0 msgr2=0x7fbe1419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe141020b0 0x7fbe1419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 -- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1410abb0 msgr2=0x7fbe141969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1410abb0 0x7fbe141969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe137fe700 1 -- 172.21.15.89:0/3388625048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040095f0 con 0x7fbe14101790 2023-10-26T06:49:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe13fff700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1410abb0 0x7fbe141969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.685+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe141020b0 0x7fbe1419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:02.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.686+0000 7fbe137fe700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe14101790 0x7fbe1419c870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400b2e0 tx=0x7fbe04005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.686+0000 7fbe10ff9700 1 -- 172.21.15.89:0/3388625048 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0401d070 con 0x7fbe14101790 2023-10-26T06:49:02.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.686+0000 7fbe10ff9700 1 -- 172.21.15.89:0/3388625048 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe0400fb90 con 0x7fbe14101790 2023-10-26T06:49:02.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.686+0000 7fbe10ff9700 1 -- 172.21.15.89:0/3388625048 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe04017650 con 0x7fbe14101790 2023-10-26T06:49:02.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.686+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe141975b0 con 0x7fbe14101790 2023-10-26T06:49:02.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.686+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe141a3860 con 0x7fbe14101790 2023-10-26T06:49:02.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.689+0000 7fbe10ff9700 1 -- 172.21.15.89:0/3388625048 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbe04021410 con 0x7fbe14101790 2023-10-26T06:49:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.691+0000 7fbe10ff9700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbdfc0a1670 0x7fbdfc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.691+0000 7fbe10ff9700 1 -- 172.21.15.89:0/3388625048 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbe040ce6e0 con 0x7fbe14101790 2023-10-26T06:49:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.691+0000 7fbe2d9ddb80 1 Processor -- start 2023-10-26T06:49:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe2d9ddb80 1 -- start start 2023-10-26T06:49:02.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc05c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fb676e0 con 0x557f1fc05830 2023-10-26T06:49:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fb4dc60 con 0x557f1fc0ad30 2023-10-26T06:49:02.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fb4b580 con 0x557f1fc0b650 2023-10-26T06:49:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.692+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbdfc0a1670 0x7fbdfc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.693+0000 7fbe12ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.693+0000 7fbe12ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc052f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:36958/0 (socket says 172.21.15.89:36958) 2023-10-26T06:49:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.693+0000 7fbe12ffd700 1 -- 172.21.15.89:0/2502510395 learned_addr learned my addr 172.21.15.89:0/2502510395 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:02.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.693+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbdfc0a1670 0x7fbdfc0a3b30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe14197b60 tx=0x7fbe080116c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:02.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.693+0000 7fbe13fff700 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc05c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.693+0000 7fbe137fe700 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe12ffd700 1 -- 172.21.15.89:0/2502510395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 msgr2=0x557f1fc0b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe12ffd700 1 -- 172.21.15.89:0/2502510395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 msgr2=0x557f1fc05c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc05c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe12ffd700 1 -- 172.21.15.89:0/2502510395 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f1fb043f0 con 0x557f1fc0b650 2023-10-26T06:49:02.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe13fff700 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc05c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:02.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe137fe700 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:02.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc052f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe08011f00 tx=0x7fbe0801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c07d2157ada99d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:02.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbdf9ffb700 1 -- 172.21.15.89:0/2502510395 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08014e90 con 0x557f1fc0b650 2023-10-26T06:49:02.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.694+0000 7fbdf9ffb700 1 -- 172.21.15.89:0/2502510395 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe080044f0 con 0x557f1fc0b650 2023-10-26T06:49:02.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.695+0000 7fbdf9ffb700 1 -- 172.21.15.89:0/2502510395 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08003d20 con 0x557f1fc0b650 2023-10-26T06:49:02.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.695+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/2502510395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 msgr2=0x557f1fc052f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.695+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc052f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe08011f00 tx=0x7fbe0801c000 comp rx=0 tx=0).stop 2023-10-26T06:49:02.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/2502510395 shutdown_connections 2023-10-26T06:49:02.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc05c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/2502510395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc052f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/2502510395 >> 172.21.15.89:0/2502510395 conn(0x557f1fb68720 msgr2=0x557f1fbeecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/2502510395 shutdown_connections 2023-10-26T06:49:02.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/2502510395 wait complete. 2023-10-26T06:49:02.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.696+0000 7fbe2d9ddb80 1 -- start start 2023-10-26T06:49:02.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 1 Processor -- start 2023-10-26T06:49:02.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc0ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc0f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fb46e70 con 0x557f1fc05830 2023-10-26T06:49:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fae3710 con 0x557f1fc0ad30 2023-10-26T06:49:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fae33b0 con 0x557f1fc0b650 2023-10-26T06:49:02.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.697+0000 7fbe2d9ddb80 10 client.0 _ll_register_callbacks cb 0x557f1fa2da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:02.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.698+0000 7fbe12ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.698+0000 7fbe12ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52920/0 (socket says 172.21.15.89:52920) 2023-10-26T06:49:02.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.698+0000 7fbe12ffd700 1 -- 172.21.15.89:0/1571787009 learned_addr learned my addr 172.21.15.89:0/1571787009 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:02.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.698+0000 7fbe13fff700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc0f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.698+0000 7fbe137fe700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc0ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe137fe700 1 -- 172.21.15.89:0/1571787009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 msgr2=0x557f1fc0f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe137fe700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc0f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe137fe700 1 -- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 msgr2=0x557f1fc0ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe137fe700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe13fff700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc0f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe137fe700 1 -- 172.21.15.89:0/1571787009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe08011c70 con 0x557f1fc05830 2023-10-26T06:49:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:49:02.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe137fe700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc0ea50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbe040cbe10 tx=0x7fbe0409b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdde126625c3514 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbdf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/1571787009 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0409bb90 con 0x557f1fc05830 2023-10-26T06:49:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f1fb4bef0 con 0x557f1fc05830 2023-10-26T06:49:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.699+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/1571787009 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbe0409f880 con 0x557f1fc05830 2023-10-26T06:49:02.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/1571787009 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe040a3a30 con 0x557f1fc05830 2023-10-26T06:49:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 10 client.15705 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557f1f971de0 con 0x557f1fc05830 2023-10-26T06:49:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/1571787009 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f1fb4bef0 con 0x557f1fc05830 2023-10-26T06:49:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/1571787009 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbe040b9ba0 con 0x557f1fc05830 2023-10-26T06:49:02.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 10 client.15705 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 20 client.15705 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc231c0 0x557f1fc255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 4 client.15705 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:02.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.701+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f1fbc38b0 con 0x557f1fc231c0 2023-10-26T06:49:02.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.702+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc231c0 0x557f1fc255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.703+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc231c0 0x557f1fc255e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f5bb736bf81a5e server_cookie=ac39ac52cbdcc532 in_seq=0 out_seq=0 2023-10-26T06:49:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.703+0000 7fbdf2ffd700 10 client.15705 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.703+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/1571787009 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f1fbc38b0 con 0x557f1fc231c0 2023-10-26T06:49:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.703+0000 7fbdf2ffd700 10 client.15705 handle_command_reply: tid=0 2023-10-26T06:49:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.703+0000 7fbe2d9ddb80 1 client.15705 shutdown 2023-10-26T06:49:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 msgr2=0x557f1fc0ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc0ea50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbe040cbe10 tx=0x7fbe0409b090 comp rx=0 tx=0).stop 2023-10-26T06:49:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 shutdown_connections 2023-10-26T06:49:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc05830 0x557f1fc0ea50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc231c0 0x557f1fc255e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc0f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1571787009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0ad30 0x557f1fc0ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 >> 172.21.15.89:0/1571787009 conn(0x557f1fb68720 msgr2=0x557f1fbf32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbdf2ffd700 0 client.15705 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 shutdown_connections 2023-10-26T06:49:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.704+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1571787009 wait complete. 2023-10-26T06:49:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.705+0000 7fbe2d9ddb80 20 client.15705 trim_cache size 0 max 16384 2023-10-26T06:49:02.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.719+0000 7fbe2d9ddb80 1 Processor -- start 2023-10-26T06:49:02.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.719+0000 7fbe2d9ddb80 1 -- start start 2023-10-26T06:49:02.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0ad30 0x557f1fbe9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0b650 0x557f1fc08ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc112b0 0x557f1fc0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1f9c7920 con 0x557f1fc112b0 2023-10-26T06:49:02.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fbef900 con 0x557f1fc0b650 2023-10-26T06:49:02.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fc10070 con 0x557f1fc0ad30 2023-10-26T06:49:02.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe137fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0ad30 0x557f1fbe9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.720+0000 7fbe13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc112b0 0x557f1fc0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.721+0000 7fbe13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc112b0 0x557f1fc0f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55144/0 (socket says 172.21.15.89:55144) 2023-10-26T06:49:02.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.721+0000 7fbe13fff700 1 -- 172.21.15.89:0/1720906155 learned_addr learned my addr 172.21.15.89:0/1720906155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:02.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.721+0000 7fbe13fff700 1 -- 172.21.15.89:0/1720906155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0ad30 msgr2=0x557f1fbe9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.721+0000 7fbe13fff700 1 --2- 172.21.15.89:0/1720906155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0ad30 0x557f1fbe9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.721+0000 7fbe13fff700 1 -- 172.21.15.89:0/1720906155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0b650 msgr2=0x557f1fc08ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.721+0000 7fbe13fff700 1 --2- 172.21.15.89:0/1720906155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0b650 0x557f1fc08ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.721+0000 7fbe13fff700 1 -- 172.21.15.89:0/1720906155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f1f95ac60 con 0x557f1fc112b0 2023-10-26T06:49:02.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.722+0000 7fbe13fff700 1 --2- 172.21.15.89:0/1720906155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc112b0 0x557f1fc0f320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00a670 tx=0x7fbe0c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3a162fac2973f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:02.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.722+0000 7fbdfa7fc700 1 -- 172.21.15.89:0/1720906155 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c017610 con 0x557f1fc112b0 2023-10-26T06:49:02.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.722+0000 7fbdfa7fc700 1 -- 172.21.15.89:0/1720906155 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe0c012050 con 0x557f1fc112b0 2023-10-26T06:49:02.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.722+0000 7fbdfa7fc700 1 -- 172.21.15.89:0/1720906155 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c016b80 con 0x557f1fc112b0 2023-10-26T06:49:02.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.723+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1720906155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc112b0 msgr2=0x557f1fc0f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.723+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1720906155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc112b0 0x557f1fc0f320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00a670 tx=0x7fbe0c0098f0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.723+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1720906155 shutdown_connections 2023-10-26T06:49:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.723+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1720906155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc112b0 0x557f1fc0f320 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.723+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1720906155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0ad30 0x557f1fbe9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.723+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/1720906155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc0b650 0x557f1fc08ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.723+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1720906155 >> 172.21.15.89:0/1720906155 conn(0x557f1fb68720 msgr2=0x557f1fc088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.724+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1720906155 shutdown_connections 2023-10-26T06:49:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.724+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/1720906155 wait complete. 2023-10-26T06:49:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.724+0000 7fbe2d9ddb80 1 -- start start 2023-10-26T06:49:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 1 Processor -- start 2023-10-26T06:49:02.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc0ad30 0x557f1fc6f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc6f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc112b0 0x557f1fc73580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fc73ac0 con 0x557f1fc0ad30 2023-10-26T06:49:02.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fc73c00 con 0x557f1fc112b0 2023-10-26T06:49:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1fc73dc0 con 0x557f1fc0b650 2023-10-26T06:49:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe2d9ddb80 10 client.0 _ll_register_callbacks cb 0x557f1fa2da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe137fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc0ad30 0x557f1fc6f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe137fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc0ad30 0x557f1fc6f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55156/0 (socket says 172.21.15.89:55156) 2023-10-26T06:49:02.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe137fe700 1 -- 172.21.15.89:0/820577044 learned_addr learned my addr 172.21.15.89:0/820577044 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.725+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe13fff700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc112b0 0x557f1fc73580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe137fe700 1 -- 172.21.15.89:0/820577044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 msgr2=0x557f1fc6f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe137fe700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc6f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe137fe700 1 -- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc112b0 msgr2=0x557f1fc73580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe137fe700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc112b0 0x557f1fc73580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe137fe700 1 -- 172.21.15.89:0/820577044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c009e00 con 0x557f1fc0ad30 2023-10-26T06:49:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc6f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:49:02.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe13fff700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc112b0 0x557f1fc73580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:02.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbe137fe700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc0ad30 0x557f1fc6f150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400b520 tx=0x7fbe040b9dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd5be67cc3a5b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:02.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbdf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:02.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/820577044 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0409f880 con 0x557f1fc0ad30 2023-10-26T06:49:02.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.726+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/820577044 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbe040a3a30 con 0x557f1fc0ad30 2023-10-26T06:49:02.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe080044f0 con 0x557f1fc0ad30 2023-10-26T06:49:02.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/820577044 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe0409b990 con 0x557f1fc0ad30 2023-10-26T06:49:02.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 10 client.15711 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:02.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557f1f998250 con 0x557f1fc0ad30 2023-10-26T06:49:02.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/820577044 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe080044f0 con 0x557f1fc0ad30 2023-10-26T06:49:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/820577044 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbe040216c0 con 0x557f1fc0ad30 2023-10-26T06:49:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 10 client.15711 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 20 client.15711 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:02.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc18cc0 0x557f1fc190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 4 client.15711 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:49:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.727+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f1fb3fe50 con 0x557f1fc18cc0 2023-10-26T06:49:02.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.728+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc18cc0 0x557f1fc190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:02.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.729+0000 7fbe12ffd700 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc18cc0 0x557f1fc190d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c088dcddb306e8 server_cookie=20a517996accc202 in_seq=0 out_seq=0 2023-10-26T06:49:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.729+0000 7fbdf2ffd700 10 client.15711 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbdf2ffd700 1 -- 172.21.15.89:0/820577044 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f1fb3fe50 con 0x557f1fc18cc0 2023-10-26T06:49:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbdf2ffd700 10 client.15711 handle_command_reply: tid=0 2023-10-26T06:49:02.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 client.15711 shutdown 2023-10-26T06:49:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc0ad30 msgr2=0x557f1fc6f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc0ad30 0x557f1fc6f150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400b520 tx=0x7fbe040b9dd0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 shutdown_connections 2023-10-26T06:49:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f1fc0ad30 0x557f1fc6f150 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557f1fc18cc0 0x557f1fc190d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbdf2ffd700 0 client.15711 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:02.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f1fc112b0 0x557f1fc73580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 --2- 172.21.15.89:0/820577044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1fc0b650 0x557f1fc6f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 >> 172.21.15.89:0/820577044 conn(0x557f1fb68720 msgr2=0x557f1fbe8700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:02.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 shutdown_connections 2023-10-26T06:49:02.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 1 -- 172.21.15.89:0/820577044 wait complete. 2023-10-26T06:49:02.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.730+0000 7fbe2d9ddb80 20 client.15711 trim_cache size 0 max 16384 2023-10-26T06:49:02.746 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:49:02.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbdfc0a1670 msgr2=0x7fbdfc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbdfc0a1670 0x7fbdfc0a3b30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe14197b60 tx=0x7fbe080116c0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe14101790 msgr2=0x7fbe1419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:02.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe14101790 0x7fbe1419c870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400b2e0 tx=0x7fbe04005f00 comp rx=0 tx=0).stop 2023-10-26T06:49:02.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 shutdown_connections 2023-10-26T06:49:02.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbdfc0a1670 0x7fbdfc0a3b30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbe1410abb0 0x7fbe141969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe14101790 0x7fbe1419c870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 --2- 172.21.15.89:0/3388625048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe141020b0 0x7fbe1419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:02.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 >> 172.21.15.89:0/3388625048 conn(0x7fbe140fafb0 msgr2=0x7fbe140fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:02.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 shutdown_connections 2023-10-26T06:49:02.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:02.731+0000 7fbe18d9d700 1 -- 172.21.15.89:0/3388625048 wait complete. 2023-10-26T06:49:03.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:03 smithi089 ceph-mon[95031]: pgmap v374: 97 pgs: 97 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 276 MiB/s wr, 87 op/s 2023-10-26T06:49:03.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:03 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/55825651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:49:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:03 smithi192 ceph-mon[131661]: pgmap v374: 97 pgs: 97 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 276 MiB/s wr, 87 op/s 2023-10-26T06:49:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:03 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/55825651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:49:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:03 smithi135 ceph-mon[131403]: pgmap v374: 97 pgs: 97 active+clean; 11 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 276 MiB/s wr, 87 op/s 2023-10-26T06:49:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:03 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/55825651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:49:03.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:49:03.759 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:49:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1589716345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae4102930 msgr2=0x7f7ae4102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1589716345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae4102930 0x7f7ae4102db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009990 tx=0x7f7acc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1589716345 shutdown_connections 2023-10-26T06:49:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1589716345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae41032f0 0x7f7ae410f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1589716345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ae4108940 0x7f7ae4108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1589716345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae4102930 0x7f7ae4102db0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1589716345 >> 172.21.15.89:0/1589716345 conn(0x7f7ae40fe3e0 msgr2=0x7f7ae4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1589716345 shutdown_connections 2023-10-26T06:49:04.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.037+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1589716345 wait complete. 2023-10-26T06:49:04.042 INFO:tasks.workunit.client.0.smithi089.stdout:Syncing()...7 sec 2023-10-26T06:49:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.038+0000 7f7ae99ff700 1 Processor -- start 2023-10-26T06:49:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.038+0000 7f7ae99ff700 1 -- start start 2023-10-26T06:49:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae99ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae4102930 0x7f7ae4198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae99ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae41032f0 0x7f7ae41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae99ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ae4108940 0x7f7ae419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae99ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae419dcb0 con 0x7f7ae41032f0 2023-10-26T06:49:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae99ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae419de20 con 0x7f7ae4102930 2023-10-26T06:49:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae99ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae419dfe0 con 0x7f7ae4108940 2023-10-26T06:49:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae41032f0 0x7f7ae41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae41032f0 0x7f7ae41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55182/0 (socket says 172.21.15.89:55182) 2023-10-26T06:49:04.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7adbfff700 1 -- 172.21.15.89:0/1087447881 learned_addr learned my addr 172.21.15.89:0/1087447881 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae4102930 0x7f7ae4198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.039+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ae4108940 0x7f7ae419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7adbfff700 1 -- 172.21.15.89:0/1087447881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ae4108940 msgr2=0x7f7ae419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7adbfff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ae4108940 0x7f7ae419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7adbfff700 1 -- 172.21.15.89:0/1087447881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae4102930 msgr2=0x7f7ae4198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7adbfff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae4102930 0x7f7ae4198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7adbfff700 1 -- 172.21.15.89:0/1087447881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc0095f0 con 0x7f7ae41032f0 2023-10-26T06:49:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7adbfff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae41032f0 0x7f7ae41992d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7acc005910 tx=0x7f7acc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7ae89fd700 1 -- 172.21.15.89:0/1087447881 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc01d070 con 0x7f7ae41032f0 2023-10-26T06:49:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7ae89fd700 1 -- 172.21.15.89:0/1087447881 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7acc022470 con 0x7f7ae41032f0 2023-10-26T06:49:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae419e260 con 0x7f7ae41032f0 2023-10-26T06:49:04.048 INFO:tasks.workunit.client.0.smithi089.stdout:Starting Actual Benchmark At: Thu Oct 26 06:49:04 2023 2023-10-26T06:49:04.048 INFO:tasks.workunit.client.0.smithi089.stdout: 2023-10-26T06:49:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41a5b00 con 0x7f7ae41032f0 2023-10-26T06:49:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.040+0000 7f7ae89fd700 1 -- 172.21.15.89:0/1087447881 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc017650 con 0x7f7ae41032f0 2023-10-26T06:49:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.044+0000 7f7ae89fd700 1 -- 172.21.15.89:0/1087447881 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7acc0177b0 con 0x7f7ae41032f0 2023-10-26T06:49:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7ae89fd700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7ad00a1670 0x7f7ad00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 Processor -- start 2023-10-26T06:49:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 -- start start 2023-10-26T06:49:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7ad00a1670 0x7f7ad00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6a9d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a97110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7ae89fd700 1 -- 172.21.15.89:0/1087447881 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7acc0ce100 con 0x7f7ae41032f0 2023-10-26T06:49:04.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6a97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6a0a760 con 0x555aa6a9d580 2023-10-26T06:49:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa69f0ce0 con 0x555aa6a97650 2023-10-26T06:49:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.046+0000 7f7afd377b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa69ee600 con 0x555aa6a9cc60 2023-10-26T06:49:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.047+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a97110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.047+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a97110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55184/0 (socket says 172.21.15.89:55184) 2023-10-26T06:49:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.047+0000 7f7adbfff700 1 -- 172.21.15.89:0/1577871417 learned_addr learned my addr 172.21.15.89:0/1577871417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.047+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6a97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6a9d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7ad00a1670 0x7f7ad00a3b30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009c30 tx=0x7f7ad4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7adbfff700 1 -- 172.21.15.89:0/1577871417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 msgr2=0x555aa6a9d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7adbfff700 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6a9d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7adbfff700 1 -- 172.21.15.89:0/1577871417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 msgr2=0x555aa6a97ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7adbfff700 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6a97ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7adbfff700 1 -- 172.21.15.89:0/1577871417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa69a7470 con 0x555aa6a9d580 2023-10-26T06:49:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6a9d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7adbfff700 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a97110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0987c0 tx=0x7f7acc09c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579e7a4d6c666360 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:04.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7ad9ffb700 1 -- 172.21.15.89:0/1577871417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc09c9c0 con 0x555aa6a9d580 2023-10-26T06:49:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7ad9ffb700 1 -- 172.21.15.89:0/1577871417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7acc09c2c0 con 0x555aa6a9d580 2023-10-26T06:49:04.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.048+0000 7f7ad9ffb700 1 -- 172.21.15.89:0/1577871417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc09c780 con 0x555aa6a9d580 2023-10-26T06:49:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.049+0000 7f7afd377b80 1 -- 172.21.15.89:0/1577871417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 msgr2=0x555aa6a97110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.049+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a97110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0987c0 tx=0x7f7acc09c160 comp rx=0 tx=0).stop 2023-10-26T06:49:04.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.049+0000 7f7afd377b80 1 -- 172.21.15.89:0/1577871417 shutdown_connections 2023-10-26T06:49:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.049+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6a97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.049+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6a9d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.049+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1577871417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a97110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.049+0000 7f7afd377b80 1 -- 172.21.15.89:0/1577871417 >> 172.21.15.89:0/1577871417 conn(0x555aa6a0b7a0 msgr2=0x555aa6a91be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:04.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.050+0000 7f7afd377b80 1 -- 172.21.15.89:0/1577871417 shutdown_connections 2023-10-26T06:49:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.050+0000 7f7afd377b80 1 -- 172.21.15.89:0/1577871417 wait complete. 2023-10-26T06:49:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.050+0000 7f7afd377b80 1 -- start start 2023-10-26T06:49:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.051+0000 7f7afd377b80 1 Processor -- start 2023-10-26T06:49:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.051+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6aa3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.051+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6aa07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.051+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6aa0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.051+0000 7f7afd377b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa69e9ef0 con 0x555aa6a9d580 2023-10-26T06:49:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.051+0000 7f7afd377b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6986790 con 0x555aa6a9cc60 2023-10-26T06:49:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.052+0000 7f7afd377b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6986430 con 0x555aa6a97650 2023-10-26T06:49:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.052+0000 7f7afd377b80 10 client.0 _ll_register_callbacks cb 0x555aa68d0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:04.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.053+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6aa07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.053+0000 7f7ae2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6aa3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.053+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6aa07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52964/0 (socket says 172.21.15.89:52964) 2023-10-26T06:49:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.053+0000 7f7adbfff700 1 -- 172.21.15.89:0/1913379373 learned_addr learned my addr 172.21.15.89:0/1913379373 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.053+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6aa0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.054+0000 7f7ae37fe700 1 -- 172.21.15.89:0/1913379373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 msgr2=0x555aa6aa3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.054+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6aa3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.054+0000 7f7ae37fe700 1 -- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 msgr2=0x555aa6aa07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.054+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6aa07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.054+0000 7f7ae37fe700 1 -- 172.21.15.89:0/1913379373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41a5b00 con 0x555aa6a9d580 2023-10-26T06:49:04.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.054+0000 7f7adbfff700 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6aa07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:49:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.055+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6aa0f00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7adc011e90 tx=0x7f7adc014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f32a904514b81d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.056+0000 7f7ac2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:04.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.056+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/1913379373 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00e040 con 0x555aa6a9d580 2023-10-26T06:49:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.056+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/1913379373 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7adc0093a0 con 0x555aa6a9d580 2023-10-26T06:49:04.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.058+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aa69eef70 con 0x555aa6a9d580 2023-10-26T06:49:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.063+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/1913379373 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7adc004db0 con 0x555aa6a9d580 2023-10-26T06:49:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.063+0000 7f7afd377b80 10 client.15723 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.063+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555aa6814e60 con 0x555aa6a9d580 2023-10-26T06:49:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.064+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/1913379373 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555aa69eef70 con 0x555aa6a9d580 2023-10-26T06:49:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.065+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/1913379373 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7adc01c030 con 0x555aa6a9d580 2023-10-26T06:49:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.065+0000 7f7afd377b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.065+0000 7f7afd377b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.065+0000 7f7afd377b80 20 client.15723 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.065+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6abaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.065+0000 7f7afd377b80 4 client.15723 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.065+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aa6a66930 con 0x555aa6ab8b70 2023-10-26T06:49:04.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.070+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6abaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.086+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6abaf90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da2b138d9b78c153 server_cookie=f8d235a37e3979e in_seq=0 out_seq=0 2023-10-26T06:49:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.086+0000 7f7ac2ffd700 10 client.15723 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.086+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/1913379373 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555aa6a66930 con 0x555aa6ab8b70 2023-10-26T06:49:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.086+0000 7f7ac2ffd700 10 client.15723 handle_command_reply: tid=0 2023-10-26T06:49:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.087+0000 7f7afd377b80 1 client.15723 shutdown 2023-10-26T06:49:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 msgr2=0x555aa6aa0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6aa0f00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7adc011e90 tx=0x7f7adc014600 comp rx=0 tx=0).stop 2023-10-26T06:49:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 shutdown_connections 2023-10-26T06:49:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6aa0f00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6abaf90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6aa3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7ac2ffd700 0 client.15723 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.091+0000 7f7afd377b80 1 --2- 172.21.15.89:0/1913379373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6aa07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.092+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 >> 172.21.15.89:0/1913379373 conn(0x555aa6a0b7a0 msgr2=0x555aa6a96100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.104+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 shutdown_connections 2023-10-26T06:49:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.104+0000 7f7afd377b80 1 -- 172.21.15.89:0/1913379373 wait complete. 2023-10-26T06:49:04.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.122+0000 7f7afd377b80 20 client.15723 trim_cache size 0 max 16384 2023-10-26T06:49:04.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.162+0000 7f7afd377b80 1 Processor -- start 2023-10-26T06:49:04.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.164+0000 7f7afd377b80 1 -- start start 2023-10-26T06:49:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.164+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6abf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.165+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6a9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.165+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a9ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.165+0000 7f7ae2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6abf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.165+0000 7f7ae2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6abf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:37024/0 (socket says 172.21.15.89:37024) 2023-10-26T06:49:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.165+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6a9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.165+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6a9a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52978/0 (socket says 172.21.15.89:52978) 2023-10-26T06:49:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.166+0000 7f7ae37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a9ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.166+0000 7f7ae37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a9ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55194/0 (socket says 172.21.15.89:55194) 2023-10-26T06:49:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.166+0000 7f7ae2ffd700 1 -- 172.21.15.89:0/607557117 learned_addr learned my addr 172.21.15.89:0/607557117 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.166+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa686a9a0 con 0x555aa6a9d580 2023-10-26T06:49:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.166+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6aa0b90 con 0x555aa6a9cc60 2023-10-26T06:49:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.166+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6a980b0 con 0x555aa6a97650 2023-10-26T06:49:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.167+0000 7f7adbfff700 1 -- 172.21.15.89:0/607557117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 msgr2=0x555aa6abf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.167+0000 7f7adbfff700 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6abf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.167+0000 7f7adbfff700 1 -- 172.21.15.89:0/607557117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 msgr2=0x555aa6a9ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.167+0000 7f7adbfff700 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a9ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.167+0000 7f7adbfff700 1 -- 172.21.15.89:0/607557117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa6a91df0 con 0x555aa6a9cc60 2023-10-26T06:49:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.167+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a9ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:49:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.168+0000 7f7adbfff700 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6a9a730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c00 tx=0x7f7acc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f4ed93c663331b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.168+0000 7f7ada7fc700 1 -- 172.21.15.89:0/607557117 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc09c9c0 con 0x555aa6a9cc60 2023-10-26T06:49:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.168+0000 7f7ada7fc700 1 -- 172.21.15.89:0/607557117 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7acc0a45c0 con 0x555aa6a9cc60 2023-10-26T06:49:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.168+0000 7f7ada7fc700 1 -- 172.21.15.89:0/607557117 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc09c780 con 0x555aa6a9cc60 2023-10-26T06:49:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.170+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 msgr2=0x555aa6a9a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.170+0000 7f7afd377b80 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6a9a730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c00 tx=0x7f7acc005fd0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.182+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 shutdown_connections 2023-10-26T06:49:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.182+0000 7f7afd377b80 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6a9ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.182+0000 7f7afd377b80 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a97650 0x555aa6abf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.182+0000 7f7afd377b80 1 --2- 172.21.15.89:0/607557117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a9cc60 0x555aa6a9a730 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.182+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 >> 172.21.15.89:0/607557117 conn(0x555aa6a0b7a0 msgr2=0x555aa6a96100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.182+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 shutdown_connections 2023-10-26T06:49:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.182+0000 7f7afd377b80 1 -- 172.21.15.89:0/607557117 wait complete. 2023-10-26T06:49:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.183+0000 7f7afd377b80 1 -- start start 2023-10-26T06:49:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.184+0000 7f7afd377b80 1 Processor -- start 2023-10-26T06:49:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.186+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6b20fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6b214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7afd377b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6abee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7afd377b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6abf430 con 0x555aa6a9d580 2023-10-26T06:49:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7afd377b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6abf5a0 con 0x555aa6a97650 2023-10-26T06:49:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7afd377b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa6abf760 con 0x555aa6a9cc60 2023-10-26T06:49:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7afd377b80 10 client.0 _ll_register_callbacks cb 0x555aa68d0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6b214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7adbfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6b214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:37040/0 (socket says 172.21.15.89:37040) 2023-10-26T06:49:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7adbfff700 1 -- 172.21.15.89:0/2930550448 learned_addr learned my addr 172.21.15.89:0/2930550448 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6b20fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7adbfff700 1 -- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 msgr2=0x555aa6b20fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7adbfff700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6b20fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.189+0000 7f7adbfff700 1 -- 172.21.15.89:0/2930550448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 msgr2=0x555aa6abee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:49:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.190+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6abee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.190+0000 7f7adbfff700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6abee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.190+0000 7f7adbfff700 1 -- 172.21.15.89:0/2930550448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555aa6814e60 con 0x555aa6a9cc60 2023-10-26T06:49:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.190+0000 7f7ae37fe700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6abee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.191+0000 7f7adbfff700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6b214e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0a4090 tx=0x7f7acc09c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cef26845dd7a131 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.191+0000 7f7ac2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:49:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.191+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/2930550448 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc09c9c0 con 0x555aa6a9cc60 2023-10-26T06:49:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.191+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/2930550448 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7acc0a0410 con 0x555aa6a9cc60 2023-10-26T06:49:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.192+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7acc09c2c0 con 0x555aa6a9cc60 2023-10-26T06:49:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.192+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/2930550448 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7acc0a4340 con 0x555aa6a9cc60 2023-10-26T06:49:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.197+0000 7f7afd377b80 10 client.25714 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.197+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555aa69a7470 con 0x555aa6a9cc60 2023-10-26T06:49:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.197+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/2930550448 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7acc09c2c0 con 0x555aa6a9cc60 2023-10-26T06:49:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.198+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/2930550448 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7acc00fac0 con 0x555aa6a9cc60 2023-10-26T06:49:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.198+0000 7f7afd377b80 10 client.25714 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.198+0000 7f7afd377b80 10 client.25714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.198+0000 7f7afd377b80 20 client.25714 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.198+0000 7f7afd377b80 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6b2a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.198+0000 7f7afd377b80 4 client.25714 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:49:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.198+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555aa69e2ed0 con 0x555aa6ab8b70 2023-10-26T06:49:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.201+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6b2a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.202+0000 7f7ae2ffd700 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6b2a9b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59899408c886c80b server_cookie=43d57100d48737d2 in_seq=0 out_seq=0 2023-10-26T06:49:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.202+0000 7f7ac2ffd700 10 client.25714 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.203+0000 7f7ac2ffd700 1 -- 172.21.15.89:0/2930550448 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555aa69e2ed0 con 0x555aa6ab8b70 2023-10-26T06:49:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.203+0000 7f7ac2ffd700 10 client.25714 handle_command_reply: tid=0 2023-10-26T06:49:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.203+0000 7f7afd377b80 1 client.25714 shutdown 2023-10-26T06:49:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 msgr2=0x555aa6b214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6b214e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0a4090 tx=0x7f7acc09c7e0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 shutdown_connections 2023-10-26T06:49:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555aa6a9d580 0x555aa6abee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555aa6ab8b70 0x555aa6b2a9b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555aa6a97650 0x555aa6b20fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 --2- 172.21.15.89:0/2930550448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555aa6a9cc60 0x555aa6b214e0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0a4090 tx=0x7f7acc09c7e0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7ac2ffd700 0 client.25714 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 >> 172.21.15.89:0/2930550448 conn(0x555aa6a0b7a0 msgr2=0x555aa6a9f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.206+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 shutdown_connections 2023-10-26T06:49:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.209+0000 7f7afd377b80 1 -- 172.21.15.89:0/2930550448 wait complete. 2023-10-26T06:49:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.210+0000 7f7afd377b80 20 client.25714 trim_cache size 0 max 16384 2023-10-26T06:49:04.215 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:49:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:49:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "c0bad5b1-05ca-403c-97f4-41378e9d91fd", 2023-10-26T06:49:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:49:04.216 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:49:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.216+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7ad00a1670 msgr2=0x7f7ad00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.216+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7ad00a1670 0x7f7ad00a3b30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009c30 tx=0x7f7ad4009260 comp rx=0 tx=0).stop 2023-10-26T06:49:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.216+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae41032f0 msgr2=0x7f7ae41992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.216+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae41032f0 0x7f7ae41992d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7acc005910 tx=0x7f7acc004730 comp rx=0 tx=0).stop 2023-10-26T06:49:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.217+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 shutdown_connections 2023-10-26T06:49:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.217+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7ad00a1670 0x7f7ad00a3b30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.217+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ae4108940 0x7f7ae419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.217+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae4102930 0x7f7ae4198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.218+0000 7f7ae99ff700 1 --2- 172.21.15.89:0/1087447881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ae41032f0 0x7f7ae41992d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.218+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 >> 172.21.15.89:0/1087447881 conn(0x7f7ae40fe3e0 msgr2=0x7f7ae40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.228+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 shutdown_connections 2023-10-26T06:49:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:04.228+0000 7f7ae99ff700 1 -- 172.21.15.89:0/1087447881 wait complete. 2023-10-26T06:49:04.392 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:49:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.364+0000 7f45d79ff700 1 -- 172.21.15.89:0/2463557410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d0119fd0 msgr2=0x7f45d011c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.364+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2463557410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d0119fd0 0x7f45d011c3c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f45c800b3e0 tx=0x7f45c800b6f0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.366+0000 7f45d79ff700 1 -- 172.21.15.89:0/2463557410 shutdown_connections 2023-10-26T06:49:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.366+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2463557410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d0119fd0 0x7f45d011c3c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.366+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2463557410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45d010ef10 0x7f45d01077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.366+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2463557410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d010f380 0x7f45d0107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.366+0000 7f45d79ff700 1 -- 172.21.15.89:0/2463557410 >> 172.21.15.89:0/2463557410 conn(0x7f45d006b840 msgr2=0x7f45d006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.367+0000 7f45d79ff700 1 -- 172.21.15.89:0/2463557410 shutdown_connections 2023-10-26T06:49:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.367+0000 7f45d79ff700 1 -- 172.21.15.89:0/2463557410 wait complete. 2023-10-26T06:49:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.368+0000 7f45d79ff700 1 Processor -- start 2023-10-26T06:49:05.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.370+0000 7f45d79ff700 1 -- start start 2023-10-26T06:49:05.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.373+0000 7f45d79ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d010ef10 0x7f45d01ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.373+0000 7f45d79ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d010f380 0x7f45d01abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.374+0000 7f45d4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d010f380 0x7f45d01abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.374+0000 7f45d4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d010f380 0x7f45d01abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55224/0 (socket says 172.21.15.89:55224) 2023-10-26T06:49:05.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.374+0000 7f45d4f9a700 1 -- 172.21.15.89:0/2041262809 learned_addr learned my addr 172.21.15.89:0/2041262809 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:05.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.374+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45d0119fd0 0x7f45d01a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.374+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01a6060 con 0x7f45d010f380 2023-10-26T06:49:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.374+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01a61d0 con 0x7f45d0119fd0 2023-10-26T06:49:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.374+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d01a6390 con 0x7f45d010ef10 2023-10-26T06:49:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.375+0000 7f45d4f9a700 1 -- 172.21.15.89:0/2041262809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d010ef10 msgr2=0x7f45d01ab8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T06:49:05.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.375+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d010ef10 0x7f45d01ab8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.375+0000 7f45d4f9a700 1 -- 172.21.15.89:0/2041262809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45d0119fd0 msgr2=0x7f45d01a5950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T06:49:05.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.375+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45d0119fd0 0x7f45d01a5950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.375+0000 7f45d4f9a700 1 -- 172.21.15.89:0/2041262809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c8009c40 con 0x7f45d010f380 2023-10-26T06:49:05.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.376+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d010f380 0x7f45d01abdf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b750 tx=0x7f45c000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.379+0000 7f45c67fc700 1 -- 172.21.15.89:0/2041262809 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000d660 con 0x7f45d010f380 2023-10-26T06:49:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.379+0000 7f45c67fc700 1 -- 172.21.15.89:0/2041262809 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c000dc80 con 0x7f45d010f380 2023-10-26T06:49:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.379+0000 7f45c67fc700 1 -- 172.21.15.89:0/2041262809 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c000edd0 con 0x7f45d010f380 2023-10-26T06:49:05.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.379+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d01a6670 con 0x7f45d010f380 2023-10-26T06:49:05.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.392+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d010f990 con 0x7f45d010f380 2023-10-26T06:49:05.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.423+0000 7f45c67fc700 1 -- 172.21.15.89:0/2041262809 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f45c000c390 con 0x7f45d010f380 2023-10-26T06:49:05.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.433+0000 7f45c67fc700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45bc0a1800 0x7f45bc0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.438+0000 7f45eb357b80 1 Processor -- start 2023-10-26T06:49:05.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.439+0000 7f45c67fc700 1 -- 172.21.15.89:0/2041262809 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f45c00cf170 con 0x7f45d010f380 2023-10-26T06:49:05.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.439+0000 7f45eb357b80 1 -- start start 2023-10-26T06:49:05.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.439+0000 7f45d579b700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45bc0a1800 0x7f45bc0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.466+0000 7f45d579b700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45bc0a1800 0x7f45bc0a3cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005fd0 tx=0x7f45cc009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:05.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.476+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f26478fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.476+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26479500 0x555f26473090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.476+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f26473a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.476+0000 7f45eb357b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f263e66e0 con 0x555f264735d0 2023-10-26T06:49:05.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.476+0000 7f45eb357b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f263ccc60 con 0x555f26479500 2023-10-26T06:49:05.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.476+0000 7f45eb357b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f263ca580 con 0x555f26478be0 2023-10-26T06:49:05.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.478+0000 7f45d5f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f26473a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.478+0000 7f45d5f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f26473a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55232/0 (socket says 172.21.15.89:55232) 2023-10-26T06:49:05.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.478+0000 7f45d5f9c700 1 -- 172.21.15.89:0/186526066 learned_addr learned my addr 172.21.15.89:0/186526066 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:05.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.486+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26479500 0x555f26473090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.488+0000 7f45d5f9c700 1 -- 172.21.15.89:0/186526066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 msgr2=0x555f26478fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.488+0000 7f45d5f9c700 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f26478fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.488+0000 7f45d5f9c700 1 -- 172.21.15.89:0/186526066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26479500 msgr2=0x555f26473090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.488+0000 7f45d5f9c700 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26479500 0x555f26473090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.488+0000 7f45d5f9c700 1 -- 172.21.15.89:0/186526066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f263833f0 con 0x555f264735d0 2023-10-26T06:49:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.489+0000 7f45d5f9c700 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f26473a50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009100 tx=0x7f45c8009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a7d64b472ef64d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.491+0000 7f45bb7fe700 1 -- 172.21.15.89:0/186526066 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c800e040 con 0x555f264735d0 2023-10-26T06:49:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.491+0000 7f45bb7fe700 1 -- 172.21.15.89:0/186526066 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c8009390 con 0x555f264735d0 2023-10-26T06:49:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.491+0000 7f45bb7fe700 1 -- 172.21.15.89:0/186526066 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c801b460 con 0x555f264735d0 2023-10-26T06:49:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.492+0000 7f45eb357b80 1 -- 172.21.15.89:0/186526066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 msgr2=0x555f26473a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.492+0000 7f45eb357b80 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f26473a50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009100 tx=0x7f45c8009130 comp rx=0 tx=0).stop 2023-10-26T06:49:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.497+0000 7f45eb357b80 1 -- 172.21.15.89:0/186526066 shutdown_connections 2023-10-26T06:49:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.497+0000 7f45eb357b80 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f26473a50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.497+0000 7f45eb357b80 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f26478fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.497+0000 7f45eb357b80 1 --2- 172.21.15.89:0/186526066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26479500 0x555f26473090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.498+0000 7f45eb357b80 1 -- 172.21.15.89:0/186526066 >> 172.21.15.89:0/186526066 conn(0x555f263e7720 msgr2=0x555f2646db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.498+0000 7f45eb357b80 1 -- 172.21.15.89:0/186526066 shutdown_connections 2023-10-26T06:49:05.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.498+0000 7f45eb357b80 1 -- 172.21.15.89:0/186526066 wait complete. 2023-10-26T06:49:05.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.505+0000 7f45eb357b80 1 -- start start 2023-10-26T06:49:05.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.509+0000 7f45eb357b80 1 Processor -- start 2023-10-26T06:49:05.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.515+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f264735d0 0x555f2647f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.516+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f2647c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.517+0000 7f45d579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f264735d0 0x555f2647f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.519+0000 7f45d579b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f264735d0 0x555f2647f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:37076/0 (socket says 172.21.15.89:37076) 2023-10-26T06:49:05.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.519+0000 7f45d579b700 1 -- 172.21.15.89:0/3632632924 learned_addr learned my addr 172.21.15.89:0/3632632924 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:05.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.519+0000 7f45d4f9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f2647c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.520+0000 7f45eb357b80 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f2647ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.523+0000 7f45d5f9c700 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f2647ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.524+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f263c5e70 con 0x555f26479500 2023-10-26T06:49:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.524+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f26362710 con 0x555f26478be0 2023-10-26T06:49:05.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.525+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f263623b0 con 0x555f264735d0 2023-10-26T06:49:05.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.528+0000 7f45d5f9c700 1 -- 172.21.15.89:0/3632632924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f264735d0 msgr2=0x555f2647f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:05.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.528+0000 7f45d5f9c700 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f264735d0 0x555f2647f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.528+0000 7f45d5f9c700 1 -- 172.21.15.89:0/3632632924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 msgr2=0x555f2647c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:05.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.528+0000 7f45d5f9c700 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f2647c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.528+0000 7f45d5f9c700 1 -- 172.21.15.89:0/3632632924 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d011cd40 con 0x555f26479500 2023-10-26T06:49:05.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.534+0000 7f45eb357b80 10 client.0 _ll_register_callbacks cb 0x555f262aca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:05.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.536+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f263caef0 con 0x555f26479500 2023-10-26T06:49:05.546 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:05 smithi089 ceph-mon[95031]: pgmap v375: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 319 MiB/s wr, 98 op/s 2023-10-26T06:49:05.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:05 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:49:05.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:05 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:05.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:05 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:49:05.547 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:05 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:05.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.545+0000 7f45d5f9c700 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f2647ccc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f45c80091a0 tx=0x7f45c80076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4999116c9d9492 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:05.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.549+0000 7f45b8ff9700 10 client.15738 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:05.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.552+0000 7f45b8ff9700 1 -- 172.21.15.89:0/3632632924 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8003e10 con 0x555f26479500 2023-10-26T06:49:05.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.552+0000 7f45b8ff9700 1 -- 172.21.15.89:0/3632632924 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f45c800be40 con 0x555f26479500 2023-10-26T06:49:05.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.554+0000 7f45b8ff9700 1 -- 172.21.15.89:0/3632632924 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45c8003e10 con 0x555f26479500 2023-10-26T06:49:05.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.554+0000 7f45eb357b80 10 client.15738 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:05.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.554+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=20}) v3 -- 0x555f261f0de0 con 0x555f26479500 2023-10-26T06:49:05.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.573+0000 7f45b8ff9700 1 -- 172.21.15.89:0/3632632924 <== mon.0 v2:172.21.15.89:3300/0 4 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f45c8019040 con 0x555f26479500 2023-10-26T06:49:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.573+0000 7f45eb357b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.573+0000 7f45eb357b80 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.573+0000 7f45eb357b80 20 client.15738 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.573+0000 7f45eb357b80 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f2648eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.573+0000 7f45eb357b80 4 client.15738 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.573+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f264428b0 con 0x555f2648c710 2023-10-26T06:49:05.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.578+0000 7f45d579b700 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f2648eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:05.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.585+0000 7f45d579b700 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f2648eb30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dd5aa0e160026d server_cookie=93e5649da2ce1146 in_seq=0 out_seq=0 2023-10-26T06:49:05.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.586+0000 7f45b8ff9700 10 client.15738 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:05.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.588+0000 7f45b8ff9700 1 -- 172.21.15.89:0/3632632924 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555f264428b0 con 0x555f2648c710 2023-10-26T06:49:05.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.588+0000 7f45b8ff9700 10 client.15738 handle_command_reply: tid=0 2023-10-26T06:49:05.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.588+0000 7f45eb357b80 1 client.15738 shutdown 2023-10-26T06:49:05.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.599+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 msgr2=0x555f2647ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:05.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.599+0000 7f45eb357b80 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f2647ccc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f45c80091a0 tx=0x7f45c80076c0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.600+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 shutdown_connections 2023-10-26T06:49:05.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.600+0000 7f45eb357b80 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f2647ccc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.600+0000 7f45eb357b80 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f2648eb30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.600+0000 7f45b8ff9700 0 client.15738 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.600+0000 7f45eb357b80 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f264735d0 0x555f2647f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.600+0000 7f45eb357b80 1 --2- 172.21.15.89:0/3632632924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f2647c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:05.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.600+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 >> 172.21.15.89:0/3632632924 conn(0x555f263e7720 msgr2=0x555f26472080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:05.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.607+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 shutdown_connections 2023-10-26T06:49:05.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.607+0000 7f45eb357b80 1 -- 172.21.15.89:0/3632632924 wait complete. 2023-10-26T06:49:05.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.607+0000 7f45eb357b80 20 client.15738 trim_cache size 0 max 16384 2023-10-26T06:49:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:05 smithi192 ceph-mon[131661]: pgmap v375: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 319 MiB/s wr, 98 op/s 2023-10-26T06:49:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:05 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:49:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:05 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:05 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:49:05.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:05 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:05 smithi135 ceph-mon[131403]: pgmap v375: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 319 MiB/s wr, 98 op/s 2023-10-26T06:49:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:05 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:49:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:05 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:05 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:49:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:05 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:05.980+0000 7f45eb357b80 1 Processor -- start 2023-10-26T06:49:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.005+0000 7f45eb357b80 1 -- start start 2023-10-26T06:49:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.009+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f264735d0 0x555f26483eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.009+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f2646d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.009+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f26488600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:06.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.029+0000 7f45eb357b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f26468af0 con 0x555f26479500 2023-10-26T06:49:06.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.029+0000 7f45eb357b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f26246920 con 0x555f264735d0 2023-10-26T06:49:06.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.029+0000 7f45eb357b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f26474030 con 0x555f26478be0 2023-10-26T06:49:06.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.029+0000 7f45d579b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f264735d0 0x555f26483eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:06.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.032+0000 7f45d4f9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f2646d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:06.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.032+0000 7f45d4f9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f2646d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:37092/0 (socket says 172.21.15.89:37092) 2023-10-26T06:49:06.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.032+0000 7f45d4f9a700 1 -- 172.21.15.89:0/1955204574 learned_addr learned my addr 172.21.15.89:0/1955204574 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.040+0000 7f45d4f9a700 1 -- 172.21.15.89:0/1955204574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f264735d0 msgr2=0x555f26483eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.040+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/1955204574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f264735d0 0x555f26483eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.040+0000 7f45d4f9a700 1 -- 172.21.15.89:0/1955204574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 msgr2=0x555f26488600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:06.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.040+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/1955204574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f26488600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.040+0000 7f45d4f9a700 1 -- 172.21.15.89:0/1955204574 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f26493760 con 0x555f26478be0 2023-10-26T06:49:06.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.043+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/1955204574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f2646d810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f45c000bf10 tx=0x7f45c0006e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e93c229733d55c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:06.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.060+0000 7f45bbfff700 1 -- 172.21.15.89:0/1955204574 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0099bd0 con 0x555f26478be0 2023-10-26T06:49:06.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.060+0000 7f45bbfff700 1 -- 172.21.15.89:0/1955204574 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c009dd20 con 0x555f26478be0 2023-10-26T06:49:06.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.060+0000 7f45bbfff700 1 -- 172.21.15.89:0/1955204574 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0099e30 con 0x555f26478be0 2023-10-26T06:49:06.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.067+0000 7f45eb357b80 1 -- 172.21.15.89:0/1955204574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 msgr2=0x555f2646d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:06.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.067+0000 7f45eb357b80 1 --2- 172.21.15.89:0/1955204574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f2646d810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f45c000bf10 tx=0x7f45c0006e80 comp rx=0 tx=0).stop 2023-10-26T06:49:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.086+0000 7f45eb357b80 1 -- 172.21.15.89:0/1955204574 shutdown_connections 2023-10-26T06:49:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.086+0000 7f45eb357b80 1 --2- 172.21.15.89:0/1955204574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f26479500 0x555f26488600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.086+0000 7f45eb357b80 1 --2- 172.21.15.89:0/1955204574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f264735d0 0x555f26483eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.086+0000 7f45eb357b80 1 --2- 172.21.15.89:0/1955204574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26478be0 0x555f2646d810 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.086+0000 7f45eb357b80 1 -- 172.21.15.89:0/1955204574 >> 172.21.15.89:0/1955204574 conn(0x555f263e7720 msgr2=0x555f264729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:06.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.097+0000 7f45eb357b80 1 -- 172.21.15.89:0/1955204574 shutdown_connections 2023-10-26T06:49:06.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.100+0000 7f45eb357b80 1 -- 172.21.15.89:0/1955204574 wait complete. 2023-10-26T06:49:06.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.104+0000 7f45eb357b80 1 -- start start 2023-10-26T06:49:06.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.105+0000 7f45eb357b80 1 Processor -- start 2023-10-26T06:49:06.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.106+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f264d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:06.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.106+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f264d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:06.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.108+0000 7f45d579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f264d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:06.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.108+0000 7f45d579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f264d8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55254/0 (socket says 172.21.15.89:55254) 2023-10-26T06:49:06.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.108+0000 7f45eb357b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26479500 0x555f264d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:06.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.108+0000 7f45d579b700 1 -- 172.21.15.89:0/2162249450 learned_addr learned my addr 172.21.15.89:0/2162249450 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:06.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.108+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f264d9bd0 con 0x555f264735d0 2023-10-26T06:49:06.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.108+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f264d9d40 con 0x555f26478be0 2023-10-26T06:49:06.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.108+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f264d9f00 con 0x555f26479500 2023-10-26T06:49:06.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.109+0000 7f45eb357b80 10 client.0 _ll_register_callbacks cb 0x555f262aca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:06.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.109+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f264d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:06.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.110+0000 7f45d579b700 1 -- 172.21.15.89:0/2162249450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26479500 msgr2=0x555f264d9600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:49:06.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.110+0000 7f45d579b700 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26479500 0x555f264d9600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.110+0000 7f45d579b700 1 -- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 msgr2=0x555f264d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:06.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.110+0000 7f45d579b700 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f264d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.110+0000 7f45d579b700 1 -- 172.21.15.89:0/2162249450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d010f990 con 0x555f264735d0 2023-10-26T06:49:06.114 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.111+0000 7f45d579b700 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f264d8840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x555f2648f410 tx=0x7f45cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eb0d70a9d52959 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:06.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.113+0000 7f45b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:06.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.120+0000 7f45b8ff9700 1 -- 172.21.15.89:0/2162249450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc022040 con 0x555f264735d0 2023-10-26T06:49:06.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.120+0000 7f45b8ff9700 1 -- 172.21.15.89:0/2162249450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f45cc025a80 con 0x555f264735d0 2023-10-26T06:49:06.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.130+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45c8009390 con 0x555f264735d0 2023-10-26T06:49:06.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.131+0000 7f45b8ff9700 1 -- 172.21.15.89:0/2162249450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45cc025c40 con 0x555f264735d0 2023-10-26T06:49:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.133+0000 7f45eb357b80 10 client.15741 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:06.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.133+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555f261d9c60 con 0x555f264735d0 2023-10-26T06:49:06.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45b8ff9700 1 -- 172.21.15.89:0/2162249450 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c8009390 con 0x555f264735d0 2023-10-26T06:49:06.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45b8ff9700 1 -- 172.21.15.89:0/2162249450 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f45cc0052d0 con 0x555f264735d0 2023-10-26T06:49:06.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45eb357b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:06.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45eb357b80 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45eb357b80 20 client.15741 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45eb357b80 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f264e3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45eb357b80 4 client.15741 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:49:06.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.135+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f263bee50 con 0x555f2648c710 2023-10-26T06:49:06.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.149+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f264e3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.155+0000 7f45d4f9a700 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f264e3870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871d0e3c6cb16920 server_cookie=eb088009c9fda277 in_seq=0 out_seq=0 2023-10-26T06:49:06.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.160+0000 7f45b8ff9700 10 client.15741 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.162+0000 7f45b8ff9700 1 -- 172.21.15.89:0/2162249450 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555f263bee50 con 0x555f2648c710 2023-10-26T06:49:06.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.162+0000 7f45b8ff9700 10 client.15741 handle_command_reply: tid=0 2023-10-26T06:49:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.162+0000 7f45eb357b80 1 client.15741 shutdown 2023-10-26T06:49:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.173+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 msgr2=0x555f264d8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:06.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.173+0000 7f45eb357b80 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f264d8840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x555f2648f410 tx=0x7f45cc0047c0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.174+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 shutdown_connections 2023-10-26T06:49:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.174+0000 7f45eb357b80 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555f264735d0 0x555f264d8840 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.174+0000 7f45eb357b80 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555f2648c710 0x555f264e3870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.174+0000 7f45eb357b80 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555f26479500 0x555f264d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.174+0000 7f45eb357b80 1 --2- 172.21.15.89:0/2162249450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f26478be0 0x555f264d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.174+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 >> 172.21.15.89:0/2162249450 conn(0x555f263e7720 msgr2=0x555f264766b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.174+0000 7f45b8ff9700 0 client.15741 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.188+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 shutdown_connections 2023-10-26T06:49:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.189+0000 7f45eb357b80 1 -- 172.21.15.89:0/2162249450 wait complete. 2023-10-26T06:49:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.191+0000 7f45eb357b80 20 client.15741 trim_cache size 0 max 16384 2023-10-26T06:49:06.195 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:49:06.195 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:49:06.195 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:49:06.196 INFO:teuthology.orchestra.run.smithi089.stdout: "c0bad5b1-05ca-403c-97f4-41378e9d91fd": { 2023-10-26T06:49:06.196 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:49:06.196 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "c0bad5b1-05ca-403c-97f4-41378e9d91fd", 2023-10-26T06:49:06.196 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:49:06.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:49:06.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:49:06.197 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:49:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.195+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45bc0a1800 msgr2=0x7f45bc0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.195+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45bc0a1800 0x7f45bc0a3cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005fd0 tx=0x7f45cc009310 comp rx=0 tx=0).stop 2023-10-26T06:49:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.196+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d010f380 msgr2=0x7f45d01abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.196+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d010f380 0x7f45d01abdf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b750 tx=0x7f45c000bb10 comp rx=0 tx=0).stop 2023-10-26T06:49:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.197+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 shutdown_connections 2023-10-26T06:49:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.197+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f45bc0a1800 0x7f45bc0a3cc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.197+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45d0119fd0 0x7f45d01a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.197+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45d010ef10 0x7f45d01ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.198+0000 7f45d79ff700 1 --2- 172.21.15.89:0/2041262809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45d010f380 0x7f45d01abdf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.198+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 >> 172.21.15.89:0/2041262809 conn(0x7f45d006b840 msgr2=0x7f45d0117fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.200+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 shutdown_connections 2023-10-26T06:49:06.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:06.201+0000 7f45d79ff700 1 -- 172.21.15.89:0/2041262809 wait complete. 2023-10-26T06:49:06.434 INFO:tasks.cephfs.filesystem:scrub status for tag:c0bad5b1-05ca-403c-97f4-41378e9d91fd - {'path': '/', 'tag': 'c0bad5b1-05ca-403c-97f4-41378e9d91fd', 'options': 'recursive,force'} 2023-10-26T06:49:07.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:07 smithi089 ceph-mon[95031]: pgmap v376: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 315 MiB/s wr, 113 op/s 2023-10-26T06:49:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:07 smithi192 ceph-mon[131661]: pgmap v376: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 315 MiB/s wr, 113 op/s 2023-10-26T06:49:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:07 smithi135 ceph-mon[131403]: pgmap v376: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 315 MiB/s wr, 113 op/s 2023-10-26T06:49:09.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:09 smithi089 ceph-mon[95031]: pgmap v377: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 301 MiB/s wr, 1.67k op/s 2023-10-26T06:49:09.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:09 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:49:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:09 smithi192 ceph-mon[131661]: pgmap v377: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 301 MiB/s wr, 1.67k op/s 2023-10-26T06:49:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:09 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:49:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:09 smithi135 ceph-mon[131403]: pgmap v377: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 301 MiB/s wr, 1.67k op/s 2023-10-26T06:49:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:09 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:49:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:11 smithi192 ceph-mon[131661]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 277 MiB/s wr, 4.75k op/s 2023-10-26T06:49:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:11 smithi135 ceph-mon[131403]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 277 MiB/s wr, 4.75k op/s 2023-10-26T06:49:11.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:11 smithi089 ceph-mon[95031]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 277 MiB/s wr, 4.75k op/s 2023-10-26T06:49:13.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:13 smithi192 ceph-mon[131661]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 234 MiB/s wr, 5.49k op/s 2023-10-26T06:49:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:13 smithi135 ceph-mon[131403]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 234 MiB/s wr, 5.49k op/s 2023-10-26T06:49:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:13 smithi089 ceph-mon[95031]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 234 MiB/s wr, 5.49k op/s 2023-10-26T06:49:15.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:15 smithi192 ceph-mon[131661]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 236 MiB/s wr, 10.37k op/s 2023-10-26T06:49:15.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:15 smithi135 ceph-mon[131403]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 236 MiB/s wr, 10.37k op/s 2023-10-26T06:49:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:15 smithi089 ceph-mon[95031]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 236 MiB/s wr, 10.37k op/s 2023-10-26T06:49:17.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:17 smithi192 ceph-mon[131661]: pgmap v381: 97 pgs: 97 active+clean; 14 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 195 MiB/s wr, 13.93k op/s 2023-10-26T06:49:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:17 smithi135 ceph-mon[131403]: pgmap v381: 97 pgs: 97 active+clean; 14 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 195 MiB/s wr, 13.93k op/s 2023-10-26T06:49:17.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:17 smithi089 ceph-mon[95031]: pgmap v381: 97 pgs: 97 active+clean; 14 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 195 MiB/s wr, 13.93k op/s 2023-10-26T06:49:19.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:19 smithi192 ceph-mon[131661]: pgmap v382: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 145 MiB/s wr, 15.81k op/s 2023-10-26T06:49:19.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:19 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:49:19.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:19 smithi135 ceph-mon[131403]: pgmap v382: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 145 MiB/s wr, 15.81k op/s 2023-10-26T06:49:19.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:19 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:49:20.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:19 smithi089 ceph-mon[95031]: pgmap v382: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 145 MiB/s wr, 15.81k op/s 2023-10-26T06:49:20.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:19 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:49:21.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:21 smithi192 ceph-mon[131661]: pgmap v383: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 150 MiB/s wr, 19.66k op/s 2023-10-26T06:49:21.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:21 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:21.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:21 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:21.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:21 smithi135 ceph-mon[131403]: pgmap v383: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 150 MiB/s wr, 19.66k op/s 2023-10-26T06:49:21.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:21 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:21.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:21 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:21.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:21 smithi089 ceph-mon[95031]: pgmap v383: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 150 MiB/s wr, 19.66k op/s 2023-10-26T06:49:21.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:21 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:21.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:21 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:22.724 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:49:22.724 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:49:22.725 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:22.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:49:22.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:49:22.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:22.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:49:22.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:49:22.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:49:23.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:23 smithi135 ceph-mon[131403]: pgmap v384: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 147 MiB/s wr, 19.14k op/s 2023-10-26T06:49:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:23 smithi089 ceph-mon[95031]: pgmap v384: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 147 MiB/s wr, 19.14k op/s 2023-10-26T06:49:23.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:23 smithi192 ceph-mon[131661]: pgmap v384: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 147 MiB/s wr, 19.14k op/s 2023-10-26T06:49:25.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:25 smithi135 ceph-mon[131403]: pgmap v385: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 169 MiB/s wr, 21.36k op/s 2023-10-26T06:49:25.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:25 smithi089 ceph-mon[95031]: pgmap v385: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 169 MiB/s wr, 21.36k op/s 2023-10-26T06:49:25.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:25 smithi192 ceph-mon[131661]: pgmap v385: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 169 MiB/s wr, 21.36k op/s 2023-10-26T06:49:27.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:27 smithi135 ceph-mon[131403]: pgmap v386: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 202 MiB/s wr, 23.84k op/s 2023-10-26T06:49:27.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:27 smithi089 ceph-mon[95031]: pgmap v386: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 202 MiB/s wr, 23.84k op/s 2023-10-26T06:49:27.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:27 smithi192 ceph-mon[131661]: pgmap v386: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 202 MiB/s wr, 23.84k op/s 2023-10-26T06:49:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:29 smithi135 ceph-mon[131403]: pgmap v387: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 179 KiB/s rd, 198 MiB/s wr, 22.70k op/s 2023-10-26T06:49:29.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:29 smithi089 ceph-mon[95031]: pgmap v387: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 179 KiB/s rd, 198 MiB/s wr, 22.70k op/s 2023-10-26T06:49:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:29 smithi192 ceph-mon[131661]: pgmap v387: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 179 KiB/s rd, 198 MiB/s wr, 22.70k op/s 2023-10-26T06:49:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:31 smithi135 ceph-mon[131403]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 234 MiB/s wr, 26.78k op/s 2023-10-26T06:49:31.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:31 smithi089 ceph-mon[95031]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 234 MiB/s wr, 26.78k op/s 2023-10-26T06:49:31.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:31 smithi192 ceph-mon[131661]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 234 MiB/s wr, 26.78k op/s 2023-10-26T06:49:33.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:33 smithi089 ceph-mon[95031]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 135 KiB/s rd, 217 MiB/s wr, 25.03k op/s 2023-10-26T06:49:33.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:33 smithi192 ceph-mon[131661]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 135 KiB/s rd, 217 MiB/s wr, 25.03k op/s 2023-10-26T06:49:33.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:33 smithi135 ceph-mon[131403]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 135 KiB/s rd, 217 MiB/s wr, 25.03k op/s 2023-10-26T06:49:35.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:35 smithi089 ceph-mon[95031]: pgmap v390: 97 pgs: 97 active+clean; 17 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 122 KiB/s rd, 208 MiB/s wr, 23.97k op/s 2023-10-26T06:49:35.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:35 smithi192 ceph-mon[131661]: pgmap v390: 97 pgs: 97 active+clean; 17 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 122 KiB/s rd, 208 MiB/s wr, 23.97k op/s 2023-10-26T06:49:35.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:35 smithi135 ceph-mon[131403]: pgmap v390: 97 pgs: 97 active+clean; 17 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 122 KiB/s rd, 208 MiB/s wr, 23.97k op/s 2023-10-26T06:49:36.437 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:49:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.834+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3114493224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc0ff640 msgr2=0x7ff8bc0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.835+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3114493224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc0ffac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800b970 tx=0x7ff8a8009260 comp rx=0 tx=0).stop 2023-10-26T06:49:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.835+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3114493224 shutdown_connections 2023-10-26T06:49:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.835+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3114493224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc100000 0x7ff8bc10c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.835+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3114493224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8bc1056e0 0x7ff8bc105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.835+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3114493224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc0ffac0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.835+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3114493224 >> 172.21.15.89:0/3114493224 conn(0x7ff8bc0767f0 msgr2=0x7ff8bc076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:36.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.836+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3114493224 shutdown_connections 2023-10-26T06:49:36.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.836+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3114493224 wait complete. 2023-10-26T06:49:36.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.837+0000 7ff8c2fff700 1 Processor -- start 2023-10-26T06:49:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.837+0000 7ff8c2fff700 1 -- start start 2023-10-26T06:49:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c2fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc194ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c2fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc100000 0x7ff8bc195000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c2fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8bc1056e0 0x7ff8bc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c2fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc199950 con 0x7ff8bc100000 2023-10-26T06:49:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c2fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc199ac0 con 0x7ff8bc1056e0 2023-10-26T06:49:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c2fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc199c80 con 0x7ff8bc0ff640 2023-10-26T06:49:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c0d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc194ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c0d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc194ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41444/0 (socket says 172.21.15.89:41444) 2023-10-26T06:49:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.838+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/3710748293 learned_addr learned my addr 172.21.15.89:0/3710748293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.839+0000 7ff8c159c700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8bc1056e0 0x7ff8bc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.839+0000 7ff8bbfff700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc100000 0x7ff8bc195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.839+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/3710748293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8bc1056e0 msgr2=0x7ff8bc199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.839+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8bc1056e0 0x7ff8bc199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.839+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc100000 msgr2=0x7ff8bc195000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.839+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc100000 0x7ff8bc195000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.839+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/3710748293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a800b5a0 con 0x7ff8bc0ff640 2023-10-26T06:49:36.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.840+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc194ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000bc50 tx=0x7ff8b000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:36.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.841+0000 7ff8b9ffb700 1 -- 172.21.15.89:0/3710748293 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00089c0 con 0x7ff8bc0ff640 2023-10-26T06:49:36.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.841+0000 7ff8b9ffb700 1 -- 172.21.15.89:0/3710748293 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b0016480 con 0x7ff8bc0ff640 2023-10-26T06:49:36.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.841+0000 7ff8b9ffb700 1 -- 172.21.15.89:0/3710748293 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0015640 con 0x7ff8bc0ff640 2023-10-26T06:49:36.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.842+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc199f60 con 0x7ff8bc0ff640 2023-10-26T06:49:36.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.843+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc1a1810 con 0x7ff8bc0ff640 2023-10-26T06:49:36.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.856+0000 7ff8b9ffb700 1 -- 172.21.15.89:0/3710748293 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff8b00165f0 con 0x7ff8bc0ff640 2023-10-26T06:49:36.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.857+0000 7ff8d6873b80 1 Processor -- start 2023-10-26T06:49:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.858+0000 7ff8b9ffb700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8ac0a17c0 0x7ff8ac0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.859+0000 7ff8b9ffb700 1 -- 172.21.15.89:0/3710748293 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff8b00cc8f0 con 0x7ff8bc0ff640 2023-10-26T06:49:36.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.859+0000 7ff8c159c700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8ac0a17c0 0x7ff8ac0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.859+0000 7ff8d6873b80 1 -- start start 2023-10-26T06:49:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.860+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab1be0 0x55fa59ab1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.860+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59aac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.860+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59aaca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.860+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59a1f6e0 con 0x55fa59ab2500 2023-10-26T06:49:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.860+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59a05c60 con 0x55fa59aac5d0 2023-10-26T06:49:36.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.860+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59a03580 con 0x55fa59ab1be0 2023-10-26T06:49:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.860+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59aac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.861+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59aac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52396/0 (socket says 172.21.15.89:52396) 2023-10-26T06:49:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.861+0000 7ff8bbfff700 1 -- 172.21.15.89:0/3061308449 learned_addr learned my addr 172.21.15.89:0/3061308449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:36.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.861+0000 7ff8c159c700 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59aaca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab1be0 0x55fa59ab1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8bbfff700 1 -- 172.21.15.89:0/3061308449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab1be0 msgr2=0x55fa59ab1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8bbfff700 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab1be0 0x55fa59ab1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8bbfff700 1 -- 172.21.15.89:0/3061308449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 msgr2=0x55fa59aaca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8bbfff700 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59aaca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8bbfff700 1 -- 172.21.15.89:0/3061308449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa599bc3f0 con 0x55fa59ab2500 2023-10-26T06:49:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8c159c700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8ac0a17c0 0x7ff8ac0a3c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4012bc0 tx=0x7ff8b4015040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:36.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.862+0000 7ff8bbfff700 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59aac090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800b970 tx=0x7ff8a8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f9ca8a0ed9e17a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.864+0000 7ff8a6ffd700 1 -- 172.21.15.89:0/3061308449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8014dc0 con 0x55fa59ab2500 2023-10-26T06:49:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.865+0000 7ff8a6ffd700 1 -- 172.21.15.89:0/3061308449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a80138c0 con 0x55fa59ab2500 2023-10-26T06:49:36.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.865+0000 7ff8a6ffd700 1 -- 172.21.15.89:0/3061308449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8009840 con 0x55fa59ab2500 2023-10-26T06:49:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 -- 172.21.15.89:0/3061308449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 msgr2=0x55fa59aac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59aac090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800b970 tx=0x7ff8a8005b00 comp rx=0 tx=0).stop 2023-10-26T06:49:36.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 -- 172.21.15.89:0/3061308449 shutdown_connections 2023-10-26T06:49:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59aaca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab1be0 0x55fa59ab1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/3061308449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59aac090 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 -- 172.21.15.89:0/3061308449 >> 172.21.15.89:0/3061308449 conn(0x55fa59a20720 msgr2=0x55fa59aa6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.866+0000 7ff8d6873b80 1 -- 172.21.15.89:0/3061308449 shutdown_connections 2023-10-26T06:49:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.867+0000 7ff8d6873b80 1 -- 172.21.15.89:0/3061308449 wait complete. 2023-10-26T06:49:36.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.867+0000 7ff8d6873b80 1 -- start start 2023-10-26T06:49:36.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.868+0000 7ff8d6873b80 1 Processor -- start 2023-10-26T06:49:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59aac5d0 0x55fa59ab5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59ab1be0 0x55fa59ab5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59ab62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa599fee70 con 0x55fa59ab2500 2023-10-26T06:49:36.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5999b710 con 0x55fa59ab1be0 2023-10-26T06:49:36.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5999b3b0 con 0x55fa59aac5d0 2023-10-26T06:49:36.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8d6873b80 10 client.0 _ll_register_callbacks cb 0x55fa598e5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:36.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8c159c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59ab62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8c159c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59ab62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52408/0 (socket says 172.21.15.89:52408) 2023-10-26T06:49:36.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.870+0000 7ff8c159c700 1 -- 172.21.15.89:0/375225656 learned_addr learned my addr 172.21.15.89:0/375225656 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:36.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.871+0000 7ff8c159c700 1 -- 172.21.15.89:0/375225656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59aac5d0 msgr2=0x55fa59ab5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.871+0000 7ff8c159c700 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59aac5d0 0x55fa59ab5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.871+0000 7ff8c159c700 1 -- 172.21.15.89:0/375225656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59ab1be0 msgr2=0x55fa59ab5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.871+0000 7ff8c159c700 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59ab1be0 0x55fa59ab5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.871+0000 7ff8c159c700 1 -- 172.21.15.89:0/375225656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc10cd90 con 0x55fa59ab2500 2023-10-26T06:49:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.871+0000 7ff8c159c700 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59ab62b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4019d80 tx=0x7ff8b4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f91697a8903d72f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:36.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.871+0000 7ff89bfff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:36.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.873+0000 7ff89bfff700 1 -- 172.21.15.89:0/375225656 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4018040 con 0x55fa59ab2500 2023-10-26T06:49:36.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.873+0000 7ff89bfff700 1 -- 172.21.15.89:0/375225656 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff8b4019e60 con 0x55fa59ab2500 2023-10-26T06:49:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.873+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa59a03ef0 con 0x55fa59ab2500 2023-10-26T06:49:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.874+0000 7ff89bfff700 1 -- 172.21.15.89:0/375225656 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b4020660 con 0x55fa59ab2500 2023-10-26T06:49:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.874+0000 7ff8d6873b80 10 client.15753 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.874+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55fa59829de0 con 0x55fa59ab2500 2023-10-26T06:49:36.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.874+0000 7ff89bfff700 1 -- 172.21.15.89:0/375225656 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fa59a03ef0 con 0x55fa59ab2500 2023-10-26T06:49:36.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff89bfff700 1 -- 172.21.15.89:0/375225656 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff8b402a8a0 con 0x55fa59ab2500 2023-10-26T06:49:36.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff8d6873b80 10 client.15753 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff8d6873b80 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff8d6873b80 20 client.15753 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59ac6010 0x55fa59ac8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff8d6873b80 4 client.15753 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:36.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa59a7b8b0 con 0x55fa59ac6010 2023-10-26T06:49:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.875+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59ac6010 0x55fa59ac8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.876+0000 7ff89bfff700 10 client.15753 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:36.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.876+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59ac6010 0x55fa59ac8470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c66f0bef2f381c1e server_cookie=c8f444bf15cf081 in_seq=0 out_seq=0 2023-10-26T06:49:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.877+0000 7ff89bfff700 1 -- 172.21.15.89:0/375225656 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fa59a7b8b0 con 0x55fa59ac6010 2023-10-26T06:49:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.877+0000 7ff89bfff700 10 client.15753 handle_command_reply: tid=0 2023-10-26T06:49:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.877+0000 7ff8d6873b80 1 client.15753 shutdown 2023-10-26T06:49:36.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.877+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 msgr2=0x55fa59ab62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.877+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59ab62b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4019d80 tx=0x7ff8b4004b40 comp rx=0 tx=0).stop 2023-10-26T06:49:36.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.879+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 shutdown_connections 2023-10-26T06:49:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.879+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab2500 0x55fa59ab62b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.879+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59ac6010 0x55fa59ac8470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.879+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59aac5d0 0x55fa59ab5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.879+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/375225656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59ab1be0 0x55fa59ab5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.879+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 >> 172.21.15.89:0/375225656 conn(0x55fa59a20720 msgr2=0x55fa59aab080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.879+0000 7ff89bfff700 0 client.15753 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.880+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 shutdown_connections 2023-10-26T06:49:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.880+0000 7ff8d6873b80 1 -- 172.21.15.89:0/375225656 wait complete. 2023-10-26T06:49:36.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.882+0000 7ff8d6873b80 20 client.15753 trim_cache size 0 max 16384 2023-10-26T06:49:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.926+0000 7ff8d6873b80 1 Processor -- start 2023-10-26T06:49:36.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.927+0000 7ff8d6873b80 1 -- start start 2023-10-26T06:49:36.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 0x55fa59aafdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59ab5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5987f920 con 0x55fa59ab1be0 2023-10-26T06:49:36.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59ab6e30 con 0x55fa59aac5d0 2023-10-26T06:49:36.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59aa3230 con 0x55fa59ab2500 2023-10-26T06:49:36.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8c0d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8c0d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:38448/0 (socket says 172.21.15.89:38448) 2023-10-26T06:49:36.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/2403567196 learned_addr learned my addr 172.21.15.89:0/2403567196 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:36.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.928+0000 7ff8bbfff700 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 0x55fa59aafdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c159c700 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59ab5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/2403567196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 msgr2=0x55fa59ab5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59ab5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/2403567196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 msgr2=0x55fa59aafdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 0x55fa59aafdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c0d9b700 1 -- 172.21.15.89:0/2403567196 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa59812c60 con 0x55fa59aac5d0 2023-10-26T06:49:36.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c159c700 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59ab5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:36.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abce80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000bfd0 tx=0x7ff8b0021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa1a09959e725a0a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:36.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.929+0000 7ff8a77fe700 1 -- 172.21.15.89:0/2403567196 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00993f0 con 0x55fa59aac5d0 2023-10-26T06:49:36.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.930+0000 7ff8a77fe700 1 -- 172.21.15.89:0/2403567196 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b00a1e90 con 0x55fa59aac5d0 2023-10-26T06:49:36.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.930+0000 7ff8a77fe700 1 -- 172.21.15.89:0/2403567196 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00168d0 con 0x55fa59aac5d0 2023-10-26T06:49:36.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.931+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2403567196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 msgr2=0x55fa59abce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.931+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abce80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000bfd0 tx=0x7ff8b0021040 comp rx=0 tx=0).stop 2023-10-26T06:49:36.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.933+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2403567196 shutdown_connections 2023-10-26T06:49:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.933+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59ab5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.933+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abce80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.933+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2403567196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 0x55fa59aafdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.933+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2403567196 >> 172.21.15.89:0/2403567196 conn(0x55fa59a20720 msgr2=0x55fa59aaf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.933+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2403567196 shutdown_connections 2023-10-26T06:49:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.933+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2403567196 wait complete. 2023-10-26T06:49:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.934+0000 7ff8d6873b80 1 -- start start 2023-10-26T06:49:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.934+0000 7ff8d6873b80 1 Processor -- start 2023-10-26T06:49:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.935+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 0x55fa59abb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8d6873b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59abbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8d6873b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59abd1e0 con 0x55fa59ab1be0 2023-10-26T06:49:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8c159c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59abbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8c159c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59abbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41490/0 (socket says 172.21.15.89:41490) 2023-10-26T06:49:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8c159c700 1 -- 172.21.15.89:0/2709598935 learned_addr learned my addr 172.21.15.89:0/2709598935 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59b19ed0 con 0x55fa59aac5d0 2023-10-26T06:49:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.936+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa59b1a010 con 0x55fa59ab2500 2023-10-26T06:49:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8d6873b80 10 client.0 _ll_register_callbacks cb 0x55fa598e5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8c159c700 1 -- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 msgr2=0x55fa59abb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8c159c700 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8c159c700 1 -- 172.21.15.89:0/2709598935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 msgr2=0x55fa59abb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8c159c700 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 0x55fa59abb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8c159c700 1 -- 172.21.15.89:0/2709598935 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc1a1810 con 0x55fa59ab2500 2023-10-26T06:49:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff8c159c700 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59abbec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4019540 tx=0x7ff8b4019d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aadc380814697d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.937+0000 7ff89bfff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:49:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.938+0000 7ff89bfff700 1 -- 172.21.15.89:0/2709598935 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4018040 con 0x55fa59ab2500 2023-10-26T06:49:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.938+0000 7ff89bfff700 1 -- 172.21.15.89:0/2709598935 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff8b4019e60 con 0x55fa59ab2500 2023-10-26T06:49:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.938+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8a80138c0 con 0x55fa59ab2500 2023-10-26T06:49:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.939+0000 7ff89bfff700 1 -- 172.21.15.89:0/2709598935 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b4020660 con 0x55fa59ab2500 2023-10-26T06:49:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.939+0000 7ff8d6873b80 10 client.25732 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.939+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55fa59850250 con 0x55fa59ab2500 2023-10-26T06:49:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.939+0000 7ff89bfff700 1 -- 172.21.15.89:0/2709598935 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a80138c0 con 0x55fa59ab2500 2023-10-26T06:49:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff89bfff700 1 -- 172.21.15.89:0/2709598935 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff8b402a8a0 con 0x55fa59ab2500 2023-10-26T06:49:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff8d6873b80 10 client.25732 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff8d6873b80 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff8d6873b80 20 client.25732 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59b1b790 0x55fa59b1dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff8d6873b80 4 client.25732 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:49:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fa599f7e50 con 0x55fa59b1b790 2023-10-26T06:49:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.940+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59b1b790 0x55fa59b1dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.941+0000 7ff89bfff700 10 client.25732 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.941+0000 7ff8c0d9b700 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59b1b790 0x55fa59b1dbb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1819f0c5bdb5459c server_cookie=f38143d40916e331 in_seq=0 out_seq=0 2023-10-26T06:49:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.941+0000 7ff89bfff700 1 -- 172.21.15.89:0/2709598935 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fa599f7e50 con 0x55fa59b1b790 2023-10-26T06:49:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.941+0000 7ff89bfff700 10 client.25732 handle_command_reply: tid=0 2023-10-26T06:49:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.941+0000 7ff8d6873b80 1 client.25732 shutdown 2023-10-26T06:49:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 msgr2=0x55fa59abbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59abbec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4019540 tx=0x7ff8b4019d10 comp rx=0 tx=0).stop 2023-10-26T06:49:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 shutdown_connections 2023-10-26T06:49:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55fa59b1b790 0x55fa59b1dbb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fa59ab2500 0x55fa59abbec0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff89bfff700 0 client.25732 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa59aac5d0 0x55fa59abb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 --2- 172.21.15.89:0/2709598935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fa59ab1be0 0x55fa59abb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 >> 172.21.15.89:0/2709598935 conn(0x55fa59a20720 msgr2=0x55fa59ab0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.943+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 shutdown_connections 2023-10-26T06:49:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.944+0000 7ff8d6873b80 1 -- 172.21.15.89:0/2709598935 wait complete. 2023-10-26T06:49:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.944+0000 7ff8d6873b80 20 client.25732 trim_cache size 0 max 16384 2023-10-26T06:49:36.950 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:49:36.951 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:49:36.951 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:49:36.951 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:49:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8ac0a17c0 msgr2=0x7ff8ac0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8ac0a17c0 0x7ff8ac0a3c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4012bc0 tx=0x7ff8b4015040 comp rx=0 tx=0).stop 2023-10-26T06:49:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc0ff640 msgr2=0x7ff8bc194ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc194ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000bc50 tx=0x7ff8b000bf60 comp rx=0 tx=0).stop 2023-10-26T06:49:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 shutdown_connections 2023-10-26T06:49:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8ac0a17c0 0x7ff8ac0a3c80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8bc1056e0 0x7ff8bc199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8bc0ff640 0x7ff8bc194ac0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 --2- 172.21.15.89:0/3710748293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8bc100000 0x7ff8bc195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.945+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 >> 172.21.15.89:0/3710748293 conn(0x7ff8bc0767f0 msgr2=0x7ff8bc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.946+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 shutdown_connections 2023-10-26T06:49:36.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:36.946+0000 7ff8c2fff700 1 -- 172.21.15.89:0/3710748293 wait complete. 2023-10-26T06:49:36.981 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:49:36.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:49:37.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.356+0000 7f9ce95ff700 1 -- 172.21.15.89:0/4006044516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 msgr2=0x7f9ce4105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:37.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.357+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/4006044516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce4105af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cd401a9a0 tx=0x7f9cd401acb0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.360+0000 7f9ce95ff700 1 -- 172.21.15.89:0/4006044516 shutdown_connections 2023-10-26T06:49:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.360+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/4006044516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ce4100290 0x7f9ce410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.360+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/4006044516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce4105af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.360+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/4006044516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ce40ff8d0 0x7f9ce40ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.360+0000 7f9ce95ff700 1 -- 172.21.15.89:0/4006044516 >> 172.21.15.89:0/4006044516 conn(0x7f9ce4076400 msgr2=0x7f9ce4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.360+0000 7f9ce95ff700 1 -- 172.21.15.89:0/4006044516 shutdown_connections 2023-10-26T06:49:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.360+0000 7f9ce95ff700 1 -- 172.21.15.89:0/4006044516 wait complete. 2023-10-26T06:49:37.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.361+0000 7f9ce95ff700 1 Processor -- start 2023-10-26T06:49:37.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.362+0000 7f9ce95ff700 1 -- start start 2023-10-26T06:49:37.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.362+0000 7f9ce95ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ce40ff8d0 0x7f9ce4191c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:37.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.362+0000 7f9ce95ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ce4100290 0x7f9ce41921b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:37.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.362+0000 7f9ce95ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce41926f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:37.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.362+0000 7f9ce95ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410ac70 con 0x7f9ce40ff8d0 2023-10-26T06:49:37.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.362+0000 7f9ce95ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410ade0 con 0x7f9ce4100290 2023-10-26T06:49:37.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.362+0000 7f9ce95ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410afa0 con 0x7f9ce4105710 2023-10-26T06:49:37.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ce40ff8d0 0x7f9ce4191c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:37.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce27fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ce4100290 0x7f9ce41921b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:37.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce41926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:37.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce41926f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41518/0 (socket says 172.21.15.89:41518) 2023-10-26T06:49:37.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 -- 172.21.15.89:0/3172590647 learned_addr learned my addr 172.21.15.89:0/3172590647 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:37.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 -- 172.21.15.89:0/3172590647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ce4100290 msgr2=0x7f9ce41921b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:37.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ce4100290 0x7f9ce41921b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 -- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ce40ff8d0 msgr2=0x7f9ce4191c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:37.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ce40ff8d0 0x7f9ce4191c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.363+0000 7f9ce37fe700 1 -- 172.21.15.89:0/3172590647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd401a600 con 0x7f9ce4105710 2023-10-26T06:49:37.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.364+0000 7f9ce37fe700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce41926f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc007ee0 tx=0x7f9cdc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:37.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.365+0000 7f9ccbfff700 1 -- 172.21.15.89:0/3172590647 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc017070 con 0x7f9ce4105710 2023-10-26T06:49:37.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.365+0000 7f9ccbfff700 1 -- 172.21.15.89:0/3172590647 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc00f040 con 0x7f9ce4105710 2023-10-26T06:49:37.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.365+0000 7f9ccbfff700 1 -- 172.21.15.89:0/3172590647 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc017070 con 0x7f9ce4105710 2023-10-26T06:49:37.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.365+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce410b280 con 0x7f9ce4105710 2023-10-26T06:49:37.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.365+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce410b680 con 0x7f9ce4105710 2023-10-26T06:49:37.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.367+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd00052e0 con 0x7f9ce4105710 2023-10-26T06:49:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.370+0000 7f9ccbfff700 1 -- 172.21.15.89:0/3172590647 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9cdc01c410 con 0x7f9ce4105710 2023-10-26T06:49:37.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.372+0000 7f9ccbfff700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9ccc0a1700 0x7f9ccc0a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.374+0000 7f9ccbfff700 1 -- 172.21.15.89:0/3172590647 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f9cdc099a90 con 0x7f9ce4105710 2023-10-26T06:49:37.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.374+0000 7f9ce2ffd700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9ccc0a1700 0x7f9ccc0a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:37.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.377+0000 7f9ce2ffd700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9ccc0a1700 0x7f9ccc0a3bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4003800 tx=0x7f9cd401ca40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:37.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.381+0000 7f9ccbfff700 1 -- 172.21.15.89:0/3172590647 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9cdc095e60 con 0x7f9ce4105710 2023-10-26T06:49:37.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:37 smithi089 ceph-mon[95031]: pgmap v391: 97 pgs: 97 active+clean; 17 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 122 KiB/s rd, 241 MiB/s wr, 28.75k op/s 2023-10-26T06:49:37.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:37 smithi192 ceph-mon[131661]: pgmap v391: 97 pgs: 97 active+clean; 17 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 122 KiB/s rd, 241 MiB/s wr, 28.75k op/s 2023-10-26T06:49:37.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.914+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cd0005590 con 0x7f9ce4105710 2023-10-26T06:49:37.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.915+0000 7f9ccbfff700 1 -- 172.21.15.89:0/3172590647 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f9cdc0b0c20 con 0x7f9ce4105710 2023-10-26T06:49:37.922 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:49:37.922 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:49:37.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9ccc0a1700 msgr2=0x7f9ccc0a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:37.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9ccc0a1700 0x7f9ccc0a3bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4003800 tx=0x7f9cd401ca40 comp rx=0 tx=0).stop 2023-10-26T06:49:37.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 msgr2=0x7f9ce41926f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:37.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce41926f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc007ee0 tx=0x7f9cdc00d3b0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 shutdown_connections 2023-10-26T06:49:37.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9ccc0a1700 0x7f9ccc0a3bc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ce40ff8d0 0x7f9ce4191c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce4105710 0x7f9ce41926f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 --2- 172.21.15.89:0/3172590647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ce4100290 0x7f9ce41921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:37.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 >> 172.21.15.89:0/3172590647 conn(0x7f9ce4076400 msgr2=0x7f9ce40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:37.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 shutdown_connections 2023-10-26T06:49:37.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:37.921+0000 7f9ce95ff700 1 -- 172.21.15.89:0/3172590647 wait complete. 2023-10-26T06:49:37.930 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:49:37.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:49:37.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:37 smithi135 ceph-mon[131403]: pgmap v391: 97 pgs: 97 active+clean; 17 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 122 KiB/s rd, 241 MiB/s wr, 28.75k op/s 2023-10-26T06:49:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.322+0000 7f91a919d700 1 -- 172.21.15.89:0/3177210676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c09be60 msgr2=0x7f919c09c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.322+0000 7f91a919d700 1 --2- 172.21.15.89:0/3177210676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c09be60 0x7f919c09c240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9190009990 tx=0x7f9190009ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.323+0000 7f91a919d700 1 -- 172.21.15.89:0/3177210676 shutdown_connections 2023-10-26T06:49:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.323+0000 7f91a919d700 1 --2- 172.21.15.89:0/3177210676 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c005150 0x7f919c0a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.323+0000 7f91a919d700 1 --2- 172.21.15.89:0/3177210676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c09be60 0x7f919c09c240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.323+0000 7f91a919d700 1 --2- 172.21.15.89:0/3177210676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919c004790 0x7f919c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.323+0000 7f91a919d700 1 -- 172.21.15.89:0/3177210676 >> 172.21.15.89:0/3177210676 conn(0x7f919c00a790 msgr2=0x7f919c00aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.323+0000 7f91a919d700 1 -- 172.21.15.89:0/3177210676 shutdown_connections 2023-10-26T06:49:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.323+0000 7f91a919d700 1 -- 172.21.15.89:0/3177210676 wait complete. 2023-10-26T06:49:38.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.326+0000 7f91a919d700 1 Processor -- start 2023-10-26T06:49:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.326+0000 7f91a919d700 1 -- start start 2023-10-26T06:49:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a919d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919c004790 0x7f919c138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a919d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c005150 0x7f919c138970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a919d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c09be60 0x7f919c0a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a919d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c0a2210 con 0x7f919c004790 2023-10-26T06:49:38.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a919d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c0a2380 con 0x7f919c09be60 2023-10-26T06:49:38.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a919d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c0a2540 con 0x7f919c005150 2023-10-26T06:49:38.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a899c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c09be60 0x7f919c0a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a899c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c09be60 0x7f919c0a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:38472/0 (socket says 172.21.15.89:38472) 2023-10-26T06:49:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a899c700 1 -- 172.21.15.89:0/448439224 learned_addr learned my addr 172.21.15.89:0/448439224 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.327+0000 7f91a3fff700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919c004790 0x7f919c138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.328+0000 7f91a37fe700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c005150 0x7f919c138970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.328+0000 7f91a899c700 1 -- 172.21.15.89:0/448439224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c005150 msgr2=0x7f919c138970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.328+0000 7f91a899c700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c005150 0x7f919c138970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.328+0000 7f91a899c700 1 -- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919c004790 msgr2=0x7f919c138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.328+0000 7f91a899c700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919c004790 0x7f919c138430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.328+0000 7f91a899c700 1 -- 172.21.15.89:0/448439224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91900095f0 con 0x7f919c09be60 2023-10-26T06:49:38.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.328+0000 7f91a899c700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c09be60 0x7f919c0a1bb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91a404f310 tx=0x7f91a406d250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:38.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.330+0000 7f91a17fa700 1 -- 172.21.15.89:0/448439224 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a406db50 con 0x7f919c09be60 2023-10-26T06:49:38.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.330+0000 7f91a17fa700 1 -- 172.21.15.89:0/448439224 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a406dcb0 con 0x7f919c09be60 2023-10-26T06:49:38.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.330+0000 7f91a17fa700 1 -- 172.21.15.89:0/448439224 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4072740 con 0x7f919c09be60 2023-10-26T06:49:38.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.330+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c0a2820 con 0x7f919c09be60 2023-10-26T06:49:38.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.330+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c150430 con 0x7f919c09be60 2023-10-26T06:49:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.342+0000 7f91a17fa700 1 -- 172.21.15.89:0/448439224 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f91a40680d0 con 0x7f919c09be60 2023-10-26T06:49:38.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.342+0000 7f91bdcf6b80 1 Processor -- start 2023-10-26T06:49:38.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.343+0000 7f91bdcf6b80 1 -- start start 2023-10-26T06:49:38.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.344+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d91d60 0x559795d92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.344+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d8c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.344+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d8cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.344+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795cff6e0 con 0x559795d91d60 2023-10-26T06:49:38.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.344+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795ce5c60 con 0x559795d8c820 2023-10-26T06:49:38.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.344+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795ce3580 con 0x559795d92680 2023-10-26T06:49:38.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.345+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d91d60 0x559795d92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.346+0000 7f91a3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d8c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.346+0000 7f91a3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d8c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41550/0 (socket says 172.21.15.89:41550) 2023-10-26T06:49:38.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.346+0000 7f91a3fff700 1 -- 172.21.15.89:0/274420895 learned_addr learned my addr 172.21.15.89:0/274420895 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:38.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.346+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d8cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.346+0000 7f91a17fa700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91940a1670 0x7f91940a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.347+0000 7f91a17fa700 1 -- 172.21.15.89:0/448439224 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f91a412e910 con 0x7f919c09be60 2023-10-26T06:49:38.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.347+0000 7f91a899c700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91940a1670 0x7f91940a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.347+0000 7f91a3fff700 1 -- 172.21.15.89:0/274420895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 msgr2=0x559795d8cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.347+0000 7f91a3fff700 1 --2- 172.21.15.89:0/274420895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d8cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.347+0000 7f91a3fff700 1 -- 172.21.15.89:0/274420895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d91d60 msgr2=0x559795d92140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.347+0000 7f91a3fff700 1 --2- 172.21.15.89:0/274420895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d91d60 0x559795d92140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.347+0000 7f91a3fff700 1 -- 172.21.15.89:0/274420895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559795c9c3f0 con 0x559795d92680 2023-10-26T06:49:38.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.348+0000 7f91a899c700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91940a1670 0x7f91940a3b30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f91a4066240 tx=0x7f91a4066570 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:38.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.349+0000 7f91a3fff700 1 --2- 172.21.15.89:0/274420895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d8c2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9190009540 tx=0x7f9190009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6c8bc618e97643 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.351+0000 7f918e7fc700 1 -- 172.21.15.89:0/274420895 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919001d070 con 0x559795d92680 2023-10-26T06:49:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.352+0000 7f918e7fc700 1 -- 172.21.15.89:0/274420895 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919000be00 con 0x559795d92680 2023-10-26T06:49:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.352+0000 7f918e7fc700 1 -- 172.21.15.89:0/274420895 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000f6c0 con 0x559795d92680 2023-10-26T06:49:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.353+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/274420895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 msgr2=0x559795d8c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.354+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/274420895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d8c2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9190009540 tx=0x7f9190009570 comp rx=0 tx=0).stop 2023-10-26T06:49:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.354+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/274420895 shutdown_connections 2023-10-26T06:49:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.355+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/274420895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d91d60 0x559795d92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.355+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/274420895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d8cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.355+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/274420895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d8c2e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.355+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/274420895 >> 172.21.15.89:0/274420895 conn(0x559795d00720 msgr2=0x559795d86cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.355+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/274420895 shutdown_connections 2023-10-26T06:49:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.355+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/274420895 wait complete. 2023-10-26T06:49:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.356+0000 7f91bdcf6b80 1 -- start start 2023-10-26T06:49:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.356+0000 7f91bdcf6b80 1 Processor -- start 2023-10-26T06:49:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.357+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d8c820 0x559795d986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d95a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d96120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795cdee70 con 0x559795d8c820 2023-10-26T06:49:38.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795c7b710 con 0x559795d91d60 2023-10-26T06:49:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795c7b3b0 con 0x559795d92680 2023-10-26T06:49:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91a3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d95a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91a3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d95a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:38496/0 (socket says 172.21.15.89:38496) 2023-10-26T06:49:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.358+0000 7f91a3fff700 1 -- 172.21.15.89:0/440341249 learned_addr learned my addr 172.21.15.89:0/440341249 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:38.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.359+0000 7f91bdcf6b80 10 client.0 _ll_register_callbacks cb 0x559795bc5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:38.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.359+0000 7f91a37fe700 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d8c820 0x559795d986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.360+0000 7f91a37fe700 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d96120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.360+0000 7f91a37fe700 1 -- 172.21.15.89:0/440341249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 msgr2=0x559795d95a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.360+0000 7f91a37fe700 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d95a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.360+0000 7f91a37fe700 1 -- 172.21.15.89:0/440341249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d8c820 msgr2=0x559795d986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.360+0000 7f91a37fe700 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d8c820 0x559795d986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.360+0000 7f91a37fe700 1 -- 172.21.15.89:0/440341249 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c0a34e0 con 0x559795d92680 2023-10-26T06:49:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91a37fe700 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d96120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f919c0a30b0 tx=0x7f9198004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57f703a25444954d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91837fe700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:49:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91837fe700 1 -- 172.21.15.89:0/440341249 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198014490 con 0x559795d92680 2023-10-26T06:49:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559795ce3ef0 con 0x559795d92680 2023-10-26T06:49:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91837fe700 1 -- 172.21.15.89:0/440341249 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f9198013ba0 con 0x559795d92680 2023-10-26T06:49:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91837fe700 1 -- 172.21.15.89:0/440341249 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9198013d60 con 0x559795d92680 2023-10-26T06:49:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91bdcf6b80 10 client.25747 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.361+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559795b09de0 con 0x559795d92680 2023-10-26T06:49:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.362+0000 7f91837fe700 1 -- 172.21.15.89:0/440341249 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559795ce3ef0 con 0x559795d92680 2023-10-26T06:49:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.362+0000 7f91837fe700 1 -- 172.21.15.89:0/440341249 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91980052f0 con 0x559795d92680 2023-10-26T06:49:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.364+0000 7f91bdcf6b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.364+0000 7f91bdcf6b80 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.364+0000 7f91bdcf6b80 20 client.25747 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.364+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795da6100 0x559795da8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.364+0000 7f91bdcf6b80 4 client.25747 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.364+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559795d5b8b0 con 0x559795da6100 2023-10-26T06:49:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.365+0000 7f91a3fff700 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795da6100 0x559795da8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.366+0000 7f91a3fff700 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795da6100 0x559795da8520 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01c2674afd47331 server_cookie=d381af8f005655c7 in_seq=0 out_seq=0 2023-10-26T06:49:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.366+0000 7f91837fe700 10 client.25747 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.367+0000 7f91837fe700 1 -- 172.21.15.89:0/440341249 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559795d5b8b0 con 0x559795da6100 2023-10-26T06:49:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.367+0000 7f91837fe700 10 client.25747 handle_command_reply: tid=0 2023-10-26T06:49:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.367+0000 7f91bdcf6b80 1 client.25747 shutdown 2023-10-26T06:49:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 msgr2=0x559795d96120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d96120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f919c0a30b0 tx=0x7f9198004750 comp rx=0 tx=0).stop 2023-10-26T06:49:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 shutdown_connections 2023-10-26T06:49:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d8c820 0x559795d986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795da6100 0x559795da8520 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d96120 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/440341249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d95a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 >> 172.21.15.89:0/440341249 conn(0x559795d00720 msgr2=0x559795d8b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.368+0000 7f91837fe700 0 client.25747 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.369+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 shutdown_connections 2023-10-26T06:49:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.369+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/440341249 wait complete. 2023-10-26T06:49:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.370+0000 7f91bdcf6b80 20 client.25747 trim_cache size 0 max 16384 2023-10-26T06:49:38.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.405+0000 7f91bdcf6b80 1 Processor -- start 2023-10-26T06:49:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.406+0000 7f91bdcf6b80 1 -- start start 2023-10-26T06:49:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.406+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d8c820 0x559795d90160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.407+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d81820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.407+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d92680 0x559795d95330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.407+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795b5f920 con 0x559795d92680 2023-10-26T06:49:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.407+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d8c820 0x559795d90160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.407+0000 7f91a3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d81820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.407+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d8c820 0x559795d90160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41570/0 (socket says 172.21.15.89:41570) 2023-10-26T06:49:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.407+0000 7f91a37fe700 1 -- 172.21.15.89:0/3898107409 learned_addr learned my addr 172.21.15.89:0/3898107409 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.408+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/3898107409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795d96f90 con 0x559795d91d60 2023-10-26T06:49:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.408+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/3898107409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795d834a0 con 0x559795d8c820 2023-10-26T06:49:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.409+0000 7f91a37fe700 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d92680 0x559795d95330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.409+0000 7f91a3fff700 1 -- 172.21.15.89:0/3898107409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d8c820 msgr2=0x559795d90160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.409+0000 7f91a3fff700 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d8c820 0x559795d90160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.409+0000 7f91a3fff700 1 -- 172.21.15.89:0/3898107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d92680 msgr2=0x559795d95330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.409+0000 7f91a3fff700 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d92680 0x559795d95330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.409+0000 7f91a3fff700 1 -- 172.21.15.89:0/3898107409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559795af2c60 con 0x559795d91d60 2023-10-26T06:49:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.409+0000 7f91a37fe700 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d92680 0x559795d95330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.410+0000 7f91a3fff700 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d81820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9190006010 tx=0x7f9190003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb01c133d49ef85 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.411+0000 7f918effd700 1 -- 172.21.15.89:0/3898107409 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919001d040 con 0x559795d91d60 2023-10-26T06:49:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.411+0000 7f918effd700 1 -- 172.21.15.89:0/3898107409 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9190003790 con 0x559795d91d60 2023-10-26T06:49:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.411+0000 7f918effd700 1 -- 172.21.15.89:0/3898107409 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91900054c0 con 0x559795d91d60 2023-10-26T06:49:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.412+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/3898107409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 msgr2=0x559795d81820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.412+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d81820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9190006010 tx=0x7f9190003ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.415+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/3898107409 shutdown_connections 2023-10-26T06:49:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.415+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d92680 0x559795d95330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.415+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d8c820 0x559795d90160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.415+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/3898107409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d91d60 0x559795d81820 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9190006010 tx=0x7f9190003ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.415+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/3898107409 >> 172.21.15.89:0/3898107409 conn(0x559795d00720 msgr2=0x559795da17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.415+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/3898107409 shutdown_connections 2023-10-26T06:49:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.415+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/3898107409 wait complete. 2023-10-26T06:49:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.416+0000 7f91bdcf6b80 1 -- start start 2023-10-26T06:49:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.416+0000 7f91bdcf6b80 1 Processor -- start 2023-10-26T06:49:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d9cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91bdcf6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d9b990 0x559795d9be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795d9d650 con 0x559795d9b990 2023-10-26T06:49:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795df9f70 con 0x559795d8c820 2023-10-26T06:49:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91bdcf6b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559795dfa130 con 0x559795d92680 2023-10-26T06:49:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91bdcf6b80 10 client.0 _ll_register_callbacks cb 0x559795bc5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d9cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91a37fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d9cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:38508/0 (socket says 172.21.15.89:38508) 2023-10-26T06:49:38.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.417+0000 7f91a37fe700 1 -- 172.21.15.89:0/2469212467 learned_addr learned my addr 172.21.15.89:0/2469212467 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91a37fe700 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d9b990 0x559795d9be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91a37fe700 1 -- 172.21.15.89:0/2469212467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 msgr2=0x559795d9b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91a37fe700 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91a37fe700 1 -- 172.21.15.89:0/2469212467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d9b990 msgr2=0x559795d9be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:38.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91a37fe700 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d9b990 0x559795d9be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91a37fe700 1 -- 172.21.15.89:0/2469212467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559795c9c3f0 con 0x559795d8c820 2023-10-26T06:49:38.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91a37fe700 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d9cef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91980094f0 tx=0x7f919802f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d332ec5d1baf5302 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:38.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.418+0000 7f91837fe700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:49:38.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.419+0000 7f91837fe700 1 -- 172.21.15.89:0/2469212467 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198032070 con 0x559795d8c820 2023-10-26T06:49:38.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.419+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559795cd7e50 con 0x559795d8c820 2023-10-26T06:49:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.419+0000 7f91837fe700 1 -- 172.21.15.89:0/2469212467 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f9198000e40 con 0x559795d8c820 2023-10-26T06:49:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.419+0000 7f91837fe700 1 -- 172.21.15.89:0/2469212467 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9198032070 con 0x559795d8c820 2023-10-26T06:49:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.419+0000 7f91bdcf6b80 10 client.25760 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.419+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559795b30250 con 0x559795d8c820 2023-10-26T06:49:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91837fe700 1 -- 172.21.15.89:0/2469212467 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f919803d020 con 0x559795d8c820 2023-10-26T06:49:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91837fe700 1 -- 172.21.15.89:0/2469212467 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9198037070 con 0x559795d8c820 2023-10-26T06:49:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91bdcf6b80 10 client.25760 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91bdcf6b80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91bdcf6b80 20 client.25760 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795dfb870 0x559795dfdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91bdcf6b80 4 client.25760 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:49:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559795cd7b70 con 0x559795dfb870 2023-10-26T06:49:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.420+0000 7f91a3fff700 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795dfb870 0x559795dfdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.421+0000 7f91837fe700 10 client.25760 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.421+0000 7f91a3fff700 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795dfb870 0x559795dfdc90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7929f65b80fc44 server_cookie=8d82092572473209 in_seq=0 out_seq=0 2023-10-26T06:49:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.421+0000 7f91837fe700 1 -- 172.21.15.89:0/2469212467 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559795cd7b70 con 0x559795dfb870 2023-10-26T06:49:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.421+0000 7f91837fe700 10 client.25760 handle_command_reply: tid=0 2023-10-26T06:49:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 client.25760 shutdown 2023-10-26T06:49:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 msgr2=0x559795d9cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d9cef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91980094f0 tx=0x7f919802f040 comp rx=0 tx=0).stop 2023-10-26T06:49:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 shutdown_connections 2023-10-26T06:49:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559795d9b990 0x559795d9be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559795dfb870 0x559795dfdc90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559795d8c820 0x559795d9cef0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 --2- 172.21.15.89:0/2469212467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559795d92680 0x559795d9b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 >> 172.21.15.89:0/2469212467 conn(0x559795d00720 msgr2=0x559795d90b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:38.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91837fe700 0 client.25760 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:38.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 shutdown_connections 2023-10-26T06:49:38.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.422+0000 7f91bdcf6b80 1 -- 172.21.15.89:0/2469212467 wait complete. 2023-10-26T06:49:38.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.423+0000 7f91bdcf6b80 20 client.25760 trim_cache size 0 max 16384 2023-10-26T06:49:38.433 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:49:38.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91940a1670 msgr2=0x7f91940a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91940a1670 0x7f91940a3b30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f91a4066240 tx=0x7f91a4066570 comp rx=0 tx=0).stop 2023-10-26T06:49:38.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c09be60 msgr2=0x7f919c0a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:38.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c09be60 0x7f919c0a1bb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91a404f310 tx=0x7f91a406d250 comp rx=0 tx=0).stop 2023-10-26T06:49:38.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 shutdown_connections 2023-10-26T06:49:38.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f91940a1670 0x7f91940a3b30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f919c004790 0x7f919c138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f919c09be60 0x7f919c0a1bb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 --2- 172.21.15.89:0/448439224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f919c005150 0x7f919c138970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:38.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.426+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 >> 172.21.15.89:0/448439224 conn(0x7f919c00a790 msgr2=0x7f919c093890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:38.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.427+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 shutdown_connections 2023-10-26T06:49:38.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:38.427+0000 7f91a919d700 1 -- 172.21.15.89:0/448439224 wait complete. 2023-10-26T06:49:38.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:38 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3172590647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:49:38.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:38 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3172590647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:49:38.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:38 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3172590647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:49:39.466 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:49:39.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:49:39.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:39 smithi089 ceph-mon[95031]: pgmap v392: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 141 KiB/s rd, 198 MiB/s wr, 24.22k op/s 2023-10-26T06:49:39.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.854+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3251319535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b48113940 msgr2=0x7f2b48115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.854+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3251319535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b48113940 0x7f2b48115d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009f90 tx=0x7f2b4000b330 comp rx=0 tx=0).stop 2023-10-26T06:49:39.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.855+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3251319535 shutdown_connections 2023-10-26T06:49:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.855+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3251319535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b48113940 0x7f2b48115d30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.855+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3251319535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 0x7f2b48107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.855+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3251319535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48108110 0x7f2b48113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.858 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.855+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3251319535 >> 172.21.15.89:0/3251319535 conn(0x7f2b4806bc20 msgr2=0x7f2b4806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:39.859 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.855+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3251319535 shutdown_connections 2023-10-26T06:49:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.855+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3251319535 wait complete. 2023-10-26T06:49:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.856+0000 7f2b4d7ff700 1 Processor -- start 2023-10-26T06:49:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.856+0000 7f2b4d7ff700 1 -- start start 2023-10-26T06:49:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.857+0000 7f2b4d7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 0x7f2b481ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.860 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.857+0000 7f2b4d7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b48108110 0x7f2b481aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.857+0000 7f2b4d7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48113940 0x7f2b481a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.857+0000 7f2b4d7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481a5d30 con 0x7f2b48113940 2023-10-26T06:49:39.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.857+0000 7f2b4d7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481a5ea0 con 0x7f2b481077f0 2023-10-26T06:49:39.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.857+0000 7f2b4d7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481a6060 con 0x7f2b48108110 2023-10-26T06:49:39.861 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.857+0000 7f2b46ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 0x7f2b481ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b46ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 0x7f2b481ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:38518/0 (socket says 172.21.15.89:38518) 2023-10-26T06:49:39.862 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b46ffd700 1 -- 172.21.15.89:0/3201945518 learned_addr learned my addr 172.21.15.89:0/3201945518 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:39.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b477fe700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48113940 0x7f2b481a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b46ffd700 1 -- 172.21.15.89:0/3201945518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b48108110 msgr2=0x7f2b481aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.863 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b48108110 0x7f2b481aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b46ffd700 1 -- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48113940 msgr2=0x7f2b481a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48113940 0x7f2b481a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.858+0000 7f2b46ffd700 1 -- 172.21.15.89:0/3201945518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40009c40 con 0x7f2b481077f0 2023-10-26T06:49:39.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.859+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 0x7f2b481ab550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000e9c0 tx=0x7f2b3000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.859+0000 7f2b3ffff700 1 -- 172.21.15.89:0/3201945518 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000cbd0 con 0x7f2b481077f0 2023-10-26T06:49:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.859+0000 7f2b3ffff700 1 -- 172.21.15.89:0/3201945518 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b3000cd30 con 0x7f2b481077f0 2023-10-26T06:49:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.859+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b481a6340 con 0x7f2b481077f0 2023-10-26T06:49:39.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.859+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b481b2540 con 0x7f2b481077f0 2023-10-26T06:49:39.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.860+0000 7f2b3ffff700 1 -- 172.21.15.89:0/3201945518 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b30005580 con 0x7f2b481077f0 2023-10-26T06:49:39.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.863+0000 7f2b3ffff700 1 -- 172.21.15.89:0/3201945518 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2b30004750 con 0x7f2b481077f0 2023-10-26T06:49:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.865+0000 7f2b3ffff700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b340a1740 0x7f2b340a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.865+0000 7f2b3ffff700 1 -- 172.21.15.89:0/3201945518 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2b30014070 con 0x7f2b481077f0 2023-10-26T06:49:39.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.866+0000 7f2b467fc700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b340a1740 0x7f2b340a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.866+0000 7f2b467fc700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b340a1740 0x7f2b340a3c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2b481a6d90 tx=0x7f2b38005bf0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:39.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.870+0000 7f2b61163b80 1 Processor -- start 2023-10-26T06:49:39.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b61163b80 1 -- start start 2023-10-26T06:49:39.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 0x5575c9014040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9014580 0x5575c900e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c900ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b61163b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c8f81760 con 0x5575c900e650 2023-10-26T06:49:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b61163b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c8f67ce0 con 0x5575c9013c60 2023-10-26T06:49:39.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b61163b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c8f65600 con 0x5575c9014580 2023-10-26T06:49:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b467fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9014580 0x5575c900e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b467fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9014580 0x5575c900e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41614/0 (socket says 172.21.15.89:41614) 2023-10-26T06:49:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b467fc700 1 -- 172.21.15.89:0/3181163318 learned_addr learned my addr 172.21.15.89:0/3181163318 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.871+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 0x5575c9014040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b477fe700 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c900ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b467fc700 1 -- 172.21.15.89:0/3181163318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 msgr2=0x5575c9014040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b467fc700 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 0x5575c9014040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b467fc700 1 -- 172.21.15.89:0/3181163318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 msgr2=0x5575c900ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b467fc700 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c900ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b467fc700 1 -- 172.21.15.89:0/3181163318 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575c8f1e470 con 0x5575c9014580 2023-10-26T06:49:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b467fc700 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9014580 0x5575c900e110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800cb50 tx=0x7f2b3800f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673eae40eb4b53f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b3d7fa700 1 -- 172.21.15.89:0/3181163318 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800fcf0 con 0x5575c9014580 2023-10-26T06:49:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b3d7fa700 1 -- 172.21.15.89:0/3181163318 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b3800fe50 con 0x5575c9014580 2023-10-26T06:49:39.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.872+0000 7f2b3d7fa700 1 -- 172.21.15.89:0/3181163318 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38003cc0 con 0x5575c9014580 2023-10-26T06:49:39.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.873+0000 7f2b61163b80 1 -- 172.21.15.89:0/3181163318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9014580 msgr2=0x5575c900e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.873+0000 7f2b61163b80 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9014580 0x5575c900e110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800cb50 tx=0x7f2b3800f000 comp rx=0 tx=0).stop 2023-10-26T06:49:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.873+0000 7f2b61163b80 1 -- 172.21.15.89:0/3181163318 shutdown_connections 2023-10-26T06:49:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.873+0000 7f2b61163b80 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c900ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.873+0000 7f2b61163b80 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 0x5575c9014040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.873+0000 7f2b61163b80 1 --2- 172.21.15.89:0/3181163318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9014580 0x5575c900e110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.873+0000 7f2b61163b80 1 -- 172.21.15.89:0/3181163318 >> 172.21.15.89:0/3181163318 conn(0x5575c8f827a0 msgr2=0x5575c9008be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:39.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.874+0000 7f2b61163b80 1 -- 172.21.15.89:0/3181163318 shutdown_connections 2023-10-26T06:49:39.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.874+0000 7f2b61163b80 1 -- 172.21.15.89:0/3181163318 wait complete. 2023-10-26T06:49:39.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.874+0000 7f2b61163b80 1 -- start start 2023-10-26T06:49:39.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.875+0000 7f2b61163b80 1 Processor -- start 2023-10-26T06:49:39.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.875+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c900e650 0x5575c901a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.875+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c90177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.875+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 0x5575c9017db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.875+0000 7f2b61163b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c8f60ef0 con 0x5575c9014580 2023-10-26T06:49:39.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.875+0000 7f2b61163b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c8efd790 con 0x5575c900e650 2023-10-26T06:49:39.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.875+0000 7f2b61163b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c8efd430 con 0x5575c9013c60 2023-10-26T06:49:39.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c900e650 0x5575c901a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c900e650 0x5575c901a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:38530/0 (socket says 172.21.15.89:38530) 2023-10-26T06:49:39.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1161256938 learned_addr learned my addr 172.21.15.89:0/1161256938 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:39.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b467fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c90177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1161256938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 msgr2=0x5575c90177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c90177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1161256938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 msgr2=0x5575c9017db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 0x5575c9017db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.876+0000 7f2b46ffd700 1 -- 172.21.15.89:0/1161256938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3800c800 con 0x5575c900e650 2023-10-26T06:49:39.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b61163b80 10 client.0 _ll_register_callbacks cb 0x5575c8e47b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:39.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c900e650 0x5575c901a520 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b300965d0 tx=0x7f2b3009a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7bc4eaa78b38b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:39.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b2e7fc700 10 client.25772 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:49:39.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575c8f65f70 con 0x5575c900e650 2023-10-26T06:49:39.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1161256938 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300a2460 con 0x5575c900e650 2023-10-26T06:49:39.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1161256938 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2b300995c0 con 0x5575c900e650 2023-10-26T06:49:39.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1161256938 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b30099780 con 0x5575c900e650 2023-10-26T06:49:39.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b61163b80 10 client.25772 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:39.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.877+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=20}) v3 -- 0x5575c8d8be60 con 0x5575c900e650 2023-10-26T06:49:39.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1161256938 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b300b8250 con 0x5575c900e650 2023-10-26T06:49:39.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b61163b80 10 client.25772 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b61163b80 10 client.25772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b61163b80 20 client.25772 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c9038c90 0x5575c903b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b61163b80 4 client.25772 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575c8fdd930 con 0x5575c9038c90 2023-10-26T06:49:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.878+0000 7f2b467fc700 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c9038c90 0x5575c903b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.879+0000 7f2b2e7fc700 10 client.25772 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:39.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.879+0000 7f2b467fc700 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c9038c90 0x5575c903b0d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c9a1bc620f01c0 server_cookie=40d2015533642aa8 in_seq=0 out_seq=0 2023-10-26T06:49:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.881+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1161256938 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5575c8fdd930 con 0x5575c9038c90 2023-10-26T06:49:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.881+0000 7f2b2e7fc700 10 client.25772 handle_command_reply: tid=0 2023-10-26T06:49:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.881+0000 7f2b61163b80 1 client.25772 shutdown 2023-10-26T06:49:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c900e650 msgr2=0x5575c901a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c900e650 0x5575c901a520 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b300965d0 tx=0x7f2b3009a7d0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 shutdown_connections 2023-10-26T06:49:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 0x5575c9017db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c9038c90 0x5575c903b0d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c900e650 0x5575c901a520 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1161256938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c90177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 >> 172.21.15.89:0/1161256938 conn(0x5575c8f827a0 msgr2=0x5575c9008b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:39.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b2e7fc700 0 client.25772 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:39.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.882+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 shutdown_connections 2023-10-26T06:49:39.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.883+0000 7f2b61163b80 1 -- 172.21.15.89:0/1161256938 wait complete. 2023-10-26T06:49:39.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.884+0000 7f2b61163b80 20 client.25772 trim_cache size 0 max 16384 2023-10-26T06:49:39.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:39 smithi192 ceph-mon[131661]: pgmap v392: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 141 KiB/s rd, 198 MiB/s wr, 24.22k op/s 2023-10-26T06:49:39.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.918+0000 7f2b61163b80 1 Processor -- start 2023-10-26T06:49:39.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.918+0000 7f2b61163b80 1 -- start start 2023-10-26T06:49:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.921+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c900e650 0x5575c90166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.921+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 0x5575c9023e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.921+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 0x5575c901efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b61163b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c90091b0 con 0x5575c9014580 2023-10-26T06:49:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b61163b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c8de19a0 con 0x5575c9013c60 2023-10-26T06:49:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b61163b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c901f500 con 0x5575c900e650 2023-10-26T06:49:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c900e650 0x5575c90166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c900e650 0x5575c90166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46368/0 (socket says 172.21.15.89:46368) 2023-10-26T06:49:39.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 -- 172.21.15.89:0/2743404095 learned_addr learned my addr 172.21.15.89:0/2743404095 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 -- 172.21.15.89:0/2743404095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 msgr2=0x5575c9023e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/2743404095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 0x5575c9023e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 -- 172.21.15.89:0/2743404095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 msgr2=0x5575c901efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/2743404095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 0x5575c901efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.922+0000 7f2b46ffd700 1 -- 172.21.15.89:0/2743404095 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575c903fd40 con 0x5575c900e650 2023-10-26T06:49:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.923+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/2743404095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c900e650 0x5575c90166f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b3009a060 tx=0x7f2b300ae000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f267a6121bf501fc server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.923+0000 7f2b3dffb700 1 -- 172.21.15.89:0/2743404095 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300a2460 con 0x5575c900e650 2023-10-26T06:49:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.923+0000 7f2b3dffb700 1 -- 172.21.15.89:0/2743404095 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575c8f65f70 con 0x5575c900e650 2023-10-26T06:49:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.923+0000 7f2b3dffb700 1 -- 172.21.15.89:0/2743404095 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300995c0 con 0x5575c900e650 2023-10-26T06:49:39.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.924+0000 7f2b61163b80 1 -- 172.21.15.89:0/2743404095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c900e650 msgr2=0x5575c90166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.924+0000 7f2b61163b80 1 --2- 172.21.15.89:0/2743404095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c900e650 0x5575c90166f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2b3009a060 tx=0x7f2b300ae000 comp rx=0 tx=0).stop 2023-10-26T06:49:39.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.924+0000 7f2b61163b80 1 -- 172.21.15.89:0/2743404095 shutdown_connections 2023-10-26T06:49:39.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.925+0000 7f2b61163b80 1 --2- 172.21.15.89:0/2743404095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c9014580 0x5575c901efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.925+0000 7f2b61163b80 1 --2- 172.21.15.89:0/2743404095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c900e650 0x5575c90166f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.925+0000 7f2b61163b80 1 --2- 172.21.15.89:0/2743404095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9013c60 0x5575c9023e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.925+0000 7f2b61163b80 1 -- 172.21.15.89:0/2743404095 >> 172.21.15.89:0/2743404095 conn(0x5575c8f827a0 msgr2=0x5575c9023710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:39.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.926+0000 7f2b61163b80 1 -- 172.21.15.89:0/2743404095 shutdown_connections 2023-10-26T06:49:39.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.927+0000 7f2b61163b80 1 -- 172.21.15.89:0/2743404095 wait complete. 2023-10-26T06:49:39.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.928+0000 7f2b61163b80 1 -- start start 2023-10-26T06:49:39.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.929+0000 7f2b61163b80 1 Processor -- start 2023-10-26T06:49:39.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.930+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c901ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c901ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b61163b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9014580 0x5575c90106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b61163b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c9010db0 con 0x5575c900e650 2023-10-26T06:49:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b61163b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c9010f20 con 0x5575c9014580 2023-10-26T06:49:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b61163b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c90110e0 con 0x5575c9013c60 2023-10-26T06:49:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b61163b80 10 client.0 _ll_register_callbacks cb 0x5575c8e47b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:39.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b467fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c901ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b467fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c901ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46384/0 (socket says 172.21.15.89:46384) 2023-10-26T06:49:39.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b467fc700 1 -- 172.21.15.89:0/1770409774 learned_addr learned my addr 172.21.15.89:0/1770409774 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:39.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.931+0000 7f2b477fe700 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9014580 0x5575c90106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c901ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b477fe700 1 -- 172.21.15.89:0/1770409774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 msgr2=0x5575c901ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b477fe700 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c901ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b477fe700 1 -- 172.21.15.89:0/1770409774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 msgr2=0x5575c901ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:39.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b477fe700 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c901ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b477fe700 1 -- 172.21.15.89:0/1770409774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575c8d8be60 con 0x5575c9014580 2023-10-26T06:49:39.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b477fe700 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9014580 0x5575c90106a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000b6c0 tx=0x7f2b40009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f78d93c501b1fc server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:39.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.932+0000 7f2b2e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:49:39.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.933+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1770409774 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40007830 con 0x5575c9014580 2023-10-26T06:49:39.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.933+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b3800fe50 con 0x5575c9014580 2023-10-26T06:49:39.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.936+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1770409774 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2b40010050 con 0x5575c9014580 2023-10-26T06:49:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.936+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1770409774 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b40007990 con 0x5575c9014580 2023-10-26T06:49:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.936+0000 7f2b61163b80 10 client.25775 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:39.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.936+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5575c8d74ce0 con 0x5575c9014580 2023-10-26T06:49:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1770409774 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b3800fe50 con 0x5575c9014580 2023-10-26T06:49:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1770409774 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b400183f0 con 0x5575c9014580 2023-10-26T06:49:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b61163b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:39.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b61163b80 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:39.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b61163b80 20 client.25775 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:39.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c908ef80 0x5575c90913a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:39.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b61163b80 4 client.25775 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:49:39.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.937+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5575c8f59ed0 con 0x5575c908ef80 2023-10-26T06:49:39.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.938+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c908ef80 0x5575c90913a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:39.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.939+0000 7f2b46ffd700 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c908ef80 0x5575c90913a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f69a7ff2584bd6b server_cookie=d1151b7fbd2fd9f5 in_seq=0 out_seq=0 2023-10-26T06:49:39.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.939+0000 7f2b2e7fc700 10 client.25775 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:39.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.940+0000 7f2b2e7fc700 1 -- 172.21.15.89:0/1770409774 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5575c8f59ed0 con 0x5575c908ef80 2023-10-26T06:49:39.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.940+0000 7f2b2e7fc700 10 client.25775 handle_command_reply: tid=0 2023-10-26T06:49:39.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.940+0000 7f2b61163b80 1 client.25775 shutdown 2023-10-26T06:49:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.943+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9014580 msgr2=0x5575c90106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.943+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9014580 0x5575c90106a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000b6c0 tx=0x7f2b40009510 comp rx=0 tx=0).stop 2023-10-26T06:49:39.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 shutdown_connections 2023-10-26T06:49:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5575c900e650 0x5575c901ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5575c908ef80 0x5575c90913a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575c9014580 0x5575c90106a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 --2- 172.21.15.89:0/1770409774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5575c9013c60 0x5575c901ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 >> 172.21.15.89:0/1770409774 conn(0x5575c8f827a0 msgr2=0x5575c9002530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b2e7fc700 0 client.25775 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 shutdown_connections 2023-10-26T06:49:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 1 -- 172.21.15.89:0/1770409774 wait complete. 2023-10-26T06:49:39.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.944+0000 7f2b61163b80 20 client.25775 trim_cache size 0 max 16384 2023-10-26T06:49:39.949 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:49:39.949 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:49:39.949 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "00b74677-c6b5-4aee-87aa-fdbdc13b30cc", 2023-10-26T06:49:39.950 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:49:39.950 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:49:39.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b340a1740 msgr2=0x7f2b340a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b340a1740 0x7f2b340a3c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2b481a6d90 tx=0x7f2b38005bf0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 msgr2=0x7f2b481ab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:39.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 0x7f2b481ab550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000e9c0 tx=0x7f2b3000ed80 comp rx=0 tx=0).stop 2023-10-26T06:49:39.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 shutdown_connections 2023-10-26T06:49:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b340a1740 0x7f2b340a3c00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b48113940 0x7f2b481a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b481077f0 0x7f2b481ab550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 --2- 172.21.15.89:0/3201945518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b48108110 0x7f2b481aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:39.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.945+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 >> 172.21.15.89:0/3201945518 conn(0x7f2b4806bc20 msgr2=0x7f2b4806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.946+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 shutdown_connections 2023-10-26T06:49:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:39.946+0000 7f2b4d7ff700 1 -- 172.21.15.89:0/3201945518 wait complete. 2023-10-26T06:49:39.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:39 smithi135 ceph-mon[131403]: pgmap v392: 97 pgs: 97 active+clean; 18 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 141 KiB/s rd, 198 MiB/s wr, 24.22k op/s 2023-10-26T06:49:39.980 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:49:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.342+0000 7f6fc47ff700 1 -- 172.21.15.89:0/908128285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fbc108aa0 msgr2=0x7f6fbc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.342+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/908128285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fbc108aa0 0x7f6fbc108e80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009990 tx=0x7f6fb8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.343+0000 7f6fc47ff700 1 -- 172.21.15.89:0/908128285 shutdown_connections 2023-10-26T06:49:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.343+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/908128285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc0fed90 0x7f6fbc0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.343+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/908128285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fbc108aa0 0x7f6fbc108e80 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009990 tx=0x7f6fb8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.343+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/908128285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fbc109450 0x7f6fbc0fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.343+0000 7f6fc47ff700 1 -- 172.21.15.89:0/908128285 >> 172.21.15.89:0/908128285 conn(0x7f6fbc0757e0 msgr2=0x7f6fbc0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.343+0000 7f6fc47ff700 1 -- 172.21.15.89:0/908128285 shutdown_connections 2023-10-26T06:49:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.343+0000 7f6fc47ff700 1 -- 172.21.15.89:0/908128285 wait complete. 2023-10-26T06:49:40.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 Processor -- start 2023-10-26T06:49:40.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 -- start start 2023-10-26T06:49:40.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fbc0fed90 0x7f6fbc072010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc072550 0x7f6fbc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fbc06b670 0x7f6fbc06baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc06fd70 con 0x7f6fbc0fed90 2023-10-26T06:49:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc06fee0 con 0x7f6fbc072550 2023-10-26T06:49:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.345+0000 7f6fc47ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0700a0 con 0x7f6fbc06b670 2023-10-26T06:49:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc072550 0x7f6fbc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc072550 0x7f6fbc06b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37028/0 (socket says 172.21.15.89:37028) 2023-10-26T06:49:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/1725231355 learned_addr learned my addr 172.21.15.89:0/1725231355 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/1725231355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fbc06b670 msgr2=0x7f6fbc06baf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T06:49:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fbc06b670 0x7f6fbc06baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fbc0fed90 msgr2=0x7f6fbc072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fbc0fed90 0x7f6fbc072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.346+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/1725231355 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80095f0 con 0x7f6fbc072550 2023-10-26T06:49:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.347+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc072550 0x7f6fbc06b070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c9b0 tx=0x7f6fac00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.347+0000 7f6fb37fe700 1 -- 172.21.15.89:0/1725231355 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac007aa0 con 0x7f6fbc072550 2023-10-26T06:49:40.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.347+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc070380 con 0x7f6fbc072550 2023-10-26T06:49:40.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.347+0000 7f6fb37fe700 1 -- 172.21.15.89:0/1725231355 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fac007c00 con 0x7f6fbc072550 2023-10-26T06:49:40.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.347+0000 7f6fb37fe700 1 -- 172.21.15.89:0/1725231355 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac018660 con 0x7f6fbc072550 2023-10-26T06:49:40.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.347+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc0708e0 con 0x7f6fbc072550 2023-10-26T06:49:40.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.354+0000 7f6fb37fe700 1 -- 172.21.15.89:0/1725231355 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6fac00f450 con 0x7f6fbc072550 2023-10-26T06:49:40.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.357+0000 7f6fb37fe700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6fa80a1670 0x7f6fa80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.357+0000 7f6fb37fe700 1 -- 172.21.15.89:0/1725231355 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f6fac0cd0d0 con 0x7f6fbc072550 2023-10-26T06:49:40.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.357+0000 7f6fc259b700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6fa80a1670 0x7f6fa80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.359+0000 7f6fc259b700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6fa80a1670 0x7f6fa80a3b30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8006010 tx=0x7f6fb800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.360+0000 7f6fd8200b80 1 Processor -- start 2023-10-26T06:49:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.361+0000 7f6fd8200b80 1 -- start start 2023-10-26T06:49:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.361+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b08fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b03a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98a766e0 con 0x564a98b08be0 2023-10-26T06:49:40.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98a5cc60 con 0x564a98b09500 2023-10-26T06:49:40.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98a5a580 con 0x564a98b035d0 2023-10-26T06:49:40.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fc1d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fc1d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b03090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37042/0 (socket says 172.21.15.89:37042) 2023-10-26T06:49:40.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/647326842 learned_addr learned my addr 172.21.15.89:0/647326842 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:40.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.362+0000 7f6fc259b700 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b08fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b03a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/647326842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 msgr2=0x564a98b03a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b03a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/647326842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 msgr2=0x564a98b08fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b08fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc1d9a700 1 -- 172.21.15.89:0/647326842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a98a133f0 con 0x564a98b09500 2023-10-26T06:49:40.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b03a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:40.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc259b700 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b08fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:40.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.363+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b03090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0965f0 tx=0x7f6fac09a930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c70b7b706ae3cf server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:40.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.364+0000 7f6fb0ff9700 1 -- 172.21.15.89:0/647326842 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac098490 con 0x564a98b09500 2023-10-26T06:49:40.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.364+0000 7f6fb0ff9700 1 -- 172.21.15.89:0/647326842 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fac096800 con 0x564a98b09500 2023-10-26T06:49:40.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.364+0000 7f6fb0ff9700 1 -- 172.21.15.89:0/647326842 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0995e0 con 0x564a98b09500 2023-10-26T06:49:40.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.365+0000 7f6fd8200b80 1 -- 172.21.15.89:0/647326842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 msgr2=0x564a98b03090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.365+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b03090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0965f0 tx=0x7f6fac09a930 comp rx=0 tx=0).stop 2023-10-26T06:49:40.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.366+0000 7f6fd8200b80 1 -- 172.21.15.89:0/647326842 shutdown_connections 2023-10-26T06:49:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.366+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b08fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.366+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b03a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.366+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/647326842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b03090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.366+0000 7f6fd8200b80 1 -- 172.21.15.89:0/647326842 >> 172.21.15.89:0/647326842 conn(0x564a98a77720 msgr2=0x564a98afdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:40.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.366+0000 7f6fd8200b80 1 -- 172.21.15.89:0/647326842 shutdown_connections 2023-10-26T06:49:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.366+0000 7f6fd8200b80 1 -- 172.21.15.89:0/647326842 wait complete. 2023-10-26T06:49:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.367+0000 7f6fd8200b80 1 -- start start 2023-10-26T06:49:40.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.367+0000 7f6fd8200b80 1 Processor -- start 2023-10-26T06:49:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.369+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b035d0 0x564a98b0f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.369+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b08be0 0x564a98b0c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.369+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b0cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.369+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98a55e70 con 0x564a98b035d0 2023-10-26T06:49:40.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.369+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a989f2710 con 0x564a98b08be0 2023-10-26T06:49:40.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.369+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a989f23b0 con 0x564a98b09500 2023-10-26T06:49:40.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.369+0000 7f6fd8200b80 10 client.0 _ll_register_callbacks cb 0x564a9893ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:40.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.370+0000 7f6fc259b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b035d0 0x564a98b0f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc259b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b035d0 0x564a98b0f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59588/0 (socket says 172.21.15.89:59588) 2023-10-26T06:49:40.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc259b700 1 -- 172.21.15.89:0/3229155546 learned_addr learned my addr 172.21.15.89:0/3229155546 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:40.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b08be0 0x564a98b0c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b0cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc259b700 1 -- 172.21.15.89:0/3229155546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 msgr2=0x564a98b0cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc259b700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b0cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc259b700 1 -- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b08be0 msgr2=0x564a98b0c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc259b700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b08be0 0x564a98b0c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.371+0000 7f6fc259b700 1 -- 172.21.15.89:0/3229155546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc0708e0 con 0x564a98b035d0 2023-10-26T06:49:40.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.372+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b0cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:40.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.373+0000 7f6fc259b700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b035d0 0x564a98b0f520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc066d30 tx=0x7f6fb8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a287c22af39c93b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:40.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.373+0000 7f6fa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:49:40.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.373+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/3229155546 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb801d070 con 0x564a98b035d0 2023-10-26T06:49:40.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.373+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a98a5aef0 con 0x564a98b035d0 2023-10-26T06:49:40.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.374+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/3229155546 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f6fb800be80 con 0x564a98b035d0 2023-10-26T06:49:40.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.375+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/3229155546 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fb8003e20 con 0x564a98b035d0 2023-10-26T06:49:40.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.375+0000 7f6fd8200b80 10 client.15777 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.375+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564a98880de0 con 0x564a98b035d0 2023-10-26T06:49:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.376+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/3229155546 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a98a5aef0 con 0x564a98b035d0 2023-10-26T06:49:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.377+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/3229155546 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fb800f460 con 0x564a98b035d0 2023-10-26T06:49:40.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.377+0000 7f6fd8200b80 10 client.15777 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.377+0000 7f6fd8200b80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.377+0000 7f6fd8200b80 20 client.15777 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.377+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b1cd00 0x564a98b1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.377+0000 7f6fd8200b80 4 client.15777 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:49:40.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.377+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a98ad28b0 con 0x564a98b1cd00 2023-10-26T06:49:40.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.379+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b1cd00 0x564a98b1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.380+0000 7f6fa5ffb700 10 client.15777 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:40.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.380+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b1cd00 0x564a98b1f120 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3b0933a8c7ab43 server_cookie=13708ae5359bec6f in_seq=0 out_seq=0 2023-10-26T06:49:40.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.381+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/3229155546 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a98ad28b0 con 0x564a98b1cd00 2023-10-26T06:49:40.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.381+0000 7f6fa5ffb700 10 client.15777 handle_command_reply: tid=0 2023-10-26T06:49:40.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.381+0000 7f6fd8200b80 1 client.15777 shutdown 2023-10-26T06:49:40.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.382+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b035d0 msgr2=0x564a98b0f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.382+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b035d0 0x564a98b0f520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc066d30 tx=0x7f6fb8009310 comp rx=0 tx=0).stop 2023-10-26T06:49:40.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.383+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 shutdown_connections 2023-10-26T06:49:40.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.383+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b035d0 0x564a98b0f520 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.383+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b1cd00 0x564a98b1f120 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.383+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b0cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.383+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/3229155546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b08be0 0x564a98b0c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.383+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 >> 172.21.15.89:0/3229155546 conn(0x564a98a77720 msgr2=0x564a98b02080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:40.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.384+0000 7f6fa5ffb700 0 client.15777 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:40.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.384+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 shutdown_connections 2023-10-26T06:49:40.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.384+0000 7f6fd8200b80 1 -- 172.21.15.89:0/3229155546 wait complete. 2023-10-26T06:49:40.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.384+0000 7f6fd8200b80 20 client.15777 trim_cache size 0 max 16384 2023-10-26T06:49:40.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.420+0000 7f6fd8200b80 1 Processor -- start 2023-10-26T06:49:40.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.420+0000 7f6fd8200b80 1 -- start start 2023-10-26T06:49:40.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.421+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b06f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.421+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b0c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.421+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b13dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.421+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a988d6920 con 0x564a98b08be0 2023-10-26T06:49:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.421+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98ad28b0 con 0x564a98b09500 2023-10-26T06:49:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.421+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98b0d7e0 con 0x564a98b035d0 2023-10-26T06:49:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.422+0000 7f6fc2d9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b13dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.422+0000 7f6fc259b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b06f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.422+0000 7f6fc2d9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b13dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37054/0 (socket says 172.21.15.89:37054) 2023-10-26T06:49:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.422+0000 7f6fc2d9c700 1 -- 172.21.15.89:0/2522721916 learned_addr learned my addr 172.21.15.89:0/2522721916 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:40.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fc2d9c700 1 -- 172.21.15.89:0/2522721916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 msgr2=0x564a98b06f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/2522721916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b06f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fc2d9c700 1 -- 172.21.15.89:0/2522721916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 msgr2=0x564a98b0c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/2522721916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b0c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fc2d9c700 1 -- 172.21.15.89:0/2522721916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a98869c60 con 0x564a98b09500 2023-10-26T06:49:40.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/2522721916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b13dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007f10 tx=0x7f6fb400e5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7efc23d5be2006 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fb17fa700 1 -- 172.21.15.89:0/2522721916 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400ec00 con 0x564a98b09500 2023-10-26T06:49:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.423+0000 7f6fb17fa700 1 -- 172.21.15.89:0/2522721916 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fb400ed60 con 0x564a98b09500 2023-10-26T06:49:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.424+0000 7f6fb17fa700 1 -- 172.21.15.89:0/2522721916 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40107a0 con 0x564a98b09500 2023-10-26T06:49:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.425+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2522721916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 msgr2=0x564a98b13dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.425+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2522721916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b13dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007f10 tx=0x7f6fb400e5e0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.427+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2522721916 shutdown_connections 2023-10-26T06:49:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.427+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2522721916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b09500 0x564a98b13dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.427+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2522721916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b035d0 0x564a98b06f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.428+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2522721916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b0c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.428+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2522721916 >> 172.21.15.89:0/2522721916 conn(0x564a98a77720 msgr2=0x564a98b18510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.428+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2522721916 shutdown_connections 2023-10-26T06:49:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.428+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2522721916 wait complete. 2023-10-26T06:49:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.429+0000 7f6fd8200b80 1 -- start start 2023-10-26T06:49:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.430+0000 7f6fd8200b80 1 Processor -- start 2023-10-26T06:49:40.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.433+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b035d0 0x564a98b6cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.433+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b6d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.433+0000 7f6fd8200b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b6d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.433+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98b12460 con 0x564a98b08be0 2023-10-26T06:49:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.433+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98b125d0 con 0x564a98b035d0 2023-10-26T06:49:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.433+0000 7f6fd8200b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98b12790 con 0x564a98b09500 2023-10-26T06:49:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.433+0000 7f6fd8200b80 10 client.0 _ll_register_callbacks cb 0x564a9893ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:49:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.434+0000 7f6fc259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b035d0 0x564a98b6cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.434+0000 7f6fc2d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b6d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.434+0000 7f6fc259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b035d0 0x564a98b6cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37064/0 (socket says 172.21.15.89:37064) 2023-10-26T06:49:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.434+0000 7f6fc259b700 1 -- 172.21.15.89:0/2050730625 learned_addr learned my addr 172.21.15.89:0/2050730625 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:49:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.435+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b6d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.437+0000 7f6fc259b700 1 -- 172.21.15.89:0/2050730625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 msgr2=0x564a98b6d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.437+0000 7f6fc259b700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b6d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.437+0000 7f6fc259b700 1 -- 172.21.15.89:0/2050730625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 msgr2=0x564a98b6d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:49:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.437+0000 7f6fc259b700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b6d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.437+0000 7f6fc259b700 1 -- 172.21.15.89:0/2050730625 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb4007b40 con 0x564a98b035d0 2023-10-26T06:49:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.437+0000 7f6fc2d9c700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b6d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:49:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.437+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b6d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:49:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.438+0000 7f6fc259b700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b035d0 0x564a98b6cb20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc066d30 tx=0x7f6fb8005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b9dbe0633dc855 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:49:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.438+0000 7f6fa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:49:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.438+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/2050730625 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb801d070 con 0x564a98b035d0 2023-10-26T06:49:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.438+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/2050730625 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f6fb800f460 con 0x564a98b035d0 2023-10-26T06:49:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.439+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fac096800 con 0x564a98b035d0 2023-10-26T06:49:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.440+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/2050730625 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fb800f920 con 0x564a98b035d0 2023-10-26T06:49:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.441+0000 7f6fd8200b80 10 client.25790 fetch_fsmap learned FSMap version 20 2023-10-26T06:49:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.441+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564a988a7250 con 0x564a98b035d0 2023-10-26T06:49:40.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/2050730625 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fac096800 con 0x564a98b035d0 2023-10-26T06:49:40.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/2050730625 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6fb802dd20 con 0x564a98b035d0 2023-10-26T06:49:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fd8200b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:49:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fd8200b80 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:49:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fd8200b80 20 client.25790 populate_metadata read hostname 'smithi089' 2023-10-26T06:49:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b83160 0x564a98b85580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:49:40.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fd8200b80 4 client.25790 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:49:40.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.442+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a98a4ee50 con 0x564a98b83160 2023-10-26T06:49:40.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.443+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b83160 0x564a98b85580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:49:40.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.444+0000 7f6fc1d9a700 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b83160 0x564a98b85580 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4725fb3f00bad3 server_cookie=30d29247ddb9d631 in_seq=0 out_seq=0 2023-10-26T06:49:40.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.444+0000 7f6fa5ffb700 10 client.25790 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:40.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.444+0000 7f6fa5ffb700 1 -- 172.21.15.89:0/2050730625 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x564a98a4ee50 con 0x564a98b83160 2023-10-26T06:49:40.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.444+0000 7f6fa5ffb700 10 client.25790 handle_command_reply: tid=0 2023-10-26T06:49:40.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.444+0000 7f6fd8200b80 1 client.25790 shutdown 2023-10-26T06:49:40.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b035d0 msgr2=0x564a98b6cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b035d0 0x564a98b6cb20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc066d30 tx=0x7f6fb8005c70 comp rx=0 tx=0).stop 2023-10-26T06:49:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 shutdown_connections 2023-10-26T06:49:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564a98b83160 0x564a98b85580 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564a98b09500 0x564a98b6d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a98b035d0 0x564a98b6cb20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fa5ffb700 0 client.25790 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:49:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 --2- 172.21.15.89:0/2050730625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564a98b08be0 0x564a98b6d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.445+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 >> 172.21.15.89:0/2050730625 conn(0x564a98a77720 msgr2=0x564a98b0b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.446+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 shutdown_connections 2023-10-26T06:49:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.446+0000 7f6fd8200b80 1 -- 172.21.15.89:0/2050730625 wait complete. 2023-10-26T06:49:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.446+0000 7f6fd8200b80 20 client.25790 trim_cache size 0 max 16384 2023-10-26T06:49:40.452 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:49:40.452 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "scrub active (192 inodes in the stack)", 2023-10-26T06:49:40.452 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:49:40.452 INFO:teuthology.orchestra.run.smithi089.stdout: "00b74677-c6b5-4aee-87aa-fdbdc13b30cc": { 2023-10-26T06:49:40.452 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:49:40.453 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "00b74677-c6b5-4aee-87aa-fdbdc13b30cc", 2023-10-26T06:49:40.453 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:49:40.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:49:40.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:49:40.453 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:49:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6fa80a1670 msgr2=0x7f6fa80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6fa80a1670 0x7f6fa80a3b30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8006010 tx=0x7f6fb800b540 comp rx=0 tx=0).stop 2023-10-26T06:49:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc072550 msgr2=0x7f6fbc06b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:49:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc072550 0x7f6fbc06b070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c9b0 tx=0x7f6fac00cd70 comp rx=0 tx=0).stop 2023-10-26T06:49:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 shutdown_connections 2023-10-26T06:49:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6fa80a1670 0x7f6fa80a3b30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fbc0fed90 0x7f6fbc072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fbc06b670 0x7f6fbc06baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 --2- 172.21.15.89:0/1725231355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fbc072550 0x7f6fbc06b070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:49:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 >> 172.21.15.89:0/1725231355 conn(0x7f6fbc0757e0 msgr2=0x7f6fbc106810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:49:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.449+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 shutdown_connections 2023-10-26T06:49:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:49:40.450+0000 7f6fc47ff700 1 -- 172.21.15.89:0/1725231355 wait complete. 2023-10-26T06:49:40.485 INFO:tasks.cephfs.filesystem:scrub status for tag:00b74677-c6b5-4aee-87aa-fdbdc13b30cc - {'path': '/', 'tag': '00b74677-c6b5-4aee-87aa-fdbdc13b30cc', 'options': 'recursive,force'} 2023-10-26T06:49:40.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:40 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:49:40.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:40 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:40.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:40 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:49:40.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:40 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:49:40.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:40 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:40.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:40 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:49:40.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:40 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:49:40.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:40 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:40.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:40 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:49:41.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:41 smithi089 ceph-mon[95031]: pgmap v393: 97 pgs: 97 active+clean; 18 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 141 KiB/s rd, 197 MiB/s wr, 24.48k op/s 2023-10-26T06:49:41.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:41 smithi192 ceph-mon[131661]: pgmap v393: 97 pgs: 97 active+clean; 18 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 141 KiB/s rd, 197 MiB/s wr, 24.48k op/s 2023-10-26T06:49:41.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:41 smithi135 ceph-mon[131403]: pgmap v393: 97 pgs: 97 active+clean; 18 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 141 KiB/s rd, 197 MiB/s wr, 24.48k op/s 2023-10-26T06:49:43.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:43 smithi089 ceph-mon[95031]: pgmap v394: 97 pgs: 97 active+clean; 18 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 152 KiB/s rd, 167 MiB/s wr, 21.35k op/s 2023-10-26T06:49:43.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:43 smithi192 ceph-mon[131661]: pgmap v394: 97 pgs: 97 active+clean; 18 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 152 KiB/s rd, 167 MiB/s wr, 21.35k op/s 2023-10-26T06:49:43.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:43 smithi135 ceph-mon[131403]: pgmap v394: 97 pgs: 97 active+clean; 18 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 152 KiB/s rd, 167 MiB/s wr, 21.35k op/s 2023-10-26T06:49:45.821 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:45 smithi089 ceph-mon[95031]: pgmap v395: 97 pgs: 97 active+clean; 18 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 31 KiB/s rd, 154 MiB/s wr, 19.59k op/s 2023-10-26T06:49:45.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:45 smithi192 ceph-mon[131661]: pgmap v395: 97 pgs: 97 active+clean; 18 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 31 KiB/s rd, 154 MiB/s wr, 19.59k op/s 2023-10-26T06:49:45.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:45 smithi135 ceph-mon[131403]: pgmap v395: 97 pgs: 97 active+clean; 18 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 31 KiB/s rd, 154 MiB/s wr, 19.59k op/s 2023-10-26T06:49:46.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:46 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:46.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:46 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:47.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:46 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:49:47.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:47 smithi192 ceph-mon[131661]: pgmap v396: 97 pgs: 97 active+clean; 19 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 31 KiB/s rd, 159 MiB/s wr, 20.44k op/s 2023-10-26T06:49:47.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:47 smithi135 ceph-mon[131403]: pgmap v396: 97 pgs: 97 active+clean; 19 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 31 KiB/s rd, 159 MiB/s wr, 20.44k op/s 2023-10-26T06:49:48.045 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:47 smithi089 ceph-mon[95031]: pgmap v396: 97 pgs: 97 active+clean; 19 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 31 KiB/s rd, 159 MiB/s wr, 20.44k op/s 2023-10-26T06:49:48.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:48 smithi192 ceph-mon[131661]: pgmap v397: 97 pgs: 97 active+clean; 19 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 89 KiB/s rd, 116 MiB/s wr, 15.07k op/s 2023-10-26T06:49:48.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:48 smithi135 ceph-mon[131403]: pgmap v397: 97 pgs: 97 active+clean; 19 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 89 KiB/s rd, 116 MiB/s wr, 15.07k op/s 2023-10-26T06:49:49.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:48 smithi089 ceph-mon[95031]: pgmap v397: 97 pgs: 97 active+clean; 19 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 89 KiB/s rd, 116 MiB/s wr, 15.07k op/s 2023-10-26T06:49:49.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:49 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:49:49.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:49 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:49:50.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:49 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:49:51.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:51 smithi135 ceph-mon[131403]: pgmap v398: 97 pgs: 97 active+clean; 19 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 174 KiB/s rd, 111 MiB/s wr, 15.29k op/s 2023-10-26T06:49:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:51 smithi089 ceph-mon[95031]: pgmap v398: 97 pgs: 97 active+clean; 19 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 174 KiB/s rd, 111 MiB/s wr, 15.29k op/s 2023-10-26T06:49:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:51 smithi192 ceph-mon[131661]: pgmap v398: 97 pgs: 97 active+clean; 19 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 174 KiB/s rd, 111 MiB/s wr, 15.29k op/s 2023-10-26T06:49:53.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:53 smithi135 ceph-mon[131403]: pgmap v399: 97 pgs: 97 active+clean; 19 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 174 KiB/s rd, 108 MiB/s wr, 15.37k op/s 2023-10-26T06:49:53.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:53 smithi089 ceph-mon[95031]: pgmap v399: 97 pgs: 97 active+clean; 19 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 174 KiB/s rd, 108 MiB/s wr, 15.37k op/s 2023-10-26T06:49:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:53 smithi192 ceph-mon[131661]: pgmap v399: 97 pgs: 97 active+clean; 19 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 174 KiB/s rd, 108 MiB/s wr, 15.37k op/s 2023-10-26T06:49:55.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:55 smithi135 ceph-mon[131403]: pgmap v400: 97 pgs: 97 active+clean; 19 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 281 KiB/s rd, 105 MiB/s wr, 15.19k op/s 2023-10-26T06:49:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:55 smithi089 ceph-mon[95031]: pgmap v400: 97 pgs: 97 active+clean; 19 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 281 KiB/s rd, 105 MiB/s wr, 15.19k op/s 2023-10-26T06:49:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:55 smithi192 ceph-mon[131661]: pgmap v400: 97 pgs: 97 active+clean; 19 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 281 KiB/s rd, 105 MiB/s wr, 15.19k op/s 2023-10-26T06:49:57.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:57 smithi089 ceph-mon[95031]: pgmap v401: 97 pgs: 97 active+clean; 19 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 316 KiB/s rd, 119 MiB/s wr, 16.70k op/s 2023-10-26T06:49:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:57 smithi192 ceph-mon[131661]: pgmap v401: 97 pgs: 97 active+clean; 19 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 316 KiB/s rd, 119 MiB/s wr, 16.70k op/s 2023-10-26T06:49:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:57 smithi135 ceph-mon[131403]: pgmap v401: 97 pgs: 97 active+clean; 19 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 316 KiB/s rd, 119 MiB/s wr, 16.70k op/s 2023-10-26T06:49:59.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:49:59 smithi089 ceph-mon[95031]: pgmap v402: 97 pgs: 97 active+clean; 19 GiB data, 63 GiB used, 1009 GiB / 1.0 TiB avail; 316 KiB/s rd, 118 MiB/s wr, 16.22k op/s 2023-10-26T06:49:59.650 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:49:59 smithi192 ceph-mon[131661]: pgmap v402: 97 pgs: 97 active+clean; 19 GiB data, 63 GiB used, 1009 GiB / 1.0 TiB avail; 316 KiB/s rd, 118 MiB/s wr, 16.22k op/s 2023-10-26T06:49:59.724 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:49:59 smithi135 ceph-mon[131403]: pgmap v402: 97 pgs: 97 active+clean; 19 GiB data, 63 GiB used, 1009 GiB / 1.0 TiB avail; 316 KiB/s rd, 118 MiB/s wr, 16.22k op/s 2023-10-26T06:50:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:00 smithi089 ceph-mon[95031]: overall HEALTH_OK 2023-10-26T06:50:00.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:00 smithi192 ceph-mon[131661]: overall HEALTH_OK 2023-10-26T06:50:00.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:00 smithi135 ceph-mon[131403]: overall HEALTH_OK 2023-10-26T06:50:01.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:01 smithi089 ceph-mon[95031]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 613 KiB/s rd, 137 MiB/s wr, 17.98k op/s 2023-10-26T06:50:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:01 smithi192 ceph-mon[131661]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 613 KiB/s rd, 137 MiB/s wr, 17.98k op/s 2023-10-26T06:50:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:01 smithi135 ceph-mon[131403]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 613 KiB/s rd, 137 MiB/s wr, 17.98k op/s 2023-10-26T06:50:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:03 smithi192 ceph-mon[131661]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 509 KiB/s rd, 142 MiB/s wr, 17.70k op/s 2023-10-26T06:50:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:03 smithi135 ceph-mon[131403]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 509 KiB/s rd, 142 MiB/s wr, 17.70k op/s 2023-10-26T06:50:03.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:03 smithi089 ceph-mon[95031]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 509 KiB/s rd, 142 MiB/s wr, 17.70k op/s 2023-10-26T06:50:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:05 smithi192 ceph-mon[131661]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 516 KiB/s rd, 139 MiB/s wr, 16.50k op/s 2023-10-26T06:50:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:05 smithi135 ceph-mon[131403]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 516 KiB/s rd, 139 MiB/s wr, 16.50k op/s 2023-10-26T06:50:05.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:05 smithi089 ceph-mon[95031]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 516 KiB/s rd, 139 MiB/s wr, 16.50k op/s 2023-10-26T06:50:07.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:07 smithi192 ceph-mon[131661]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 443 KiB/s rd, 154 MiB/s wr, 17.75k op/s 2023-10-26T06:50:07.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:07 smithi135 ceph-mon[131403]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 443 KiB/s rd, 154 MiB/s wr, 17.75k op/s 2023-10-26T06:50:07.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:07 smithi089 ceph-mon[95031]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 443 KiB/s rd, 154 MiB/s wr, 17.75k op/s 2023-10-26T06:50:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:09 smithi192 ceph-mon[131661]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 407 KiB/s rd, 146 MiB/s wr, 16.66k op/s 2023-10-26T06:50:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:09 smithi135 ceph-mon[131403]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 407 KiB/s rd, 146 MiB/s wr, 16.66k op/s 2023-10-26T06:50:09.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:09 smithi089 ceph-mon[95031]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 407 KiB/s rd, 146 MiB/s wr, 16.66k op/s 2023-10-26T06:50:10.486 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:50:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.920+0000 7f3f8bdff700 1 -- 172.21.15.89:0/1877021955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84102920 msgr2=0x7f3f84102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:10.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.920+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/1877021955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84102920 0x7f3f84102da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3f700098f0 tx=0x7f3f70009c00 comp rx=0 tx=0).stop 2023-10-26T06:50:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.920+0000 7f3f8bdff700 1 -- 172.21.15.89:0/1877021955 shutdown_connections 2023-10-26T06:50:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.921+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/1877021955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 0x7f3f8410f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.921+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/1877021955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f84108930 0x7f3f84108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.921+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/1877021955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84102920 0x7f3f84102da0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.921+0000 7f3f8bdff700 1 -- 172.21.15.89:0/1877021955 >> 172.21.15.89:0/1877021955 conn(0x7f3f840fe3b0 msgr2=0x7f3f841007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.921+0000 7f3f8bdff700 1 -- 172.21.15.89:0/1877021955 shutdown_connections 2023-10-26T06:50:10.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.922+0000 7f3f8bdff700 1 -- 172.21.15.89:0/1877021955 wait complete. 2023-10-26T06:50:10.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.926+0000 7f3f8bdff700 1 Processor -- start 2023-10-26T06:50:10.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.927+0000 7f3f8bdff700 1 -- start start 2023-10-26T06:50:10.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.928+0000 7f3f8bdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f84102920 0x7f3f84198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.928+0000 7f3f8bdff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 0x7f3f841992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.929+0000 7f3f8bdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84108930 0x7f3f8419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.929+0000 7f3f8939a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 0x7f3f841992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.929+0000 7f3f8939a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 0x7f3f841992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:49898/0 (socket says 172.21.15.89:49898) 2023-10-26T06:50:10.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.929+0000 7f3f8939a700 1 -- 172.21.15.89:0/3261754807 learned_addr learned my addr 172.21.15.89:0/3261754807 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:10.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.930+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8419dcb0 con 0x7f3f84108930 2023-10-26T06:50:10.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.930+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8419de20 con 0x7f3f841032e0 2023-10-26T06:50:10.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.930+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8419dfe0 con 0x7f3f84102920 2023-10-26T06:50:10.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.930+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f84102920 0x7f3f84198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.930+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84108930 0x7f3f8419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.931+0000 7f3f89b9b700 1 -- 172.21.15.89:0/3261754807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 msgr2=0x7f3f841992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:10.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.931+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 0x7f3f841992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.931+0000 7f3f89b9b700 1 -- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84108930 msgr2=0x7f3f8419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:10.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.931+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84108930 0x7f3f8419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.932+0000 7f3f89b9b700 1 -- 172.21.15.89:0/3261754807 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f70009520 con 0x7f3f84102920 2023-10-26T06:50:10.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.932+0000 7f3f8939a700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 0x7f3f841992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:50:10.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.932+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84108930 0x7f3f8419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:50:10.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.938+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f84102920 0x7f3f84198d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007fd0 tx=0x7f3f7800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.939+0000 7f3f82ffd700 1 -- 172.21.15.89:0/3261754807 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800ddd0 con 0x7f3f84102920 2023-10-26T06:50:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.939+0000 7f3f82ffd700 1 -- 172.21.15.89:0/3261754807 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7800f040 con 0x7f3f84102920 2023-10-26T06:50:10.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.939+0000 7f3f82ffd700 1 -- 172.21.15.89:0/3261754807 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78014600 con 0x7f3f84102920 2023-10-26T06:50:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.939+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8419e2c0 con 0x7f3f84102920 2023-10-26T06:50:10.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.939+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f841a5b00 con 0x7f3f84102920 2023-10-26T06:50:10.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.953+0000 7f3f82ffd700 1 -- 172.21.15.89:0/3261754807 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3f78015400 con 0x7f3f84102920 2023-10-26T06:50:10.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.955+0000 7f3f82ffd700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f740a1730 0x7f3f740a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.955+0000 7f3f82ffd700 1 -- 172.21.15.89:0/3261754807 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3f780cc9a0 con 0x7f3f84102920 2023-10-26T06:50:10.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.958+0000 7f3f8939a700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f740a1730 0x7f3f740a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.961+0000 7f3f8939a700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f740a1730 0x7f3f740a3bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f70000c00 tx=0x7f3f70005b40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:10.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.963+0000 7f3f9f80bb80 1 Processor -- start 2023-10-26T06:50:10.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.964+0000 7f3f9f80bb80 1 -- start start 2023-10-26T06:50:10.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea9be0 0x55a65aea9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aeaa500 0x55a65aea4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aea4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65ae176e0 con 0x55a65aeaa500 2023-10-26T06:50:10.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65adfdc60 con 0x55a65aea45d0 2023-10-26T06:50:10.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65adfb580 con 0x55a65aea9be0 2023-10-26T06:50:10.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f8a39c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aea4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f8a39c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aea4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:49906/0 (socket says 172.21.15.89:49906) 2023-10-26T06:50:10.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f8a39c700 1 -- 172.21.15.89:0/2593129989 learned_addr learned my addr 172.21.15.89:0/2593129989 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:10.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.965+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea9be0 0x55a65aea9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.966+0000 7f3f8939a700 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aeaa500 0x55a65aea4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.968+0000 7f3f8a39c700 1 -- 172.21.15.89:0/2593129989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea9be0 msgr2=0x55a65aea9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:10.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.968+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea9be0 0x55a65aea9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.968+0000 7f3f8a39c700 1 -- 172.21.15.89:0/2593129989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aeaa500 msgr2=0x55a65aea4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:10.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.968+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aeaa500 0x55a65aea4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.968+0000 7f3f8a39c700 1 -- 172.21.15.89:0/2593129989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a65adb43f0 con 0x55a65aea45d0 2023-10-26T06:50:10.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.968+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea9be0 0x55a65aea9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:50:10.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.970+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aea4a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00f570 tx=0x7f3f7c00f880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b05b92d5569e744e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:10.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.974+0000 7f3f6ffff700 1 -- 172.21.15.89:0/2593129989 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c010840 con 0x55a65aea45d0 2023-10-26T06:50:10.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.974+0000 7f3f6ffff700 1 -- 172.21.15.89:0/2593129989 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c010e60 con 0x55a65aea45d0 2023-10-26T06:50:10.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.975+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/2593129989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 msgr2=0x55a65aea4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:10.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.975+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aea4a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00f570 tx=0x7f3f7c00f880 comp rx=0 tx=0).stop 2023-10-26T06:50:10.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.975+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/2593129989 shutdown_connections 2023-10-26T06:50:10.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.975+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aea4a50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.975+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea9be0 0x55a65aea9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.976+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/2593129989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aeaa500 0x55a65aea4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.976+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/2593129989 >> 172.21.15.89:0/2593129989 conn(0x55a65ae18720 msgr2=0x55a65ae9eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:10.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.978+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/2593129989 shutdown_connections 2023-10-26T06:50:10.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.978+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/2593129989 wait complete. 2023-10-26T06:50:10.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.979+0000 7f3f9f80bb80 1 -- start start 2023-10-26T06:50:10.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.979+0000 7f3f9f80bb80 1 Processor -- start 2023-10-26T06:50:10.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.983+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea45d0 0x55a65aeb0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.983+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aead6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.983+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aeaa500 0x55a65aeadc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.983+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65adf6e70 con 0x55a65aea9be0 2023-10-26T06:50:10.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.983+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65ad93710 con 0x55a65aeaa500 2023-10-26T06:50:10.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.983+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65ad933b0 con 0x55a65aea45d0 2023-10-26T06:50:10.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.983+0000 7f3f9f80bb80 10 client.0 _ll_register_callbacks cb 0x55a65acdda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:10.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.986+0000 7f3f89b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea45d0 0x55a65aeb0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.986+0000 7f3f89b9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea45d0 0x55a65aeb0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44046/0 (socket says 172.21.15.89:44046) 2023-10-26T06:50:10.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.986+0000 7f3f89b9b700 1 -- 172.21.15.89:0/1070886098 learned_addr learned my addr 172.21.15.89:0/1070886098 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.987+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aeaa500 0x55a65aeadc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.987+0000 7f3f8939a700 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aead6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.987+0000 7f3f89b9b700 1 -- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aeaa500 msgr2=0x55a65aeadc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.987+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aeaa500 0x55a65aeadc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.987+0000 7f3f89b9b700 1 -- 172.21.15.89:0/1070886098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 msgr2=0x55a65aead6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.987+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aead6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.987+0000 7f3f89b9b700 1 -- 172.21.15.89:0/1070886098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f841a5b00 con 0x55a65aea45d0 2023-10-26T06:50:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.988+0000 7f3f6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:50:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.989+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea45d0 0x55a65aeb0380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800d7b0 tx=0x7f3f7809a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf79c9234b57bd server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:10.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.990+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/1070886098 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780b92b0 con 0x55a65aea45d0 2023-10-26T06:50:10.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.990+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/1070886098 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3f780b2200 con 0x55a65aea45d0 2023-10-26T06:50:10.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.990+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a65adfbef0 con 0x55a65aea45d0 2023-10-26T06:50:10.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.992+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/1070886098 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f780b92b0 con 0x55a65aea45d0 2023-10-26T06:50:10.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.992+0000 7f3f9f80bb80 10 client.25789 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:10.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.992+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f3f7c00ae70 con 0x55a65aea45d0 2023-10-26T06:50:10.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.992+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/1070886098 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f780a1cf0 con 0x55a65aea45d0 2023-10-26T06:50:10.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.993+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/1070886098 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3f78098ee0 con 0x55a65aea45d0 2023-10-26T06:50:10.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.993+0000 7f3f9f80bb80 10 client.25789 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:10.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.993+0000 7f3f9f80bb80 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:10.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.993+0000 7f3f9f80bb80 20 client.25789 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.993+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65aec6570 0x55a65aec89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.993+0000 7f3f9f80bb80 4 client.25789 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.993+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a65adfbef0 con 0x55a65aec6570 2023-10-26T06:50:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.994+0000 7f3f8939a700 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65aec6570 0x55a65aec89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:10.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.997+0000 7f3f8939a700 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65aec6570 0x55a65aec89d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3220ecd056b3dc7d server_cookie=33da08918f69a70b in_seq=0 out_seq=0 2023-10-26T06:50:10.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.997+0000 7f3f6d7fa700 10 client.25789 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:10.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.998+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/1070886098 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a65adfbef0 con 0x55a65aec6570 2023-10-26T06:50:11.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:10.999+0000 7f3f6d7fa700 10 client.25789 handle_command_reply: tid=0 2023-10-26T06:50:11.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.000+0000 7f3f9f80bb80 1 client.25789 shutdown 2023-10-26T06:50:11.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.000+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea45d0 msgr2=0x55a65aeb0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.000+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea45d0 0x55a65aeb0380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800d7b0 tx=0x7f3f7809a110 comp rx=0 tx=0).stop 2023-10-26T06:50:11.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.002+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 shutdown_connections 2023-10-26T06:50:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.002+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65aec6570 0x55a65aec89d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.002+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aeaa500 0x55a65aeadc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.002+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aea45d0 0x55a65aeb0380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.002+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/1070886098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aead6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.002+0000 7f3f6d7fa700 0 client.25789 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:11.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.002+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 >> 172.21.15.89:0/1070886098 conn(0x55a65ae18720 msgr2=0x55a65aea3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:11.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.003+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 shutdown_connections 2023-10-26T06:50:11.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.003+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/1070886098 wait complete. 2023-10-26T06:50:11.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.003+0000 7f3f9f80bb80 20 client.25789 trim_cache size 0 max 16384 2023-10-26T06:50:11.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.052+0000 7f3f9f80bb80 1 Processor -- start 2023-10-26T06:50:11.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.052+0000 7f3f9f80bb80 1 -- start start 2023-10-26T06:50:11.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.053+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aebd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.053+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aeafeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.053+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aea7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.053+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65ac77920 con 0x55a65aea9be0 2023-10-26T06:50:11.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.053+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65ae9f7a0 con 0x55a65aea45d0 2023-10-26T06:50:11.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.053+0000 7f3f9f80bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65aeaec70 con 0x55a65aeaa500 2023-10-26T06:50:11.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.054+0000 7f3f8939a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aeafeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.054+0000 7f3f8939a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aeafeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54996/0 (socket says 172.21.15.89:54996) 2023-10-26T06:50:11.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.054+0000 7f3f8939a700 1 -- 172.21.15.89:0/849865251 learned_addr learned my addr 172.21.15.89:0/849865251 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:11.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.055+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aebd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.055+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aea7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.056+0000 7f3f8a39c700 1 -- 172.21.15.89:0/849865251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 msgr2=0x55a65aebd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:11.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.056+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aebd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.056+0000 7f3f8a39c700 1 -- 172.21.15.89:0/849865251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 msgr2=0x55a65aeafeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:11.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.056+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aeafeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.056+0000 7f3f8a39c700 1 -- 172.21.15.89:0/849865251 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a65ac21de0 con 0x55a65aeaa500 2023-10-26T06:50:11.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.057+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aea7720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00fbd0 tx=0x7f3f7c00aac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584df18e69251fde server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:11.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.059+0000 7f3f80bff700 1 -- 172.21.15.89:0/849865251 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c018640 con 0x55a65aeaa500 2023-10-26T06:50:11.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.059+0000 7f3f80bff700 1 -- 172.21.15.89:0/849865251 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c010b00 con 0x55a65aeaa500 2023-10-26T06:50:11.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.059+0000 7f3f80bff700 1 -- 172.21.15.89:0/849865251 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0053d0 con 0x55a65aeaa500 2023-10-26T06:50:11.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.060+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/849865251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 msgr2=0x55a65aea7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.060+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aea7720 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00fbd0 tx=0x7f3f7c00aac0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.060+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/849865251 shutdown_connections 2023-10-26T06:50:11.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.060+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aea7720 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.060+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea45d0 0x55a65aebd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.060+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/849865251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea9be0 0x55a65aeafeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.060+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/849865251 >> 172.21.15.89:0/849865251 conn(0x55a65ae18720 msgr2=0x55a65aea3890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:11.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.063+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/849865251 shutdown_connections 2023-10-26T06:50:11.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.063+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/849865251 wait complete. 2023-10-26T06:50:11.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.064+0000 7f3f9f80bb80 1 -- start start 2023-10-26T06:50:11.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.065+0000 7f3f9f80bb80 1 Processor -- start 2023-10-26T06:50:11.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.066+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea45d0 0x55a65aebd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.069+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea9be0 0x55a65aebbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.068+0000 7f3f89b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea45d0 0x55a65aebd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.069+0000 7f3f9f80bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aebc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.069+0000 7f3f89b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea45d0 0x55a65aebd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55000/0 (socket says 172.21.15.89:55000) 2023-10-26T06:50:11.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.069+0000 7f3f89b9b700 1 -- 172.21.15.89:0/629228048 learned_addr learned my addr 172.21.15.89:0/629228048 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:11.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.070+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65aebc800 con 0x55a65aea45d0 2023-10-26T06:50:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.070+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65aebc970 con 0x55a65aea9be0 2023-10-26T06:50:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.069+0000 7f3f8939a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea9be0 0x55a65aebbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a65af1a3f0 con 0x55a65aeaa500 2023-10-26T06:50:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f9f80bb80 10 client.0 _ll_register_callbacks cb 0x55a65acdda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.070+0000 7f3f8a39c700 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aebc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f8939a700 1 -- 172.21.15.89:0/629228048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 msgr2=0x55a65aebc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f8939a700 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aebc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f8939a700 1 -- 172.21.15.89:0/629228048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea45d0 msgr2=0x55a65aebd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f8939a700 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea45d0 0x55a65aebd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f8939a700 1 -- 172.21.15.89:0/629228048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a65adb43f0 con 0x55a65aea9be0 2023-10-26T06:50:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.071+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea45d0 0x55a65aebd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:50:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.072+0000 7f3f8939a700 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea9be0 0x55a65aebbc50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a65aea9910 tx=0x7f3f70009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6da745f24dfdc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.072+0000 7f3f6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:50:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.074+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/629228048 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70032e50 con 0x55a65aea9be0 2023-10-26T06:50:11.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.075+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/629228048 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3f700322c0 con 0x55a65aea9be0 2023-10-26T06:50:11.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.075+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f7c010e60 con 0x55a65aea9be0 2023-10-26T06:50:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.076+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/629228048 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f70032480 con 0x55a65aea9be0 2023-10-26T06:50:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.076+0000 7f3f9f80bb80 10 client.25802 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.076+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a65ac0ac60 con 0x55a65aea9be0 2023-10-26T06:50:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/629228048 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c010e60 con 0x55a65aea9be0 2023-10-26T06:50:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/629228048 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3f7000f8e0 con 0x55a65aea9be0 2023-10-26T06:50:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f9f80bb80 10 client.25802 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f9f80bb80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f9f80bb80 20 client.25802 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65af1ba60 0x55a65af1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f9f80bb80 4 client.25802 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:50:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.078+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a65ae738b0 con 0x55a65af1ba60 2023-10-26T06:50:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.079+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65af1ba60 0x55a65af1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.081+0000 7f3f89b9b700 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65af1ba60 0x55a65af1de80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6037c68c5ae272f server_cookie=697b20567f2550f2 in_seq=0 out_seq=0 2023-10-26T06:50:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.081+0000 7f3f6d7fa700 10 client.25802 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:11.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.081+0000 7f3f6d7fa700 1 -- 172.21.15.89:0/629228048 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a65ae738b0 con 0x55a65af1ba60 2023-10-26T06:50:11.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.081+0000 7f3f6d7fa700 10 client.25802 handle_command_reply: tid=0 2023-10-26T06:50:11.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.081+0000 7f3f9f80bb80 1 client.25802 shutdown 2023-10-26T06:50:11.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.081+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea9be0 msgr2=0x55a65aebbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.081+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea9be0 0x55a65aebbc50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a65aea9910 tx=0x7f3f70009c00 comp rx=0 tx=0).stop 2023-10-26T06:50:11.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.082+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 shutdown_connections 2023-10-26T06:50:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.082+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a65aea45d0 0x55a65aebd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.082+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a65af1ba60 0x55a65af1de80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.082+0000 7f3f6d7fa700 0 client.25802 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.082+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a65aeaa500 0x55a65aebc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.082+0000 7f3f9f80bb80 1 --2- 172.21.15.89:0/629228048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a65aea9be0 0x55a65aebbc50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.082+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 >> 172.21.15.89:0/629228048 conn(0x55a65ae18720 msgr2=0x55a65aeabf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.083+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 shutdown_connections 2023-10-26T06:50:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.083+0000 7f3f9f80bb80 1 -- 172.21.15.89:0/629228048 wait complete. 2023-10-26T06:50:11.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.085+0000 7f3f9f80bb80 20 client.25802 trim_cache size 0 max 16384 2023-10-26T06:50:11.087 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:50:11.087 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:50:11.088 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:50:11.088 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:50:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f740a1730 msgr2=0x7f3f740a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f740a1730 0x7f3f740a3bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f70000c00 tx=0x7f3f70005b40 comp rx=0 tx=0).stop 2023-10-26T06:50:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f84102920 msgr2=0x7f3f84198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f84102920 0x7f3f84198d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007fd0 tx=0x7f3f7800d3b0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 shutdown_connections 2023-10-26T06:50:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3f740a1730 0x7f3f740a3bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f84108930 0x7f3f8419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f84102920 0x7f3f84198d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 --2- 172.21.15.89:0/3261754807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f841032e0 0x7f3f841992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.086+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 >> 172.21.15.89:0/3261754807 conn(0x7f3f840fe3b0 msgr2=0x7f3f840fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.087+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 shutdown_connections 2023-10-26T06:50:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.087+0000 7f3f8bdff700 1 -- 172.21.15.89:0/3261754807 wait complete. 2023-10-26T06:50:11.118 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:50:11.119 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:50:11.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/2685961696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb8096bb0 msgr2=0x7f0fb809a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/2685961696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb8096bb0 0x7f0fb809a9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0099c0 tx=0x7f0fac009cd0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/2685961696 shutdown_connections 2023-10-26T06:50:11.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/2685961696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fb8096200 0x7f0fb80965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/2685961696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb809b0f0 0x7f0fb80a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/2685961696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb8096bb0 0x7f0fb809a9c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/2685961696 >> 172.21.15.89:0/2685961696 conn(0x7f0fb800b920 msgr2=0x7f0fb800bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:11.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/2685961696 shutdown_connections 2023-10-26T06:50:11.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.470+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/2685961696 wait complete. 2023-10-26T06:50:11.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.473+0000 7f0fc5d9d700 1 Processor -- start 2023-10-26T06:50:11.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.473+0000 7f0fc5d9d700 1 -- start start 2023-10-26T06:50:11.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.473+0000 7f0fc5d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb8096200 0x7f0fb8129ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.474+0000 7f0fc5d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb809b0f0 0x7f0fb812a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.474+0000 7f0fc4d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb8096200 0x7f0fb8129ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.474+0000 7f0fc4d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb8096200 0x7f0fb8129ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:49936/0 (socket says 172.21.15.89:49936) 2023-10-26T06:50:11.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.474+0000 7f0fc4d9b700 1 -- 172.21.15.89:0/3002051899 learned_addr learned my addr 172.21.15.89:0/3002051899 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fbffff700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb809b0f0 0x7f0fb812a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fb812e8d0 0x7f0fb812ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb812f350 con 0x7f0fb812e8d0 2023-10-26T06:50:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb812f4c0 con 0x7f0fb8096200 2023-10-26T06:50:11.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb812f680 con 0x7f0fb809b0f0 2023-10-26T06:50:11.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc559c700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fb812e8d0 0x7f0fb812ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc4d9b700 1 -- 172.21.15.89:0/3002051899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb809b0f0 msgr2=0x7f0fb812a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc4d9b700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb809b0f0 0x7f0fb812a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc4d9b700 1 -- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fb812e8d0 msgr2=0x7f0fb812ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:11.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc4d9b700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fb812e8d0 0x7f0fb812ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fc4d9b700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac0095f0 con 0x7f0fb8096200 2023-10-26T06:50:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.475+0000 7f0fbffff700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb809b0f0 0x7f0fb812a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:50:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.476+0000 7f0fc4d9b700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb8096200 0x7f0fb8129ff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400d980 tx=0x7f0fb400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.476+0000 7f0fbdffb700 1 -- 172.21.15.89:0/3002051899 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4004220 con 0x7f0fb8096200 2023-10-26T06:50:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.476+0000 7f0fbdffb700 1 -- 172.21.15.89:0/3002051899 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb4007640 con 0x7f0fb8096200 2023-10-26T06:50:11.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.476+0000 7f0fbdffb700 1 -- 172.21.15.89:0/3002051899 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4003c20 con 0x7f0fb8096200 2023-10-26T06:50:11.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.476+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb8136be0 con 0x7f0fb8096200 2023-10-26T06:50:11.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.476+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb81370c0 con 0x7f0fb8096200 2023-10-26T06:50:11.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.478+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb809e380 con 0x7f0fb8096200 2023-10-26T06:50:11.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.482+0000 7f0fbdffb700 1 -- 172.21.15.89:0/3002051899 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0fb4004380 con 0x7f0fb8096200 2023-10-26T06:50:11.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.484+0000 7f0fbdffb700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0fb00a5ac0 0x7f0fb00a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:11.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.485+0000 7f0fbdffb700 1 -- 172.21.15.89:0/3002051899 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f0fb40ccb50 con 0x7f0fb8096200 2023-10-26T06:50:11.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.485+0000 7f0fbffff700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0fb00a5ac0 0x7f0fb00a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:11.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.485+0000 7f0fbdffb700 1 -- 172.21.15.89:0/3002051899 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0fb4095eb0 con 0x7f0fb8096200 2023-10-26T06:50:11.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.490+0000 7f0fbffff700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0fb00a5ac0 0x7f0fb00a7f80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0098e0 tx=0x7f0fac000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:11.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:11 smithi089 ceph-mon[95031]: pgmap v408: 97 pgs: 97 active+clean; 21 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 407 KiB/s rd, 171 MiB/s wr, 19.44k op/s 2023-10-26T06:50:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:11 smithi192 ceph-mon[131661]: pgmap v408: 97 pgs: 97 active+clean; 21 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 407 KiB/s rd, 171 MiB/s wr, 19.44k op/s 2023-10-26T06:50:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:11 smithi135 ceph-mon[131403]: pgmap v408: 97 pgs: 97 active+clean; 21 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 407 KiB/s rd, 171 MiB/s wr, 19.44k op/s 2023-10-26T06:50:11.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.988+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fb80974a0 con 0x7f0fb8096200 2023-10-26T06:50:11.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:11.995+0000 7f0fbdffb700 1 -- 172.21.15.89:0/3002051899 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f0fb4016020 con 0x7f0fb8096200 2023-10-26T06:50:11.999 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:50:11.999 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:50:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.001+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0fb00a5ac0 msgr2=0x7f0fb00a7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.001+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0fb00a5ac0 0x7f0fb00a7f80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0098e0 tx=0x7f0fac000bc0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.001+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb8096200 msgr2=0x7f0fb8129ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.001+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb8096200 0x7f0fb8129ff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400d980 tx=0x7f0fb400dc90 comp rx=0 tx=0).stop 2023-10-26T06:50:12.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 shutdown_connections 2023-10-26T06:50:12.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0fb00a5ac0 0x7f0fb00a7f80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fb812e8d0 0x7f0fb812ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fb8096200 0x7f0fb8129ff0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 --2- 172.21.15.89:0/3002051899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fb809b0f0 0x7f0fb812a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 >> 172.21.15.89:0/3002051899 conn(0x7f0fb800b920 msgr2=0x7f0fb8094150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 shutdown_connections 2023-10-26T06:50:12.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.002+0000 7f0fc5d9d700 1 -- 172.21.15.89:0/3002051899 wait complete. 2023-10-26T06:50:12.006 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:50:12.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:50:12.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.466+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4025135050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc067d40 msgr2=0x7fd4dc068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.466+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4025135050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc067d40 0x7fd4dc068120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80099c0 tx=0x7fd4d8009cd0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.467+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4025135050 shutdown_connections 2023-10-26T06:50:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.467+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4025135050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4dc105b30 0x7fd4dc10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.467+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4025135050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc067d40 0x7fd4dc068120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.467+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4025135050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc105170 0x7fd4dc1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.467+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4025135050 >> 172.21.15.89:0/4025135050 conn(0x7fd4dc0767f0 msgr2=0x7fd4dc076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:12.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.468+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4025135050 shutdown_connections 2023-10-26T06:50:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.468+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4025135050 wait complete. 2023-10-26T06:50:12.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.470+0000 7fd4e41ff700 1 Processor -- start 2023-10-26T06:50:12.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.472+0000 7fd4e41ff700 1 -- start start 2023-10-26T06:50:12.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.472+0000 7fd4e41ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4dc067d40 0x7fd4dc109760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e41ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc105170 0x7fd4dc109ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e41ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc105b30 0x7fd4dc10a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e41ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0722f0 con 0x7fd4dc067d40 2023-10-26T06:50:12.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e41ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc072460 con 0x7fd4dc105b30 2023-10-26T06:50:12.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e41ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc072620 con 0x7fd4dc105170 2023-10-26T06:50:12.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e1f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4dc067d40 0x7fd4dc109760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e179a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc105170 0x7fd4dc109ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e179a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc105170 0x7fd4dc109ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44104/0 (socket says 172.21.15.89:44104) 2023-10-26T06:50:12.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e179a700 1 -- 172.21.15.89:0/4163872050 learned_addr learned my addr 172.21.15.89:0/4163872050 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:12.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e1f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4dc067d40 0x7fd4dc109760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55044/0 (socket says 172.21.15.89:55044) 2023-10-26T06:50:12.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.473+0000 7fd4e279c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc105b30 0x7fd4dc10a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e179a700 1 -- 172.21.15.89:0/4163872050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc105b30 msgr2=0x7fd4dc10a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e179a700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc105b30 0x7fd4dc10a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e179a700 1 -- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4dc067d40 msgr2=0x7fd4dc109760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e179a700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4dc067d40 0x7fd4dc109760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e179a700 1 -- 172.21.15.89:0/4163872050 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc009520 con 0x7fd4dc105170 2023-10-26T06:50:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.474+0000 7fd4e279c700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc105b30 0x7fd4dc10a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:50:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.475+0000 7fd4e179a700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc105170 0x7fd4dc109ca0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00eb30 tx=0x7fd4cc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.475+0000 7fd4d2ffd700 1 -- 172.21.15.89:0/4163872050 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc009880 con 0x7fd4dc105170 2023-10-26T06:50:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.475+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d4009c40 con 0x7fd4dc105170 2023-10-26T06:50:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.475+0000 7fd4d2ffd700 1 -- 172.21.15.89:0/4163872050 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4cc0044e0 con 0x7fd4dc105170 2023-10-26T06:50:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.475+0000 7fd4d2ffd700 1 -- 172.21.15.89:0/4163872050 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc010430 con 0x7fd4dc105170 2023-10-26T06:50:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.475+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d80095f0 con 0x7fd4dc105170 2023-10-26T06:50:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.487+0000 7fd4d2ffd700 1 -- 172.21.15.89:0/4163872050 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd4cc003ee0 con 0x7fd4dc105170 2023-10-26T06:50:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.488+0000 7fd4f7ab4b80 1 Processor -- start 2023-10-26T06:50:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.489+0000 7fd4f7ab4b80 1 -- start start 2023-10-26T06:50:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.489+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9eac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eaca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9e1f6e0 con 0x55ebf9eb1be0 2023-10-26T06:50:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9e05c60 con 0x55ebf9eac5d0 2023-10-26T06:50:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9e03580 con 0x55ebf9eb2500 2023-10-26T06:50:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4e179a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eaca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4e179a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eaca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:49964/0 (socket says 172.21.15.89:49964) 2023-10-26T06:50:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4e179a700 1 -- 172.21.15.89:0/240775459 learned_addr learned my addr 172.21.15.89:0/240775459 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.490+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9eac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4e179a700 1 -- 172.21.15.89:0/240775459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 msgr2=0x55ebf9eac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4e179a700 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9eac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4e179a700 1 -- 172.21.15.89:0/240775459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb1be0 msgr2=0x55ebf9eb1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4d2ffd700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4c80a1670 0x7fd4c80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4e279c700 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4d2ffd700 1 -- 172.21.15.89:0/4163872050 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd4cc014070 con 0x7fd4dc105170 2023-10-26T06:50:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4e179a700 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.491+0000 7fd4e179a700 1 -- 172.21.15.89:0/240775459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebf9dbc3f0 con 0x55ebf9eac5d0 2023-10-26T06:50:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.492+0000 7fd4e279c700 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:50:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.492+0000 7fd4e179a700 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eaca50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cac00 tx=0x7fd4cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0642e59a0d1ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.492+0000 7fd4c7fff700 1 -- 172.21.15.89:0/240775459 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc096070 con 0x55ebf9eac5d0 2023-10-26T06:50:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.492+0000 7fd4c7fff700 1 -- 172.21.15.89:0/240775459 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4cc09a060 con 0x55ebf9eac5d0 2023-10-26T06:50:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.492+0000 7fd4c7fff700 1 -- 172.21.15.89:0/240775459 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc09ac20 con 0x55ebf9eac5d0 2023-10-26T06:50:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.494+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4c80a1670 0x7fd4c80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.494+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/240775459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 msgr2=0x55ebf9eaca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.494+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eaca50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cac00 tx=0x7fd4cc004750 comp rx=0 tx=0).stop 2023-10-26T06:50:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.495+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/240775459 shutdown_connections 2023-10-26T06:50:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.495+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.495+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eaca50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.495+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/240775459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9eac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.495+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/240775459 >> 172.21.15.89:0/240775459 conn(0x55ebf9e20720 msgr2=0x55ebf9ea6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.496+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/240775459 shutdown_connections 2023-10-26T06:50:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.496+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/240775459 wait complete. 2023-10-26T06:50:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.496+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4c80a1670 0x7fd4c80a3b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009fd0 tx=0x7fd4d8003680 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.497+0000 7fd4f7ab4b80 1 -- start start 2023-10-26T06:50:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.501+0000 7fd4f7ab4b80 1 Processor -- start 2023-10-26T06:50:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eb8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 0x55ebf9eb5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9dfee70 con 0x55ebf9eb2500 2023-10-26T06:50:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4e1f9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eb8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9d9b710 con 0x55ebf9eb1be0 2023-10-26T06:50:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9d9b3b0 con 0x55ebf9eac5d0 2023-10-26T06:50:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4e1f9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eb8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44122/0 (socket says 172.21.15.89:44122) 2023-10-26T06:50:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4e1f9b700 1 -- 172.21.15.89:0/1542534914 learned_addr learned my addr 172.21.15.89:0/1542534914 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.505+0000 7fd4f7ab4b80 10 client.0 _ll_register_callbacks cb 0x55ebf9ce5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.506+0000 7fd4e179a700 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.506+0000 7fd4e279c700 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 0x55ebf9eb5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.506+0000 7fd4e179a700 1 -- 172.21.15.89:0/1542534914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eac5d0 msgr2=0x55ebf9eb8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.506+0000 7fd4e179a700 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eb8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.506+0000 7fd4e179a700 1 -- 172.21.15.89:0/1542534914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 msgr2=0x55ebf9eb5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.506+0000 7fd4e179a700 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 0x55ebf9eb5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.506+0000 7fd4e179a700 1 -- 172.21.15.89:0/1542534914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d80095f0 con 0x55ebf9eb1be0 2023-10-26T06:50:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.507+0000 7fd4e179a700 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb56f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cac00 tx=0x7fd4cc0d0c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=558d72de1cac2c86 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.507+0000 7fd4c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:50:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.507+0000 7fd4c57fa700 1 -- 172.21.15.89:0/1542534914 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00cd40 con 0x55ebf9eb1be0 2023-10-26T06:50:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.508+0000 7fd4c57fa700 1 -- 172.21.15.89:0/1542534914 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd4cc09ac20 con 0x55ebf9eb1be0 2023-10-26T06:50:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.509+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebf9e03ef0 con 0x55ebf9eb1be0 2023-10-26T06:50:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.510+0000 7fd4c57fa700 1 -- 172.21.15.89:0/1542534914 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4cc09a520 con 0x55ebf9eb1be0 2023-10-26T06:50:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.510+0000 7fd4f7ab4b80 10 client.25817 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.510+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ebf9c29de0 con 0x55ebf9eb1be0 2023-10-26T06:50:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.510+0000 7fd4c57fa700 1 -- 172.21.15.89:0/1542534914 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ebf9e03ef0 con 0x55ebf9eb1be0 2023-10-26T06:50:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.510+0000 7fd4c57fa700 1 -- 172.21.15.89:0/1542534914 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd4cc099a10 con 0x55ebf9eb1be0 2023-10-26T06:50:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.510+0000 7fd4f7ab4b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.511+0000 7fd4f7ab4b80 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.511+0000 7fd4f7ab4b80 20 client.25817 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.511+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9ec5a50 0x55ebf9ec7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.511+0000 7fd4f7ab4b80 4 client.25817 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.511+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebf9e7b8b0 con 0x55ebf9ec5a50 2023-10-26T06:50:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.512+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9ec5a50 0x55ebf9ec7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.513+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9ec5a50 0x55ebf9ec7e70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19bf5b805d5e8be server_cookie=b26cec2750573e21 in_seq=0 out_seq=0 2023-10-26T06:50:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.514+0000 7fd4c57fa700 10 client.25817 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.514+0000 7fd4c57fa700 1 -- 172.21.15.89:0/1542534914 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ebf9e7b8b0 con 0x55ebf9ec5a50 2023-10-26T06:50:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.515+0000 7fd4c57fa700 10 client.25817 handle_command_reply: tid=0 2023-10-26T06:50:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.515+0000 7fd4f7ab4b80 1 client.25817 shutdown 2023-10-26T06:50:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 msgr2=0x55ebf9eb56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb56f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cac00 tx=0x7fd4cc0d0c20 comp rx=0 tx=0).stop 2023-10-26T06:50:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 shutdown_connections 2023-10-26T06:50:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 0x55ebf9eb5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9ec5a50 0x55ebf9ec7e70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eac5d0 0x55ebf9eb8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4c57fa700 0 client.25817 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.517+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/1542534914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9eb56f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.518+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 >> 172.21.15.89:0/1542534914 conn(0x55ebf9e20720 msgr2=0x55ebf9eab080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.518+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 shutdown_connections 2023-10-26T06:50:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.518+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/1542534914 wait complete. 2023-10-26T06:50:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.518+0000 7fd4f7ab4b80 20 client.25817 trim_cache size 0 max 16384 2023-10-26T06:50:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.554+0000 7fd4f7ab4b80 1 Processor -- start 2023-10-26T06:50:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.554+0000 7fd4f7ab4b80 1 -- start start 2023-10-26T06:50:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.555+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9ea15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.555+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.555+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 0x55ebf9eab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.555+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9c7f920 con 0x55ebf9eb2500 2023-10-26T06:50:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.555+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9eb8c90 con 0x55ebf9eac5d0 2023-10-26T06:50:12.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.555+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9ea83b0 con 0x55ebf9eb1be0 2023-10-26T06:50:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.556+0000 7fd4e1f9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9ea15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.556+0000 7fd4e1f9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9ea15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:49974/0 (socket says 172.21.15.89:49974) 2023-10-26T06:50:12.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.556+0000 7fd4e1f9b700 1 -- 172.21.15.89:0/30553048 learned_addr learned my addr 172.21.15.89:0/30553048 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.556+0000 7fd4e179a700 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.557+0000 7fd4e1f9b700 1 -- 172.21.15.89:0/30553048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb1be0 msgr2=0x55ebf9ebcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.557+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.557+0000 7fd4e1f9b700 1 -- 172.21.15.89:0/30553048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 msgr2=0x55ebf9eab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.557+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 0x55ebf9eab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.557+0000 7fd4e1f9b700 1 -- 172.21.15.89:0/30553048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebf9c12c60 con 0x55ebf9eac5d0 2023-10-26T06:50:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.557+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9ea15d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80098e0 tx=0x7fd4d801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef28db5d0a3e9aa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.558+0000 7fd4d0ff9700 1 -- 172.21.15.89:0/30553048 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d802b040 con 0x55ebf9eac5d0 2023-10-26T06:50:12.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.558+0000 7fd4d0ff9700 1 -- 172.21.15.89:0/30553048 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d801d040 con 0x55ebf9eac5d0 2023-10-26T06:50:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.558+0000 7fd4d0ff9700 1 -- 172.21.15.89:0/30553048 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8030ea0 con 0x55ebf9eac5d0 2023-10-26T06:50:12.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.559+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/30553048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 msgr2=0x55ebf9ea15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.559+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9ea15d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80098e0 tx=0x7fd4d801e000 comp rx=0 tx=0).stop 2023-10-26T06:50:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.560+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/30553048 shutdown_connections 2023-10-26T06:50:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.560+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9eb2500 0x55ebf9eab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.560+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eac5d0 0x55ebf9ea15d0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80098e0 tx=0x7fd4d801e000 comp rx=0 tx=0).stop 2023-10-26T06:50:12.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.560+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/30553048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.560+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/30553048 >> 172.21.15.89:0/30553048 conn(0x55ebf9e20720 msgr2=0x55ebf9eab080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.560+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/30553048 shutdown_connections 2023-10-26T06:50:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.560+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/30553048 wait complete. 2023-10-26T06:50:12.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.561+0000 7fd4f7ab4b80 1 -- start start 2023-10-26T06:50:12.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.563+0000 7fd4f7ab4b80 1 Processor -- start 2023-10-26T06:50:12.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.565+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.565+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9ebc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.565+0000 7fd4f7ab4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9ebcc20 0x55ebf9f19870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.565+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9f19e00 con 0x55ebf9ebcc20 2023-10-26T06:50:12.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.565+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9f19f70 con 0x55ebf9eb1be0 2023-10-26T06:50:12.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.565+0000 7fd4f7ab4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf9f1a130 con 0x55ebf9eb2500 2023-10-26T06:50:12.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.565+0000 7fd4f7ab4b80 10 client.0 _ll_register_callbacks cb 0x55ebf9ce5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:12.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e1f9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9ebc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9ebc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44134/0 (socket says 172.21.15.89:44134) 2023-10-26T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 -- 172.21.15.89:0/673547984 learned_addr learned my addr 172.21.15.89:0/673547984 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 -- 172.21.15.89:0/673547984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 msgr2=0x55ebf9ebc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 -- 172.21.15.89:0/673547984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9ebcc20 msgr2=0x55ebf9f19870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9ebcc20 0x55ebf9f19870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.566+0000 7fd4e179a700 1 -- 172.21.15.89:0/673547984 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc10de70 con 0x55ebf9eb2500 2023-10-26T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.567+0000 7fd4e179a700 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9ebc6e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0850b0 tx=0x7fd4cc0042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde5eac70b9cc426 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:12.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.567+0000 7fd4c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.567+0000 7fd4c57fa700 1 -- 172.21.15.89:0/673547984 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc096040 con 0x55ebf9eb2500 2023-10-26T06:50:12.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.567+0000 7fd4c57fa700 1 -- 172.21.15.89:0/673547984 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd4cc096660 con 0x55ebf9eb2500 2023-10-26T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.568+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4cc09a060 con 0x55ebf9eb2500 2023-10-26T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.568+0000 7fd4c57fa700 1 -- 172.21.15.89:0/673547984 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4cc099e80 con 0x55ebf9eb2500 2023-10-26T06:50:12.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.569+0000 7fd4f7ab4b80 10 client.25816 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.569+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ebf9c50250 con 0x55ebf9eb2500 2023-10-26T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.569+0000 7fd4c57fa700 1 -- 172.21.15.89:0/673547984 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4cc09a060 con 0x55ebf9eb2500 2023-10-26T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.569+0000 7fd4c57fa700 1 -- 172.21.15.89:0/673547984 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd4cc0a2420 con 0x55ebf9eb2500 2023-10-26T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.570+0000 7fd4f7ab4b80 10 client.25816 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.571+0000 7fd4f7ab4b80 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:12.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.571+0000 7fd4f7ab4b80 20 client.25816 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.571+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9f1b400 0x55ebf9f1d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.571+0000 7fd4f7ab4b80 4 client.25816 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.571+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ebf9df7e50 con 0x55ebf9f1b400 2023-10-26T06:50:12.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.571+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9f1b400 0x55ebf9f1d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:12.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.572+0000 7fd4e1f9b700 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9f1b400 0x55ebf9f1d7f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f311058f78c7d571 server_cookie=96eba4a6b56173a0 in_seq=0 out_seq=0 2023-10-26T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.572+0000 7fd4c57fa700 10 client.25816 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.572+0000 7fd4c57fa700 1 -- 172.21.15.89:0/673547984 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ebf9df7e50 con 0x55ebf9f1b400 2023-10-26T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.572+0000 7fd4c57fa700 10 client.25816 handle_command_reply: tid=0 2023-10-26T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.572+0000 7fd4f7ab4b80 1 client.25816 shutdown 2023-10-26T06:50:12.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 msgr2=0x55ebf9ebc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9ebc6e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0850b0 tx=0x7fd4cc0042c0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 shutdown_connections 2023-10-26T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ebf9ebcc20 0x55ebf9f19870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ebf9f1b400 0x55ebf9f1d7f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebf9eb1be0 0x55ebf9ebc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 --2- 172.21.15.89:0/673547984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ebf9eb2500 0x55ebf9ebc6e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4c57fa700 0 client.25816 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.573+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 >> 172.21.15.89:0/673547984 conn(0x55ebf9e20720 msgr2=0x55ebf9eb3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.574+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 shutdown_connections 2023-10-26T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.574+0000 7fd4f7ab4b80 1 -- 172.21.15.89:0/673547984 wait complete. 2023-10-26T06:50:12.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.574+0000 7fd4f7ab4b80 20 client.25816 trim_cache size 0 max 16384 2023-10-26T06:50:12.579 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.575+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4c80a1670 msgr2=0x7fd4c80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.575+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4c80a1670 0x7fd4c80a3b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009fd0 tx=0x7fd4d8003680 comp rx=0 tx=0).stop 2023-10-26T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.575+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc105170 msgr2=0x7fd4dc109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.575+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc105170 0x7fd4dc109ca0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00eb30 tx=0x7fd4cc00eef0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 shutdown_connections 2023-10-26T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4c80a1670 0x7fd4c80a3b30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4dc067d40 0x7fd4dc109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc105b30 0x7fd4dc10a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 --2- 172.21.15.89:0/4163872050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4dc105170 0x7fd4dc109ca0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 >> 172.21.15.89:0/4163872050 conn(0x7fd4dc0767f0 msgr2=0x7fd4dc066b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:12.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 shutdown_connections 2023-10-26T06:50:12.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:12.576+0000 7fd4e41ff700 1 -- 172.21.15.89:0/4163872050 wait complete. 2023-10-26T06:50:12.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:12 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3002051899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:50:12.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:12 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3002051899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:50:12.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:12 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3002051899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:50:13.614 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:50:13.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:50:13.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:13 smithi192 ceph-mon[131661]: pgmap v409: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 97 KiB/s rd, 159 MiB/s wr, 18.24k op/s 2023-10-26T06:50:13.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:13 smithi135 ceph-mon[131403]: pgmap v409: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 97 KiB/s rd, 159 MiB/s wr, 18.24k op/s 2023-10-26T06:50:13.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:13 smithi089 ceph-mon[95031]: pgmap v409: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 97 KiB/s rd, 159 MiB/s wr, 18.24k op/s 2023-10-26T06:50:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.992+0000 7f39ff3ff700 1 -- 172.21.15.89:0/1408449590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f81056e0 msgr2=0x7f39f8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.992+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/1408449590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f81056e0 0x7f39f8105ac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39f40098f0 tx=0x7f39f4009c00 comp rx=0 tx=0).stop 2023-10-26T06:50:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.993+0000 7f39ff3ff700 1 -- 172.21.15.89:0/1408449590 shutdown_connections 2023-10-26T06:50:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.993+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/1408449590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39f8100000 0x7f39f810c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:13.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.993+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/1408449590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f81056e0 0x7f39f8105ac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.993+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/1408449590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f80ff640 0x7f39f80ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.993+0000 7f39ff3ff700 1 -- 172.21.15.89:0/1408449590 >> 172.21.15.89:0/1408449590 conn(0x7f39f80767f0 msgr2=0x7f39f8076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:13.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.993+0000 7f39ff3ff700 1 -- 172.21.15.89:0/1408449590 shutdown_connections 2023-10-26T06:50:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.993+0000 7f39ff3ff700 1 -- 172.21.15.89:0/1408449590 wait complete. 2023-10-26T06:50:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.995+0000 7f39ff3ff700 1 Processor -- start 2023-10-26T06:50:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.996+0000 7f39ff3ff700 1 -- start start 2023-10-26T06:50:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.998+0000 7f39ff3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39f80ff640 0x7f39f819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.998+0000 7f39ff3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f8100000 0x7f39f819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.998+0000 7f39ff3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f81056e0 0x7f39f8194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.999+0000 7f39ff3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81954f0 con 0x7f39f81056e0 2023-10-26T06:50:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.999+0000 7f39ff3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8195660 con 0x7f39f8100000 2023-10-26T06:50:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.999+0000 7f39ff3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8195820 con 0x7f39f80ff640 2023-10-26T06:50:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.999+0000 7f39fd99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f81056e0 0x7f39f8194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:13.999+0000 7f39fc99a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f8100000 0x7f39f819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.000+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39f80ff640 0x7f39f819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.000+0000 7f39fd99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f81056e0 0x7f39f8194d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55084/0 (socket says 172.21.15.89:55084) 2023-10-26T06:50:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.000+0000 7f39fd99c700 1 -- 172.21.15.89:0/3584016942 learned_addr learned my addr 172.21.15.89:0/3584016942 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.001+0000 7f39fd99c700 1 -- 172.21.15.89:0/3584016942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39f80ff640 msgr2=0x7f39f819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.001+0000 7f39fd99c700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39f80ff640 0x7f39f819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.001+0000 7f39fd99c700 1 -- 172.21.15.89:0/3584016942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f8100000 msgr2=0x7f39f819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.001+0000 7f39fd99c700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f8100000 0x7f39f819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.001+0000 7f39fd99c700 1 -- 172.21.15.89:0/3584016942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4009520 con 0x7f39f81056e0 2023-10-26T06:50:14.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.003+0000 7f39fd99c700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f81056e0 0x7f39f8194d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f39f0007fd0 tx=0x7f39f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.006+0000 7f39ee7fc700 1 -- 172.21.15.89:0/3584016942 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0017070 con 0x7f39f81056e0 2023-10-26T06:50:14.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.006+0000 7f39ee7fc700 1 -- 172.21.15.89:0/3584016942 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f000f040 con 0x7f39f81056e0 2023-10-26T06:50:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.009+0000 7f39ee7fc700 1 -- 172.21.15.89:0/3584016942 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0005740 con 0x7f39f81056e0 2023-10-26T06:50:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.009+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8195b00 con 0x7f39f81056e0 2023-10-26T06:50:14.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.009+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81b27c0 con 0x7f39f81056e0 2023-10-26T06:50:14.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.018+0000 7f3a12d28b80 1 Processor -- start 2023-10-26T06:50:14.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.025+0000 7f39ee7fc700 1 -- 172.21.15.89:0/3584016942 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f39f00058a0 con 0x7f39f81056e0 2023-10-26T06:50:14.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.028+0000 7f39ee7fc700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f39e40a1650 0x7f39e40a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.031+0000 7f3a12d28b80 1 -- start start 2023-10-26T06:50:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.031+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f14040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.031+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 0x555a20f0e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.031+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f39fc99a700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f39e40a1650 0x7f39e40a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f14040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f14040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:50016/0 (socket says 172.21.15.89:50016) 2023-10-26T06:50:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20e81760 con 0x555a20f0e650 2023-10-26T06:50:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20e67ce0 con 0x555a20f13c60 2023-10-26T06:50:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20e65600 con 0x555a20f14580 2023-10-26T06:50:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f39ee7fc700 1 -- 172.21.15.89:0/3584016942 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f39f0099700 con 0x7f39f81056e0 2023-10-26T06:50:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.032+0000 7f39fd19b700 1 -- 172.21.15.89:0/3772987943 learned_addr learned my addr 172.21.15.89:0/3772987943 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.033+0000 7f39fd19b700 1 -- 172.21.15.89:0/3772987943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 msgr2=0x555a20f0e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.033+0000 7f39fd19b700 1 --2- 172.21.15.89:0/3772987943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 0x555a20f0e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.033+0000 7f39fd19b700 1 -- 172.21.15.89:0/3772987943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 msgr2=0x555a20f0ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.033+0000 7f39fd19b700 1 --2- 172.21.15.89:0/3772987943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.033+0000 7f39fd19b700 1 -- 172.21.15.89:0/3772987943 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a20e1e470 con 0x555a20f13c60 2023-10-26T06:50:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.034+0000 7f39fc99a700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f39e40a1650 0x7f39e40a3b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39e8006fd0 tx=0x7f39e8009d10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.034+0000 7f39fd19b700 1 --2- 172.21.15.89:0/3772987943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f14040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39f40098f0 tx=0x7f39f4010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1a75e613fb0943a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.034+0000 7f39e37fe700 1 -- 172.21.15.89:0/3772987943 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4010f10 con 0x555a20f13c60 2023-10-26T06:50:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.035+0000 7f39e37fe700 1 -- 172.21.15.89:0/3772987943 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f401d040 con 0x555a20f13c60 2023-10-26T06:50:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.035+0000 7f39e37fe700 1 -- 172.21.15.89:0/3772987943 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40052a0 con 0x555a20f13c60 2023-10-26T06:50:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 msgr2=0x555a20f14040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/3772987943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f14040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39f40098f0 tx=0x7f39f4010cf0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 shutdown_connections 2023-10-26T06:50:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/3772987943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f0ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/3772987943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f14040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/3772987943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 0x555a20f0e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 >> 172.21.15.89:0/3772987943 conn(0x555a20e827a0 msgr2=0x555a20f08be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 shutdown_connections 2023-10-26T06:50:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.036+0000 7f3a12d28b80 1 -- 172.21.15.89:0/3772987943 wait complete. 2023-10-26T06:50:14.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.037+0000 7f3a12d28b80 1 -- start start 2023-10-26T06:50:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.038+0000 7f3a12d28b80 1 Processor -- start 2023-10-26T06:50:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.039+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f1a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.039+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f17880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.039+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f1a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f1a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55100/0 (socket says 172.21.15.89:55100) 2023-10-26T06:50:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f39fd19b700 1 -- 172.21.15.89:0/2617364898 learned_addr learned my addr 172.21.15.89:0/2617364898 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 0x555a20f17eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f39fc99a700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f17880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20e60ef0 con 0x555a20f0e650 2023-10-26T06:50:14.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20dfd790 con 0x555a20f13c60 2023-10-26T06:50:14.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20dfd430 con 0x555a20f14580 2023-10-26T06:50:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.040+0000 7f39fd99c700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 0x555a20f17eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.041+0000 7f39fd19b700 1 -- 172.21.15.89:0/2617364898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 msgr2=0x555a20f17eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.042+0000 7f39fd19b700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 0x555a20f17eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.042+0000 7f39fd19b700 1 -- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 msgr2=0x555a20f17880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.042+0000 7f39fd19b700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f17880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.042+0000 7f39fd19b700 1 -- 172.21.15.89:0/2617364898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f810cd90 con 0x555a20f0e650 2023-10-26T06:50:14.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.042+0000 7f39fc99a700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f17880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:50:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.042+0000 7f3a12d28b80 10 client.0 _ll_register_callbacks cb 0x555a20d47b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.042+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a20e65f70 con 0x555a20f0e650 2023-10-26T06:50:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.043+0000 7f39fd19b700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f1a590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f39f40098c0 tx=0x7f39f400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15655d6d0e1c201b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.043+0000 7f39e0ff9700 10 client.15804 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:50:14.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.043+0000 7f39e0ff9700 1 -- 172.21.15.89:0/2617364898 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4010f10 con 0x555a20f0e650 2023-10-26T06:50:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.043+0000 7f39e0ff9700 1 -- 172.21.15.89:0/2617364898 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f39f400b610 con 0x555a20f0e650 2023-10-26T06:50:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.044+0000 7f39e0ff9700 1 -- 172.21.15.89:0/2617364898 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f402a430 con 0x555a20f0e650 2023-10-26T06:50:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.044+0000 7f3a12d28b80 10 client.15804 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.044+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=20}) v3 -- 0x555a20c8be60 con 0x555a20f0e650 2023-10-26T06:50:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.044+0000 7f39e0ff9700 1 -- 172.21.15.89:0/2617364898 <== mon.0 v2:172.21.15.89:3300/0 4 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39f4015020 con 0x555a20f0e650 2023-10-26T06:50:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.045+0000 7f3a12d28b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.045+0000 7f3a12d28b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.045+0000 7f3a12d28b80 20 client.15804 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.045+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f27820 0x555a20f29ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.045+0000 7f3a12d28b80 4 client.15804 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.045+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a20edd930 con 0x555a20f27820 2023-10-26T06:50:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.045+0000 7f39fc99a700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f27820 0x555a20f29ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.046+0000 7f39fc99a700 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f27820 0x555a20f29ca0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7add4e4ac33d4dc server_cookie=584beeb6611bc56c in_seq=0 out_seq=0 2023-10-26T06:50:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.046+0000 7f39e0ff9700 10 client.15804 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.048+0000 7f39e0ff9700 1 -- 172.21.15.89:0/2617364898 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a20edd930 con 0x555a20f27820 2023-10-26T06:50:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.048+0000 7f39e0ff9700 10 client.15804 handle_command_reply: tid=0 2023-10-26T06:50:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.048+0000 7f3a12d28b80 1 client.15804 shutdown 2023-10-26T06:50:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 msgr2=0x555a20f1a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f1a590 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f39f40098c0 tx=0x7f39f400fac0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 shutdown_connections 2023-10-26T06:50:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f0e650 0x555a20f1a590 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f27820 0x555a20f29ca0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f14580 0x555a20f17eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/2617364898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f17880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.049+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 >> 172.21.15.89:0/2617364898 conn(0x555a20e827a0 msgr2=0x555a20f0d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.050+0000 7f39e0ff9700 0 client.15804 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.051+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 shutdown_connections 2023-10-26T06:50:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.052+0000 7f3a12d28b80 1 -- 172.21.15.89:0/2617364898 wait complete. 2023-10-26T06:50:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.053+0000 7f3a12d28b80 20 client.15804 trim_cache size 0 max 16384 2023-10-26T06:50:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.103+0000 7f3a12d28b80 1 Processor -- start 2023-10-26T06:50:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.104+0000 7f3a12d28b80 1 -- start start 2023-10-26T06:50:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.104+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f0e650 0x555a20f02dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.104+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f1ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.104+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f14580 0x555a20f13080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.104+0000 7f3a12d28b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20f03b70 con 0x555a20f14580 2023-10-26T06:50:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.104+0000 7f3a12d28b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20ce19a0 con 0x555a20f13c60 2023-10-26T06:50:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.104+0000 7f3a12d28b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20f18e80 con 0x555a20f0e650 2023-10-26T06:50:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.105+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f0e650 0x555a20f02dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.106+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f0e650 0x555a20f02dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44164/0 (socket says 172.21.15.89:44164) 2023-10-26T06:50:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.106+0000 7f39fd19b700 1 -- 172.21.15.89:0/1120188353 learned_addr learned my addr 172.21.15.89:0/1120188353 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.107+0000 7f39fd99c700 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f14580 0x555a20f13080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.108+0000 7f39fc99a700 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f1ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.111+0000 7f39fd19b700 1 -- 172.21.15.89:0/1120188353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 msgr2=0x555a20f1ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.111+0000 7f39fd19b700 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f1ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.111+0000 7f39fd19b700 1 -- 172.21.15.89:0/1120188353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f14580 msgr2=0x555a20f13080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.111+0000 7f39fd19b700 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f14580 0x555a20f13080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.111+0000 7f39fd19b700 1 -- 172.21.15.89:0/1120188353 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a20c74ce0 con 0x555a20f0e650 2023-10-26T06:50:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.114+0000 7f39fd19b700 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f0e650 0x555a20f02dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009810 tx=0x7f39f402a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78008e4af8fb4604 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.117+0000 7f39e3fff700 1 -- 172.21.15.89:0/1120188353 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4010f10 con 0x555a20f0e650 2023-10-26T06:50:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.117+0000 7f39e3fff700 1 -- 172.21.15.89:0/1120188353 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a20e65f70 con 0x555a20f0e650 2023-10-26T06:50:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.119+0000 7f39e3fff700 1 -- 172.21.15.89:0/1120188353 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4010a10 con 0x555a20f0e650 2023-10-26T06:50:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.120+0000 7f3a12d28b80 1 -- 172.21.15.89:0/1120188353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f0e650 msgr2=0x555a20f02dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.120+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f0e650 0x555a20f02dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009810 tx=0x7f39f402a000 comp rx=0 tx=0).stop 2023-10-26T06:50:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.120+0000 7f3a12d28b80 1 -- 172.21.15.89:0/1120188353 shutdown_connections 2023-10-26T06:50:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.120+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f14580 0x555a20f13080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.120+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f0e650 0x555a20f02dc0 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009810 tx=0x7f39f402a000 comp rx=0 tx=0).stop 2023-10-26T06:50:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.120+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/1120188353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f13c60 0x555a20f1ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.120+0000 7f3a12d28b80 1 -- 172.21.15.89:0/1120188353 >> 172.21.15.89:0/1120188353 conn(0x555a20e827a0 msgr2=0x555a20f12970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.124+0000 7f3a12d28b80 1 -- 172.21.15.89:0/1120188353 shutdown_connections 2023-10-26T06:50:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.124+0000 7f3a12d28b80 1 -- 172.21.15.89:0/1120188353 wait complete. 2023-10-26T06:50:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.125+0000 7f3a12d28b80 1 -- start start 2023-10-26T06:50:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.125+0000 7f3a12d28b80 1 Processor -- start 2023-10-26T06:50:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.126+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f13c60 0x555a20f1de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.126+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f14580 0x555a20f1e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.126+0000 7f3a12d28b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f756d0 0x555a20f75b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.126+0000 7f3a12d28b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20f76090 con 0x555a20f756d0 2023-10-26T06:50:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.126+0000 7f3a12d28b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20f76200 con 0x555a20f14580 2023-10-26T06:50:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.126+0000 7f3a12d28b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a20f763c0 con 0x555a20f13c60 2023-10-26T06:50:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.126+0000 7f3a12d28b80 10 client.0 _ll_register_callbacks cb 0x555a20d47b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f13c60 0x555a20f1de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f13c60 0x555a20f1de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44174/0 (socket says 172.21.15.89:44174) 2023-10-26T06:50:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 -- 172.21.15.89:0/163273337 learned_addr learned my addr 172.21.15.89:0/163273337 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 -- 172.21.15.89:0/163273337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f14580 msgr2=0x555a20f1e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f14580 0x555a20f1e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 -- 172.21.15.89:0/163273337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f756d0 msgr2=0x555a20f75b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f756d0 0x555a20f75b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.128+0000 7f39fd19b700 1 -- 172.21.15.89:0/163273337 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a20c8be60 con 0x555a20f13c60 2023-10-26T06:50:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.138+0000 7f39fd19b700 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f13c60 0x555a20f1de30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39f400b360 tx=0x7f39f402a800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb588456f3529a1a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.138+0000 7f39e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:50:14.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.144+0000 7f39e0ff9700 1 -- 172.21.15.89:0/163273337 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f402aaf0 con 0x555a20f13c60 2023-10-26T06:50:14.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.144+0000 7f39e0ff9700 1 -- 172.21.15.89:0/163273337 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f39f4000e40 con 0x555a20f13c60 2023-10-26T06:50:14.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.144+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39f401d040 con 0x555a20f13c60 2023-10-26T06:50:14.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.147+0000 7f39e0ff9700 1 -- 172.21.15.89:0/163273337 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f400b610 con 0x555a20f13c60 2023-10-26T06:50:14.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.147+0000 7f3a12d28b80 10 client.25831 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:14.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.148+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555a20cb22d0 con 0x555a20f13c60 2023-10-26T06:50:14.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.152+0000 7f39e0ff9700 1 -- 172.21.15.89:0/163273337 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f401d040 con 0x555a20f13c60 2023-10-26T06:50:14.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.153+0000 7f39e0ff9700 1 -- 172.21.15.89:0/163273337 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39f4015030 con 0x555a20f13c60 2023-10-26T06:50:14.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.153+0000 7f3a12d28b80 10 client.25831 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:14.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.153+0000 7f3a12d28b80 10 client.25831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:14.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.153+0000 7f3a12d28b80 20 client.25831 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:14.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.153+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f7d670 0x555a20f7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.153+0000 7f3a12d28b80 4 client.25831 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:50:14.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.153+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555a20e59ed0 con 0x555a20f7d670 2023-10-26T06:50:14.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.154+0000 7f39fc99a700 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f7d670 0x555a20f7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.176+0000 7f39fc99a700 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f7d670 0x555a20f7fa60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba4ee56bba376184 server_cookie=903faafa46e903 in_seq=0 out_seq=0 2023-10-26T06:50:14.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.176+0000 7f39e0ff9700 10 client.25831 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.182+0000 7f39e0ff9700 1 -- 172.21.15.89:0/163273337 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555a20e59ed0 con 0x555a20f7d670 2023-10-26T06:50:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.182+0000 7f39e0ff9700 10 client.25831 handle_command_reply: tid=0 2023-10-26T06:50:14.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.182+0000 7f3a12d28b80 1 client.25831 shutdown 2023-10-26T06:50:14.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.183+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f13c60 msgr2=0x555a20f1de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.183+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f13c60 0x555a20f1de30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39f400b360 tx=0x7f39f402a800 comp rx=0 tx=0).stop 2023-10-26T06:50:14.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.184+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 shutdown_connections 2023-10-26T06:50:14.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.184+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a20f756d0 0x555a20f75b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.184+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555a20f7d670 0x555a20f7fa60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.184+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a20f13c60 0x555a20f1de30 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39f400b360 tx=0x7f39f402a800 comp rx=0 tx=0).stop 2023-10-26T06:50:14.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.184+0000 7f3a12d28b80 1 --2- 172.21.15.89:0/163273337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a20f14580 0x555a20f1e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.184+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 >> 172.21.15.89:0/163273337 conn(0x555a20e827a0 msgr2=0x555a20f02c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.184+0000 7f39e0ff9700 0 client.25831 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.185+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 shutdown_connections 2023-10-26T06:50:14.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.185+0000 7f3a12d28b80 1 -- 172.21.15.89:0/163273337 wait complete. 2023-10-26T06:50:14.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.185+0000 7f3a12d28b80 20 client.25831 trim_cache size 0 max 16384 2023-10-26T06:50:14.188 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:50:14.188 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:50:14.188 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "e432eecc-80e7-4380-b92c-5331bc663cdb", 2023-10-26T06:50:14.188 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:50:14.188 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:50:14.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f39e40a1650 msgr2=0x7f39e40a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f39e40a1650 0x7f39e40a3b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39e8006fd0 tx=0x7f39e8009d10 comp rx=0 tx=0).stop 2023-10-26T06:50:14.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f81056e0 msgr2=0x7f39f8194d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f81056e0 0x7f39f8194d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f39f0007fd0 tx=0x7f39f000d3b0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 shutdown_connections 2023-10-26T06:50:14.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f39e40a1650 0x7f39e40a3b10 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f39e8006fd0 tx=0x7f39e8009d10 comp rx=0 tx=0).stop 2023-10-26T06:50:14.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39f81056e0 0x7f39f8194d20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39f80ff640 0x7f39f819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 --2- 172.21.15.89:0/3584016942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39f8100000 0x7f39f819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.187+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 >> 172.21.15.89:0/3584016942 conn(0x7f39f80767f0 msgr2=0x7f39f8067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.190+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 shutdown_connections 2023-10-26T06:50:14.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.190+0000 7f39ff3ff700 1 -- 172.21.15.89:0/3584016942 wait complete. 2023-10-26T06:50:14.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:50:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.603+0000 7f295d3ff700 1 -- 172.21.15.89:0/744063529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2958105880 msgr2=0x7f2958105c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.603+0000 7f295d3ff700 1 --2- 172.21.15.89:0/744063529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2958105880 0x7f2958105c60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2948009990 tx=0x7f2948009ca0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.603+0000 7f295d3ff700 1 -- 172.21.15.89:0/744063529 shutdown_connections 2023-10-26T06:50:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.603+0000 7f295d3ff700 1 --2- 172.21.15.89:0/744063529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29580ff210 0x7f295810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.603+0000 7f295d3ff700 1 --2- 172.21.15.89:0/744063529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2958105880 0x7f2958105c60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.603+0000 7f295d3ff700 1 --2- 172.21.15.89:0/744063529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29580fe850 0x7f29580fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.603+0000 7f295d3ff700 1 -- 172.21.15.89:0/744063529 >> 172.21.15.89:0/744063529 conn(0x7f29580757e0 msgr2=0x7f29580fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.604+0000 7f295d3ff700 1 -- 172.21.15.89:0/744063529 shutdown_connections 2023-10-26T06:50:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.604+0000 7f295d3ff700 1 -- 172.21.15.89:0/744063529 wait complete. 2023-10-26T06:50:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.605+0000 7f295d3ff700 1 Processor -- start 2023-10-26T06:50:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.606+0000 7f295d3ff700 1 -- start start 2023-10-26T06:50:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f295d3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29580fe850 0x7f2958198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f295d3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29580ff210 0x7f2958199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f295d3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f295819d8c0 0x7f295819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f295d3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295819e340 con 0x7f29580ff210 2023-10-26T06:50:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f295d3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295819e4b0 con 0x7f295819d8c0 2023-10-26T06:50:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f295d3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295819e670 con 0x7f29580fe850 2023-10-26T06:50:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29580fe850 0x7f2958198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f29577fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f295819d8c0 0x7f295819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29580fe850 0x7f2958198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44192/0 (socket says 172.21.15.89:44192) 2023-10-26T06:50:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.607+0000 7f2956ffd700 1 -- 172.21.15.89:0/3738230578 learned_addr learned my addr 172.21.15.89:0/3738230578 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f294e7fc700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29580ff210 0x7f2958199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f2956ffd700 1 -- 172.21.15.89:0/3738230578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f295819d8c0 msgr2=0x7f295819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f2956ffd700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f295819d8c0 0x7f295819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f2956ffd700 1 -- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29580ff210 msgr2=0x7f2958199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f2956ffd700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29580ff210 0x7f2958199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f2956ffd700 1 -- 172.21.15.89:0/3738230578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29480095f0 con 0x7f29580fe850 2023-10-26T06:50:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f2956ffd700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29580fe850 0x7f2958198fe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2948003730 tx=0x7f2948005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.608+0000 7f2954ff9700 1 -- 172.21.15.89:0/3738230578 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948023ce0 con 0x7f29580fe850 2023-10-26T06:50:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.609+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2958106400 con 0x7f29580fe850 2023-10-26T06:50:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.609+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2958106860 con 0x7f29580fe850 2023-10-26T06:50:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.609+0000 7f2954ff9700 1 -- 172.21.15.89:0/3738230578 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2948023e40 con 0x7f29580fe850 2023-10-26T06:50:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.609+0000 7f2954ff9700 1 -- 172.21.15.89:0/3738230578 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294802cb10 con 0x7f29580fe850 2023-10-26T06:50:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.618+0000 7f2954ff9700 1 -- 172.21.15.89:0/3738230578 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2948029070 con 0x7f29580fe850 2023-10-26T06:50:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.621+0000 7f2954ff9700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f29440a1630 0x7f29440a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.621+0000 7f2954ff9700 1 -- 172.21.15.89:0/3738230578 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f29480d6a10 con 0x7f29580fe850 2023-10-26T06:50:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.622+0000 7f294e7fc700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f29440a1630 0x7f29440a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.622+0000 7f2970c3eb80 1 Processor -- start 2023-10-26T06:50:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.622+0000 7f2970c3eb80 1 -- start start 2023-10-26T06:50:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 0x556f4559ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4550f6e0 con 0x556f455a2500 2023-10-26T06:50:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f454f5c60 con 0x556f4559c5d0 2023-10-26T06:50:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f454f3580 con 0x556f455a1be0 2023-10-26T06:50:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f294e7fc700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f29440a1630 0x7f29440a3af0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f295819a530 tx=0x7f294000b5b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455a1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44194/0 (socket says 172.21.15.89:44194) 2023-10-26T06:50:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2956ffd700 1 -- 172.21.15.89:0/1016116413 learned_addr learned my addr 172.21.15.89:0/1016116413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2956ffd700 1 -- 172.21.15.89:0/1016116413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 msgr2=0x556f4559ca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:50:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2956ffd700 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 0x556f4559ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.623+0000 7f2956ffd700 1 -- 172.21.15.89:0/1016116413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 msgr2=0x556f4559c090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:50:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.624+0000 7f294e7fc700 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.624+0000 7f2956ffd700 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.624+0000 7f2956ffd700 1 -- 172.21.15.89:0/1016116413 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f454ac3f0 con 0x556f455a1be0 2023-10-26T06:50:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.625+0000 7f2956ffd700 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455a1fc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29480a0f70 tx=0x7f29480a4420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952648cbb453d381 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.625+0000 7f294d7fa700 1 -- 172.21.15.89:0/1016116413 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480a8bc0 con 0x556f455a1be0 2023-10-26T06:50:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.625+0000 7f294d7fa700 1 -- 172.21.15.89:0/1016116413 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29480acd70 con 0x556f455a1be0 2023-10-26T06:50:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.625+0000 7f294d7fa700 1 -- 172.21.15.89:0/1016116413 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480a41c0 con 0x556f455a1be0 2023-10-26T06:50:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.626+0000 7f2970c3eb80 1 -- 172.21.15.89:0/1016116413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 msgr2=0x556f455a1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.626+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455a1fc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29480a0f70 tx=0x7f29480a4420 comp rx=0 tx=0).stop 2023-10-26T06:50:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.627+0000 7f2970c3eb80 1 -- 172.21.15.89:0/1016116413 shutdown_connections 2023-10-26T06:50:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.627+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 0x556f4559ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.627+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455a1fc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.627+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/1016116413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.627+0000 7f2970c3eb80 1 -- 172.21.15.89:0/1016116413 >> 172.21.15.89:0/1016116413 conn(0x556f45510720 msgr2=0x556f45596b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.629+0000 7f2970c3eb80 1 -- 172.21.15.89:0/1016116413 shutdown_connections 2023-10-26T06:50:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.629+0000 7f2970c3eb80 1 -- 172.21.15.89:0/1016116413 wait complete. 2023-10-26T06:50:14.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.631+0000 7f2970c3eb80 1 -- start start 2023-10-26T06:50:14.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.632+0000 7f2970c3eb80 1 Processor -- start 2023-10-26T06:50:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.635+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.635+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f455a5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.635+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f455a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.635+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f454eee70 con 0x556f455a1be0 2023-10-26T06:50:14.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.635+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4548b710 con 0x556f455a2500 2023-10-26T06:50:14.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.635+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4548b3b0 con 0x556f4559c5d0 2023-10-26T06:50:14.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.635+0000 7f2970c3eb80 10 client.0 _ll_register_callbacks cb 0x556f453d5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:14.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.636+0000 7f294e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f455a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.636+0000 7f294e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f455a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55160/0 (socket says 172.21.15.89:55160) 2023-10-26T06:50:14.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.636+0000 7f294e7fc700 1 -- 172.21.15.89:0/3934358736 learned_addr learned my addr 172.21.15.89:0/3934358736 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.636+0000 7f2956ffd700 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f294e7fc700 1 -- 172.21.15.89:0/3934358736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 msgr2=0x556f455a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f294e7fc700 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f294e7fc700 1 -- 172.21.15.89:0/3934358736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 msgr2=0x556f455a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f294e7fc700 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f455a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f294e7fc700 1 -- 172.21.15.89:0/3934358736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2958106860 con 0x556f455a1be0 2023-10-26T06:50:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f294e7fc700 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f455a5750 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f294000fe40 tx=0x7f2940004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7dec56165a189a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f293e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:50:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.637+0000 7f293e7fc700 1 -- 172.21.15.89:0/3934358736 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294001e620 con 0x556f455a1be0 2023-10-26T06:50:14.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.638+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f454f3ef0 con 0x556f455a1be0 2023-10-26T06:50:14.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.638+0000 7f293e7fc700 1 -- 172.21.15.89:0/3934358736 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f294001e780 con 0x556f455a1be0 2023-10-26T06:50:14.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.638+0000 7f293e7fc700 1 -- 172.21.15.89:0/3934358736 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2940011e70 con 0x556f455a1be0 2023-10-26T06:50:14.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.638+0000 7f2970c3eb80 10 client.15813 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:14.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.638+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556f45319de0 con 0x556f455a1be0 2023-10-26T06:50:14.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.639+0000 7f293e7fc700 1 -- 172.21.15.89:0/3934358736 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f454f3ef0 con 0x556f455a1be0 2023-10-26T06:50:14.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.639+0000 7f293e7fc700 1 -- 172.21.15.89:0/3934358736 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2940024020 con 0x556f455a1be0 2023-10-26T06:50:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.639+0000 7f2970c3eb80 10 client.15813 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.639+0000 7f2970c3eb80 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.640+0000 7f2970c3eb80 20 client.15813 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.640+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f455b5cf0 0x556f455b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.640+0000 7f2970c3eb80 4 client.15813 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.640+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f4556b8b0 con 0x556f455b5cf0 2023-10-26T06:50:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.640+0000 7f2956ffd700 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f455b5cf0 0x556f455b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.641+0000 7f2956ffd700 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f455b5cf0 0x556f455b8110 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ab13086c319762 server_cookie=88b6daac96c3d324 in_seq=0 out_seq=0 2023-10-26T06:50:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.641+0000 7f293e7fc700 10 client.15813 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.642+0000 7f293e7fc700 1 -- 172.21.15.89:0/3934358736 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f4556b8b0 con 0x556f455b5cf0 2023-10-26T06:50:14.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.642+0000 7f293e7fc700 10 client.15813 handle_command_reply: tid=0 2023-10-26T06:50:14.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.642+0000 7f2970c3eb80 1 client.15813 shutdown 2023-10-26T06:50:14.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 msgr2=0x556f455a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f455a5750 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f294000fe40 tx=0x7f2940004750 comp rx=0 tx=0).stop 2023-10-26T06:50:14.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 shutdown_connections 2023-10-26T06:50:14.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f455b5cf0 0x556f455b8110 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f455a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/3934358736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f455a5750 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 >> 172.21.15.89:0/3934358736 conn(0x556f45510720 msgr2=0x556f4559b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f293e7fc700 0 client.15813 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 shutdown_connections 2023-10-26T06:50:14.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.646+0000 7f2970c3eb80 1 -- 172.21.15.89:0/3934358736 wait complete. 2023-10-26T06:50:14.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.648+0000 7f2970c3eb80 20 client.15813 trim_cache size 0 max 16384 2023-10-26T06:50:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 Processor -- start 2023-10-26T06:50:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 -- start start 2023-10-26T06:50:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 0x556f455a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455acc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f4536f920 con 0x556f455a2500 2023-10-26T06:50:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f455a63d0 con 0x556f4559c5d0 2023-10-26T06:50:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.674+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f455983b0 con 0x556f455a1be0 2023-10-26T06:50:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.675+0000 7f29577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.675+0000 7f29577fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55166/0 (socket says 172.21.15.89:55166) 2023-10-26T06:50:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.675+0000 7f29577fe700 1 -- 172.21.15.89:0/4094637996 learned_addr learned my addr 172.21.15.89:0/4094637996 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f2956ffd700 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 0x556f455a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f294e7fc700 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f29577fe700 1 -- 172.21.15.89:0/4094637996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 msgr2=0x556f455acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f29577fe700 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455acc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f29577fe700 1 -- 172.21.15.89:0/4094637996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 msgr2=0x556f455a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f29577fe700 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 0x556f455a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f29577fe700 1 -- 172.21.15.89:0/4094637996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f45302c60 con 0x556f455a2500 2023-10-26T06:50:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.676+0000 7f294e7fc700 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:50:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.677+0000 7f29577fe700 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559b930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2950009e00 tx=0x7f2950009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42614ad5796c2ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.677+0000 7f294dffb700 1 -- 172.21.15.89:0/4094637996 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950011070 con 0x556f455a2500 2023-10-26T06:50:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.678+0000 7f294dffb700 1 -- 172.21.15.89:0/4094637996 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2950009930 con 0x556f455a2500 2023-10-26T06:50:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.678+0000 7f294dffb700 1 -- 172.21.15.89:0/4094637996 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950015d60 con 0x556f455a2500 2023-10-26T06:50:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 -- 172.21.15.89:0/4094637996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 msgr2=0x556f4559b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559b930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2950009e00 tx=0x7f2950009e30 comp rx=0 tx=0).stop 2023-10-26T06:50:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 -- 172.21.15.89:0/4094637996 shutdown_connections 2023-10-26T06:50:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a2500 0x556f4559b930 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f4559c5d0 0x556f455a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/4094637996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f455a1be0 0x556f455acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 -- 172.21.15.89:0/4094637996 >> 172.21.15.89:0/4094637996 conn(0x556f45510720 msgr2=0x556f455969c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 -- 172.21.15.89:0/4094637996 shutdown_connections 2023-10-26T06:50:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.679+0000 7f2970c3eb80 1 -- 172.21.15.89:0/4094637996 wait complete. 2023-10-26T06:50:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.680+0000 7f2970c3eb80 1 -- start start 2023-10-26T06:50:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.680+0000 7f2970c3eb80 1 Processor -- start 2023-10-26T06:50:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.682+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.682+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f45605af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.682+0000 7f2970c3eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f45606230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.682+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f45606940 con 0x556f455a1be0 2023-10-26T06:50:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.682+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f455ab210 con 0x556f455a2500 2023-10-26T06:50:14.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.682+0000 7f2970c3eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f455ab3a0 con 0x556f4559c5d0 2023-10-26T06:50:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.682+0000 7f2970c3eb80 10 client.0 _ll_register_callbacks cb 0x556f453d5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f29577fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f45606230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f29577fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f45606230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:50126/0 (socket says 172.21.15.89:50126) 2023-10-26T06:50:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:44214/0 (socket says 172.21.15.89:44214) 2023-10-26T06:50:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f29577fe700 1 -- 172.21.15.89:0/730814981 learned_addr learned my addr 172.21.15.89:0/730814981 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f294e7fc700 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f45605af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f2956ffd700 1 -- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 msgr2=0x556f45606230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f2956ffd700 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f45606230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f2956ffd700 1 -- 172.21.15.89:0/730814981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 msgr2=0x556f45605af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f2956ffd700 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f45605af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.683+0000 7f2956ffd700 1 -- 172.21.15.89:0/730814981 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2950009ab0 con 0x556f4559c5d0 2023-10-26T06:50:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.684+0000 7f29577fe700 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f45606230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:50:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.684+0000 7f2956ffd700 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455ace20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x556f455a8b70 tx=0x7f29480a4bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc7e6b712d88f1d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.684+0000 7f293e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:50:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.684+0000 7f293e7fc700 1 -- 172.21.15.89:0/730814981 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480a2c90 con 0x556f4559c5d0 2023-10-26T06:50:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.684+0000 7f293e7fc700 1 -- 172.21.15.89:0/730814981 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f29480a4160 con 0x556f4559c5d0 2023-10-26T06:50:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.684+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29480acd70 con 0x556f4559c5d0 2023-10-26T06:50:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.684+0000 7f293e7fc700 1 -- 172.21.15.89:0/730814981 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29480a4cd0 con 0x556f4559c5d0 2023-10-26T06:50:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 10 client.25846 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556f45340250 con 0x556f4559c5d0 2023-10-26T06:50:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f293e7fc700 1 -- 172.21.15.89:0/730814981 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29480c0080 con 0x556f4559c5d0 2023-10-26T06:50:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f293e7fc700 1 -- 172.21.15.89:0/730814981 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29480ac970 con 0x556f4559c5d0 2023-10-26T06:50:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 10 client.25846 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 20 client.25846 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f4561bfc0 0x556f4561e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 4 client.25846 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:50:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.685+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f454e7e50 con 0x556f4561bfc0 2023-10-26T06:50:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.686+0000 7f294e7fc700 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f4561bfc0 0x556f4561e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:14.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.686+0000 7f294e7fc700 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f4561bfc0 0x556f4561e3b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5cbd5fe7b707272 server_cookie=390af13d969cf216 in_seq=0 out_seq=0 2023-10-26T06:50:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.686+0000 7f293e7fc700 10 client.25846 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f293e7fc700 1 -- 172.21.15.89:0/730814981 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x556f454e7e50 con 0x556f4561bfc0 2023-10-26T06:50:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f293e7fc700 10 client.25846 handle_command_reply: tid=0 2023-10-26T06:50:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 client.25846 shutdown 2023-10-26T06:50:14.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 msgr2=0x556f455ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455ace20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x556f455a8b70 tx=0x7f29480a4bf0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 shutdown_connections 2023-10-26T06:50:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x556f4561bfc0 0x556f4561e3b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556f455a2500 0x556f45606230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4559c5d0 0x556f455ace20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 --2- 172.21.15.89:0/730814981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f455a1be0 0x556f45605af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 >> 172.21.15.89:0/730814981 conn(0x556f45510720 msgr2=0x556f4559f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f293e7fc700 0 client.25846 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.687+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 shutdown_connections 2023-10-26T06:50:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.688+0000 7f2970c3eb80 1 -- 172.21.15.89:0/730814981 wait complete. 2023-10-26T06:50:14.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.688+0000 7f2970c3eb80 20 client.25846 trim_cache size 0 max 16384 2023-10-26T06:50:14.697 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:50:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "scrub active (203 inodes in the stack)", 2023-10-26T06:50:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:50:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "e432eecc-80e7-4380-b92c-5331bc663cdb": { 2023-10-26T06:50:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:50:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "e432eecc-80e7-4380-b92c-5331bc663cdb", 2023-10-26T06:50:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:50:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:50:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:50:14.699 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:50:14.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f29440a1630 msgr2=0x7f29440a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f29440a1630 0x7f29440a3af0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f295819a530 tx=0x7f294000b5b0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29580fe850 msgr2=0x7f2958198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29580fe850 0x7f2958198fe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2948003730 tx=0x7f2948005720 comp rx=0 tx=0).stop 2023-10-26T06:50:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 shutdown_connections 2023-10-26T06:50:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f29440a1630 0x7f29440a3af0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f295819d8c0 0x7f295819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29580fe850 0x7f2958198fe0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 --2- 172.21.15.89:0/3738230578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29580ff210 0x7f2958199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.689+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 >> 172.21.15.89:0/3738230578 conn(0x7f29580757e0 msgr2=0x7f29581017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.690+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 shutdown_connections 2023-10-26T06:50:14.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:14.690+0000 7f295d3ff700 1 -- 172.21.15.89:0/3738230578 wait complete. 2023-10-26T06:50:14.720 INFO:tasks.cephfs.filesystem:scrub status for tag:e432eecc-80e7-4380-b92c-5331bc663cdb - {'path': '/', 'tag': 'e432eecc-80e7-4380-b92c-5331bc663cdb', 'options': 'recursive,force'} 2023-10-26T06:50:15.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:15 smithi135 ceph-mon[131403]: pgmap v410: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1002 GiB / 1.0 TiB avail; 97 KiB/s rd, 149 MiB/s wr, 16.97k op/s 2023-10-26T06:50:15.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:15 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:50:15.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:15 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:15.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:15 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:50:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:15 smithi089 ceph-mon[95031]: pgmap v410: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1002 GiB / 1.0 TiB avail; 97 KiB/s rd, 149 MiB/s wr, 16.97k op/s 2023-10-26T06:50:15.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:15 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:50:15.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:15 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:15.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:15 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:50:15.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:15 smithi192 ceph-mon[131661]: pgmap v410: 97 pgs: 97 active+clean; 21 GiB data, 70 GiB used, 1002 GiB / 1.0 TiB avail; 97 KiB/s rd, 149 MiB/s wr, 16.97k op/s 2023-10-26T06:50:15.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:15 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:50:15.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:15 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:15.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:15 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:50:17.433 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:17 smithi192 ceph-mon[131661]: pgmap v411: 97 pgs: 97 active+clean; 21 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 127 KiB/s rd, 172 MiB/s wr, 20.18k op/s 2023-10-26T06:50:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:17 smithi135 ceph-mon[131403]: pgmap v411: 97 pgs: 97 active+clean; 21 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 127 KiB/s rd, 172 MiB/s wr, 20.18k op/s 2023-10-26T06:50:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:17 smithi089 ceph-mon[95031]: pgmap v411: 97 pgs: 97 active+clean; 21 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 127 KiB/s rd, 172 MiB/s wr, 20.18k op/s 2023-10-26T06:50:19.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:19 smithi135 ceph-mon[131403]: pgmap v412: 97 pgs: 97 active+clean; 21 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 82 KiB/s rd, 153 MiB/s wr, 18.37k op/s 2023-10-26T06:50:19.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:19 smithi089 ceph-mon[95031]: pgmap v412: 97 pgs: 97 active+clean; 21 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 82 KiB/s rd, 153 MiB/s wr, 18.37k op/s 2023-10-26T06:50:19.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:19 smithi192 ceph-mon[131661]: pgmap v412: 97 pgs: 97 active+clean; 21 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 82 KiB/s rd, 153 MiB/s wr, 18.37k op/s 2023-10-26T06:50:21.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:21 smithi135 ceph-mon[131403]: pgmap v413: 97 pgs: 97 active+clean; 21 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 83 KiB/s rd, 144 MiB/s wr, 18.01k op/s 2023-10-26T06:50:21.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:21 smithi089 ceph-mon[95031]: pgmap v413: 97 pgs: 97 active+clean; 21 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 83 KiB/s rd, 144 MiB/s wr, 18.01k op/s 2023-10-26T06:50:21.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:21 smithi192 ceph-mon[131661]: pgmap v413: 97 pgs: 97 active+clean; 21 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 83 KiB/s rd, 144 MiB/s wr, 18.01k op/s 2023-10-26T06:50:22.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:22 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:50:22.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:22 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:50:22.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:22 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:50:23.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:23 smithi089 ceph-mon[95031]: pgmap v414: 97 pgs: 97 active+clean; 21 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 95 KiB/s rd, 136 MiB/s wr, 17.81k op/s 2023-10-26T06:50:23.902 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:23 smithi192 ceph-mon[131661]: pgmap v414: 97 pgs: 97 active+clean; 21 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 95 KiB/s rd, 136 MiB/s wr, 17.81k op/s 2023-10-26T06:50:23.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:23 smithi135 ceph-mon[131403]: pgmap v414: 97 pgs: 97 active+clean; 21 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 95 KiB/s rd, 136 MiB/s wr, 17.81k op/s 2023-10-26T06:50:24.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:24 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:50:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:50:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:50:24.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:24 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:50:24.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:24 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:24.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:50:24.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:50:24.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:50:24.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:24 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:50:24.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:24 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:24.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:50:24.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:50:24.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:50:24.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:24 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:50:25.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:25 smithi089 ceph-mon[95031]: pgmap v415: 97 pgs: 97 active+clean; 21 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 51 KiB/s rd, 122 MiB/s wr, 16.36k op/s 2023-10-26T06:50:25.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:25 smithi192 ceph-mon[131661]: pgmap v415: 97 pgs: 97 active+clean; 21 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 51 KiB/s rd, 122 MiB/s wr, 16.36k op/s 2023-10-26T06:50:25.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:25 smithi135 ceph-mon[131403]: pgmap v415: 97 pgs: 97 active+clean; 21 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 51 KiB/s rd, 122 MiB/s wr, 16.36k op/s 2023-10-26T06:50:27.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:27 smithi089 ceph-mon[95031]: pgmap v416: 97 pgs: 97 active+clean; 21 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 101 KiB/s rd, 132 MiB/s wr, 18.13k op/s 2023-10-26T06:50:27.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:27 smithi192 ceph-mon[131661]: pgmap v416: 97 pgs: 97 active+clean; 21 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 101 KiB/s rd, 132 MiB/s wr, 18.13k op/s 2023-10-26T06:50:27.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:27 smithi135 ceph-mon[131403]: pgmap v416: 97 pgs: 97 active+clean; 21 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 101 KiB/s rd, 132 MiB/s wr, 18.13k op/s 2023-10-26T06:50:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:29 smithi089 ceph-mon[95031]: pgmap v417: 97 pgs: 97 active+clean; 22 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 64 KiB/s rd, 103 MiB/s wr, 14.63k op/s 2023-10-26T06:50:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:29 smithi192 ceph-mon[131661]: pgmap v417: 97 pgs: 97 active+clean; 22 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 64 KiB/s rd, 103 MiB/s wr, 14.63k op/s 2023-10-26T06:50:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:29 smithi135 ceph-mon[131403]: pgmap v417: 97 pgs: 97 active+clean; 22 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 64 KiB/s rd, 103 MiB/s wr, 14.63k op/s 2023-10-26T06:50:31.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:31 smithi089 ceph-mon[95031]: pgmap v418: 97 pgs: 97 active+clean; 22 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 64 KiB/s rd, 93 MiB/s wr, 13.30k op/s 2023-10-26T06:50:31.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:31 smithi192 ceph-mon[131661]: pgmap v418: 97 pgs: 97 active+clean; 22 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 64 KiB/s rd, 93 MiB/s wr, 13.30k op/s 2023-10-26T06:50:31.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:31 smithi135 ceph-mon[131403]: pgmap v418: 97 pgs: 97 active+clean; 22 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 64 KiB/s rd, 93 MiB/s wr, 13.30k op/s 2023-10-26T06:50:33.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:33 smithi089 ceph-mon[95031]: pgmap v419: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 86 KiB/s rd, 98 MiB/s wr, 13.94k op/s 2023-10-26T06:50:33.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:33 smithi192 ceph-mon[131661]: pgmap v419: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 86 KiB/s rd, 98 MiB/s wr, 13.94k op/s 2023-10-26T06:50:33.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:33 smithi135 ceph-mon[131403]: pgmap v419: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 86 KiB/s rd, 98 MiB/s wr, 13.94k op/s 2023-10-26T06:50:35.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:35 smithi089 ceph-mon[95031]: pgmap v420: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 74 KiB/s rd, 71 MiB/s wr, 10.37k op/s 2023-10-26T06:50:35.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:35 smithi192 ceph-mon[131661]: pgmap v420: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 74 KiB/s rd, 71 MiB/s wr, 10.37k op/s 2023-10-26T06:50:35.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:35 smithi135 ceph-mon[131403]: pgmap v420: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 74 KiB/s rd, 71 MiB/s wr, 10.37k op/s 2023-10-26T06:50:37.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:37 smithi192 ceph-mon[131661]: pgmap v421: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 99 KiB/s rd, 76 MiB/s wr, 11.02k op/s 2023-10-26T06:50:37.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:37 smithi135 ceph-mon[131403]: pgmap v421: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 99 KiB/s rd, 76 MiB/s wr, 11.02k op/s 2023-10-26T06:50:38.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:37 smithi089 ceph-mon[95031]: pgmap v421: 97 pgs: 97 active+clean; 22 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 99 KiB/s rd, 76 MiB/s wr, 11.02k op/s 2023-10-26T06:50:39.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:39 smithi192 ceph-mon[131661]: pgmap v422: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 48 KiB/s rd, 66 MiB/s wr, 9.47k op/s 2023-10-26T06:50:39.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:39 smithi135 ceph-mon[131403]: pgmap v422: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 48 KiB/s rd, 66 MiB/s wr, 9.47k op/s 2023-10-26T06:50:40.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:39 smithi089 ceph-mon[95031]: pgmap v422: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 48 KiB/s rd, 66 MiB/s wr, 9.47k op/s 2023-10-26T06:50:41.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:41 smithi192 ceph-mon[131661]: pgmap v423: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 49 KiB/s rd, 63 MiB/s wr, 9.03k op/s 2023-10-26T06:50:41.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:41 smithi135 ceph-mon[131403]: pgmap v423: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 49 KiB/s rd, 63 MiB/s wr, 9.03k op/s 2023-10-26T06:50:42.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:41 smithi089 ceph-mon[95031]: pgmap v423: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 49 KiB/s rd, 63 MiB/s wr, 9.03k op/s 2023-10-26T06:50:42.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:42 smithi192 ceph-mon[131661]: pgmap v424: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 71 KiB/s rd, 80 MiB/s wr, 11.39k op/s 2023-10-26T06:50:42.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:42 smithi135 ceph-mon[131403]: pgmap v424: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 71 KiB/s rd, 80 MiB/s wr, 11.39k op/s 2023-10-26T06:50:43.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:42 smithi089 ceph-mon[95031]: pgmap v424: 97 pgs: 97 active+clean; 22 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 71 KiB/s rd, 80 MiB/s wr, 11.39k op/s 2023-10-26T06:50:44.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:50:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.117+0000 7fcc55bff700 1 -- 172.21.15.89:0/3934279801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc50067890 msgr2=0x7fcc50067d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.117+0000 7fcc55bff700 1 --2- 172.21.15.89:0/3934279801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc50067890 0x7fcc50067d10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009990 tx=0x7fcc48009ca0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.117+0000 7fcc55bff700 1 -- 172.21.15.89:0/3934279801 shutdown_connections 2023-10-26T06:50:45.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.117+0000 7fcc55bff700 1 --2- 172.21.15.89:0/3934279801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc50067890 0x7fcc50067d10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.117+0000 7fcc55bff700 1 --2- 172.21.15.89:0/3934279801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 0x7fcc50103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.117+0000 7fcc55bff700 1 --2- 172.21.15.89:0/3934279801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc50103690 0x7fcc50103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.117+0000 7fcc55bff700 1 -- 172.21.15.89:0/3934279801 >> 172.21.15.89:0/3934279801 conn(0x7fcc500767f0 msgr2=0x7fcc50076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:45.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.118+0000 7fcc55bff700 1 -- 172.21.15.89:0/3934279801 shutdown_connections 2023-10-26T06:50:45.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.118+0000 7fcc55bff700 1 -- 172.21.15.89:0/3934279801 wait complete. 2023-10-26T06:50:45.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.120+0000 7fcc55bff700 1 Processor -- start 2023-10-26T06:50:45.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.120+0000 7fcc55bff700 1 -- start start 2023-10-26T06:50:45.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc55bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc50067890 0x7fcc5019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc55bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc50103690 0x7fcc5019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc55bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 0x7fcc501a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc55bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a20a0 con 0x7fcc50067890 2023-10-26T06:50:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc55bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a2210 con 0x7fcc50103690 2023-10-26T06:50:45.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc55bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501a23d0 con 0x7fcc501056d0 2023-10-26T06:50:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 0x7fcc501a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.121+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc50103690 0x7fcc5019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc50103690 0x7fcc5019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36128/0 (socket says 172.21.15.89:36128) 2023-10-26T06:50:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 0x7fcc501a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39130/0 (socket says 172.21.15.89:39130) 2023-10-26T06:50:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc46ffd700 1 -- 172.21.15.89:0/2804477020 learned_addr learned my addr 172.21.15.89:0/2804477020 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc4ffff700 1 -- 172.21.15.89:0/2804477020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc50103690 msgr2=0x7fcc5019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc50103690 0x7fcc5019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc4ffff700 1 -- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc50067890 msgr2=0x7fcc5019d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc50067890 0x7fcc5019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc4ffff700 1 -- 172.21.15.89:0/2804477020 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009520 con 0x7fcc501056d0 2023-10-26T06:50:45.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.122+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 0x7fcc501a1a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009990 tx=0x7fcc48005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.123+0000 7fcc4d7fa700 1 -- 172.21.15.89:0/2804477020 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4801d070 con 0x7fcc501056d0 2023-10-26T06:50:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.123+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc480095f0 con 0x7fcc501056d0 2023-10-26T06:50:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.123+0000 7fcc4d7fa700 1 -- 172.21.15.89:0/2804477020 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc48022470 con 0x7fcc501056d0 2023-10-26T06:50:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.123+0000 7fcc4d7fa700 1 -- 172.21.15.89:0/2804477020 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800bee0 con 0x7fcc501056d0 2023-10-26T06:50:45.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.123+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc501a9d10 con 0x7fcc501056d0 2023-10-26T06:50:45.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.130+0000 7fcc4d7fa700 1 -- 172.21.15.89:0/2804477020 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcc4800fa10 con 0x7fcc501056d0 2023-10-26T06:50:45.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.137+0000 7fcc4d7fa700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcc3c0a1670 0x7fcc3c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.137+0000 7fcc4d7fa700 1 -- 172.21.15.89:0/2804477020 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fcc480ce340 con 0x7fcc501056d0 2023-10-26T06:50:45.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.138+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcc3c0a1670 0x7fcc3c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.142+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcc3c0a1670 0x7fcc3c0a3b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e320 tx=0x7fcc4000d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.143+0000 7fcc69614b80 1 Processor -- start 2023-10-26T06:50:45.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.144+0000 7fcc69614b80 1 -- start start 2023-10-26T06:50:45.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.144+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.144+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.144+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1e0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.144+0000 7fcc69614b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd1536e0 con 0x55a2dd1e6500 2023-10-26T06:50:45.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.144+0000 7fcc69614b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd139c60 con 0x55a2dd1e5be0 2023-10-26T06:50:45.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.144+0000 7fcc69614b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd137580 con 0x55a2dd1e05d0 2023-10-26T06:50:45.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.145+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.145+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45418/0 (socket says 172.21.15.89:45418) 2023-10-26T06:50:45.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.145+0000 7fcc46ffd700 1 -- 172.21.15.89:0/535529543 learned_addr learned my addr 172.21.15.89:0/535529543 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:45.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.145+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.146+0000 7fcc46ffd700 1 -- 172.21.15.89:0/535529543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 msgr2=0x55a2dd1e0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:45.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.146+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1e0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.146+0000 7fcc46ffd700 1 -- 172.21.15.89:0/535529543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e5be0 msgr2=0x55a2dd1e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:45.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.146+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.146+0000 7fcc46ffd700 1 -- 172.21.15.89:0/535529543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2dd0f03f0 con 0x55a2dd1e6500 2023-10-26T06:50:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.146+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e0090 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcc38011440 tx=0x7fcc38011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5c06aca0062b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.147+0000 7fcc45ffb700 1 -- 172.21.15.89:0/535529543 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3802a620 con 0x55a2dd1e6500 2023-10-26T06:50:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.147+0000 7fcc45ffb700 1 -- 172.21.15.89:0/535529543 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc38011e80 con 0x55a2dd1e6500 2023-10-26T06:50:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.147+0000 7fcc45ffb700 1 -- 172.21.15.89:0/535529543 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38029b90 con 0x55a2dd1e6500 2023-10-26T06:50:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.148+0000 7fcc69614b80 1 -- 172.21.15.89:0/535529543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e6500 msgr2=0x55a2dd1e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.148+0000 7fcc69614b80 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e0090 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcc38011440 tx=0x7fcc38011750 comp rx=0 tx=0).stop 2023-10-26T06:50:45.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.149+0000 7fcc69614b80 1 -- 172.21.15.89:0/535529543 shutdown_connections 2023-10-26T06:50:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.149+0000 7fcc69614b80 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1e0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.149+0000 7fcc69614b80 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.149+0000 7fcc69614b80 1 --2- 172.21.15.89:0/535529543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e0090 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.149+0000 7fcc69614b80 1 -- 172.21.15.89:0/535529543 >> 172.21.15.89:0/535529543 conn(0x55a2dd154720 msgr2=0x55a2dd1dab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:45.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.149+0000 7fcc69614b80 1 -- 172.21.15.89:0/535529543 shutdown_connections 2023-10-26T06:50:45.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.149+0000 7fcc69614b80 1 -- 172.21.15.89:0/535529543 wait complete. 2023-10-26T06:50:45.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.151+0000 7fcc69614b80 1 -- start start 2023-10-26T06:50:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.151+0000 7fcc69614b80 1 Processor -- start 2023-10-26T06:50:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.153+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.153+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.153+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.153+0000 7fcc69614b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd132e70 con 0x55a2dd1e5be0 2023-10-26T06:50:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.153+0000 7fcc69614b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd0cf710 con 0x55a2dd1e6500 2023-10-26T06:50:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.153+0000 7fcc69614b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd0cf3b0 con 0x55a2dd1e05d0 2023-10-26T06:50:45.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.153+0000 7fcc69614b80 10 client.0 _ll_register_callbacks cb 0x55a2dd019a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.154+0000 7fcc4f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.155+0000 7fcc4f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39158/0 (socket says 172.21.15.89:39158) 2023-10-26T06:50:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.155+0000 7fcc4f7fe700 1 -- 172.21.15.89:0/2422079332 learned_addr learned my addr 172.21.15.89:0/2422079332 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:45.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.155+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.155+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.155+0000 7fcc4ffff700 1 -- 172.21.15.89:0/2422079332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 msgr2=0x55a2dd1ec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.155+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc4ffff700 1 -- 172.21.15.89:0/2422079332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e5be0 msgr2=0x55a2dd1e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc4ffff700 1 -- 172.21.15.89:0/2422079332 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc38009bf0 con 0x55a2dd1e6500 2023-10-26T06:50:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e9fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc480cbad0 tx=0x7fcc480978a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac528106080d30a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc2effd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:50:45.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc2effd700 1 -- 172.21.15.89:0/2422079332 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4809bb70 con 0x55a2dd1e6500 2023-10-26T06:50:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc2effd700 1 -- 172.21.15.89:0/2422079332 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fcc4809b470 con 0x55a2dd1e6500 2023-10-26T06:50:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2dd137ef0 con 0x55a2dd1e6500 2023-10-26T06:50:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc2effd700 1 -- 172.21.15.89:0/2422079332 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc480b3ea0 con 0x55a2dd1e6500 2023-10-26T06:50:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc69614b80 10 client.25862 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:45.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.156+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a2dcf5dde0 con 0x55a2dd1e6500 2023-10-26T06:50:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.157+0000 7fcc2effd700 1 -- 172.21.15.89:0/2422079332 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2dd137ef0 con 0x55a2dd1e6500 2023-10-26T06:50:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.157+0000 7fcc2effd700 1 -- 172.21.15.89:0/2422079332 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc480b2070 con 0x55a2dd1e6500 2023-10-26T06:50:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.157+0000 7fcc69614b80 10 client.25862 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.157+0000 7fcc69614b80 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:45.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.157+0000 7fcc69614b80 20 client.25862 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.157+0000 7fcc69614b80 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd1f9b70 0x55a2dd1fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.158+0000 7fcc69614b80 4 client.25862 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.158+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2dd1af8b0 con 0x55a2dd1f9b70 2023-10-26T06:50:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.158+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd1f9b70 0x55a2dd1fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.159+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd1f9b70 0x55a2dd1fbf90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdb50da47c58ce3 server_cookie=3240b95da591e96 in_seq=0 out_seq=0 2023-10-26T06:50:45.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.159+0000 7fcc2effd700 10 client.25862 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.160+0000 7fcc2effd700 1 -- 172.21.15.89:0/2422079332 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a2dd1af8b0 con 0x55a2dd1f9b70 2023-10-26T06:50:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.160+0000 7fcc2effd700 10 client.25862 handle_command_reply: tid=0 2023-10-26T06:50:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.160+0000 7fcc69614b80 1 client.25862 shutdown 2023-10-26T06:50:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 msgr2=0x55a2dd1e9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e9fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc480cbad0 tx=0x7fcc480978a0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 shutdown_connections 2023-10-26T06:50:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd1f9b70 0x55a2dd1fbf90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1e9fb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 --2- 172.21.15.89:0/2422079332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 >> 172.21.15.89:0/2422079332 conn(0x55a2dd154720 msgr2=0x55a2dd1df080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:45.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.161+0000 7fcc2effd700 0 client.25862 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.163+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 shutdown_connections 2023-10-26T06:50:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.164+0000 7fcc69614b80 1 -- 172.21.15.89:0/2422079332 wait complete. 2023-10-26T06:50:45.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.164+0000 7fcc69614b80 20 client.25862 trim_cache size 0 max 16384 2023-10-26T06:50:45.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:44 smithi135 ceph-mon[131403]: pgmap v425: 97 pgs: 97 active+clean; 22 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 49 KiB/s rd, 80 MiB/s wr, 11.65k op/s 2023-10-26T06:50:45.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 Processor -- start 2023-10-26T06:50:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 -- start start 2023-10-26T06:50:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1e5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dcfb3920 con 0x55a2dd1e05d0 2023-10-26T06:50:45.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd1eae70 con 0x55a2dd1e6500 2023-10-26T06:50:45.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.222+0000 7fcc69614b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd1d7260 con 0x55a2dd1e5be0 2023-10-26T06:50:45.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39172/0 (socket says 172.21.15.89:39172) 2023-10-26T06:50:45.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 -- 172.21.15.89:0/1655655975 learned_addr learned my addr 172.21.15.89:0/1655655975 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:45.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 -- 172.21.15.89:0/1655655975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 msgr2=0x55a2dd1f5d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:50:45.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/1655655975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1f5d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 -- 172.21.15.89:0/1655655975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 msgr2=0x55a2dd1e5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:45.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/1655655975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 -- 172.21.15.89:0/1655655975 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2dcf46c60 con 0x55a2dd1e5be0 2023-10-26T06:50:45.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.223+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/1655655975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1f0ee0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc38000c00 tx=0x7fcc38009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af358d350cd4b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.224+0000 7fcc467fc700 1 -- 172.21.15.89:0/1655655975 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38005240 con 0x55a2dd1e5be0 2023-10-26T06:50:45.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.224+0000 7fcc467fc700 1 -- 172.21.15.89:0/1655655975 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc38003680 con 0x55a2dd1e5be0 2023-10-26T06:50:45.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.224+0000 7fcc467fc700 1 -- 172.21.15.89:0/1655655975 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38035680 con 0x55a2dd1e5be0 2023-10-26T06:50:45.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.224+0000 7fcc69614b80 1 -- 172.21.15.89:0/1655655975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 msgr2=0x55a2dd1f0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.224+0000 7fcc69614b80 1 --2- 172.21.15.89:0/1655655975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1f0ee0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc38000c00 tx=0x7fcc38009420 comp rx=0 tx=0).stop 2023-10-26T06:50:45.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.225+0000 7fcc69614b80 1 -- 172.21.15.89:0/1655655975 shutdown_connections 2023-10-26T06:50:45.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.225+0000 7fcc69614b80 1 --2- 172.21.15.89:0/1655655975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1e5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.225+0000 7fcc69614b80 1 --2- 172.21.15.89:0/1655655975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.225+0000 7fcc69614b80 1 --2- 172.21.15.89:0/1655655975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1f0ee0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.225+0000 7fcc69614b80 1 -- 172.21.15.89:0/1655655975 >> 172.21.15.89:0/1655655975 conn(0x55a2dd154720 msgr2=0x55a2dd1f5630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:45.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.226+0000 7fcc69614b80 1 -- 172.21.15.89:0/1655655975 shutdown_connections 2023-10-26T06:50:45.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.226+0000 7fcc69614b80 1 -- 172.21.15.89:0/1655655975 wait complete. 2023-10-26T06:50:45.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.227+0000 7fcc69614b80 1 -- start start 2023-10-26T06:50:45.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.227+0000 7fcc69614b80 1 Processor -- start 2023-10-26T06:50:45.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.228+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.228+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.228+0000 7fcc69614b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.228+0000 7fcc69614b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd247940 con 0x55a2dd1e05d0 2023-10-26T06:50:45.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.228+0000 7fcc69614b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd247a80 con 0x55a2dd1e6500 2023-10-26T06:50:45.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.228+0000 7fcc69614b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dd247c40 con 0x55a2dd1e5be0 2023-10-26T06:50:45.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc69614b80 10 client.0 _ll_register_callbacks cb 0x55a2dd019a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:45.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1efcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39182/0 (socket says 172.21.15.89:39182) 2023-10-26T06:50:45.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 -- 172.21.15.89:0/214591700 learned_addr learned my addr 172.21.15.89:0/214591700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:45.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc4ffff700 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 -- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 msgr2=0x55a2dd1f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 -- 172.21.15.89:0/214591700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 msgr2=0x55a2dd1ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:45.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ef780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.229+0000 7fcc46ffd700 1 -- 172.21.15.89:0/214591700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2dd0f03f0 con 0x55a2dd1e5be0 2023-10-26T06:50:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.237+0000 7fcc46ffd700 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1efcc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009340 tx=0x7fcc38003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae5017495f3841 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.237+0000 7fcc2effd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:50:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.237+0000 7fcc2effd700 1 -- 172.21.15.89:0/214591700 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38043070 con 0x55a2dd1e5be0 2023-10-26T06:50:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.237+0000 7fcc2effd700 1 -- 172.21.15.89:0/214591700 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fcc380052d0 con 0x55a2dd1e5be0 2023-10-26T06:50:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.237+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc38011e80 con 0x55a2dd1e5be0 2023-10-26T06:50:45.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.237+0000 7fcc2effd700 1 -- 172.21.15.89:0/214591700 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc380054d0 con 0x55a2dd1e5be0 2023-10-26T06:50:45.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.238+0000 7fcc69614b80 10 client.25861 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:45.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.238+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55a2dcf84250 con 0x55a2dd1e5be0 2023-10-26T06:50:45.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.239+0000 7fcc2effd700 1 -- 172.21.15.89:0/214591700 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc38011e80 con 0x55a2dd1e5be0 2023-10-26T06:50:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc2effd700 1 -- 172.21.15.89:0/214591700 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc38031030 con 0x55a2dd1e5be0 2023-10-26T06:50:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc69614b80 10 client.25861 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc69614b80 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:45.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc69614b80 20 client.25861 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc69614b80 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd248f40 0x55a2dd249350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc69614b80 4 client.25861 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:50:45.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a2dd12be50 con 0x55a2dd248f40 2023-10-26T06:50:45.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.242+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd248f40 0x55a2dd249350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.243+0000 7fcc2effd700 10 client.25861 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.243+0000 7fcc4f7fe700 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd248f40 0x55a2dd249350 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0edf10847227d2b server_cookie=ebf024959626d7dd in_seq=0 out_seq=0 2023-10-26T06:50:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.243+0000 7fcc2effd700 1 -- 172.21.15.89:0/214591700 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a2dd12be50 con 0x55a2dd248f40 2023-10-26T06:50:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.243+0000 7fcc2effd700 10 client.25861 handle_command_reply: tid=0 2023-10-26T06:50:45.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.243+0000 7fcc69614b80 1 client.25861 shutdown 2023-10-26T06:50:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 msgr2=0x55a2dd1efcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1efcc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009340 tx=0x7fcc38003e90 comp rx=0 tx=0).stop 2023-10-26T06:50:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 shutdown_connections 2023-10-26T06:50:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a2dd1e05d0 0x55a2dd1ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55a2dd248f40 0x55a2dd249350 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2dd1e6500 0x55a2dd1f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 --2- 172.21.15.89:0/214591700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a2dd1e5be0 0x55a2dd1efcc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 >> 172.21.15.89:0/214591700 conn(0x55a2dd154720 msgr2=0x55a2dd1d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:45.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.244+0000 7fcc2effd700 0 client.25861 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.246+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 shutdown_connections 2023-10-26T06:50:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.246+0000 7fcc69614b80 1 -- 172.21.15.89:0/214591700 wait complete. 2023-10-26T06:50:45.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.246+0000 7fcc69614b80 20 client.25861 trim_cache size 0 max 16384 2023-10-26T06:50:45.250 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:50:45.250 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:50:45.250 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:50:45.250 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:50:45.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.247+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcc3c0a1670 msgr2=0x7fcc3c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.247+0000 7fcc55bff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcc3c0a1670 0x7fcc3c0a3b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e320 tx=0x7fcc4000d040 comp rx=0 tx=0).stop 2023-10-26T06:50:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.247+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 msgr2=0x7fcc501a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.247+0000 7fcc55bff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 0x7fcc501a1a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009990 tx=0x7fcc48005740 comp rx=0 tx=0).stop 2023-10-26T06:50:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 shutdown_connections 2023-10-26T06:50:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcc3c0a1670 0x7fcc3c0a3b30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcc50067890 0x7fcc5019d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc501056d0 0x7fcc501a1a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 --2- 172.21.15.89:0/2804477020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc50103690 0x7fcc5019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 >> 172.21.15.89:0/2804477020 conn(0x7fcc500767f0 msgr2=0x7fcc500fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:45.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 shutdown_connections 2023-10-26T06:50:45.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.248+0000 7fcc55bff700 1 -- 172.21.15.89:0/2804477020 wait complete. 2023-10-26T06:50:45.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:44 smithi089 ceph-mon[95031]: pgmap v425: 97 pgs: 97 active+clean; 22 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 49 KiB/s rd, 80 MiB/s wr, 11.65k op/s 2023-10-26T06:50:45.298 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:50:45.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:50:45.400 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:44 smithi192 ceph-mon[131661]: pgmap v425: 97 pgs: 97 active+clean; 22 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 49 KiB/s rd, 80 MiB/s wr, 11.65k op/s 2023-10-26T06:50:45.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.655+0000 7fbb0ffff700 1 -- 172.21.15.89:0/526530453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 msgr2=0x7fbb08099fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.655+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/526530453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb08099fc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009920 tx=0x7fbb00009c30 comp rx=0 tx=0).stop 2023-10-26T06:50:45.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.655+0000 7fbb0ffff700 1 -- 172.21.15.89:0/526530453 shutdown_connections 2023-10-26T06:50:45.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.655+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/526530453 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb08099fc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.655+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/526530453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08095830 0x7fbb080a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.655+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/526530453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb08094e70 0x7fbb080952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.655+0000 7fbb0ffff700 1 -- 172.21.15.89:0/526530453 >> 172.21.15.89:0/526530453 conn(0x7fbb0800a760 msgr2=0x7fbb0800ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.656+0000 7fbb0ffff700 1 -- 172.21.15.89:0/526530453 shutdown_connections 2023-10-26T06:50:45.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.656+0000 7fbb0ffff700 1 -- 172.21.15.89:0/526530453 wait complete. 2023-10-26T06:50:45.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.658+0000 7fbb0ffff700 1 Processor -- start 2023-10-26T06:50:45.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.658+0000 7fbb0ffff700 1 -- start start 2023-10-26T06:50:45.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.660+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb08094e70 0x7fbb08142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.660+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08095830 0x7fbb081433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.660+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb0813b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.660+0000 7fbb0ffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0813bb50 con 0x7fbb08099be0 2023-10-26T06:50:45.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.660+0000 7fbb0ffff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0813bcc0 con 0x7fbb08094e70 2023-10-26T06:50:45.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.660+0000 7fbb0ffff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0813be80 con 0x7fbb08095830 2023-10-26T06:50:45.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.661+0000 7fbb0f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb0813b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.661+0000 7fbb0f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb0813b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45442/0 (socket says 172.21.15.89:45442) 2023-10-26T06:50:45.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.661+0000 7fbb0f7fe700 1 -- 172.21.15.89:0/3831355439 learned_addr learned my addr 172.21.15.89:0/3831355439 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:45.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.662+0000 7fbb0f7fe700 1 -- 172.21.15.89:0/3831355439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08095830 msgr2=0x7fbb081433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.662+0000 7fbb0f7fe700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08095830 0x7fbb081433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.662+0000 7fbb0f7fe700 1 -- 172.21.15.89:0/3831355439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb08094e70 msgr2=0x7fbb08142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:45.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.662+0000 7fbb0f7fe700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb08094e70 0x7fbb08142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:45.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.662+0000 7fbb0f7fe700 1 -- 172.21.15.89:0/3831355439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00009520 con 0x7fbb08099be0 2023-10-26T06:50:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.663+0000 7fbb0f7fe700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb0813b380 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb1004fd40 tx=0x7fbb1006ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.664+0000 7fbaf7fff700 1 -- 172.21.15.89:0/3831355439 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1006cb40 con 0x7fbb08099be0 2023-10-26T06:50:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.664+0000 7fbaf7fff700 1 -- 172.21.15.89:0/3831355439 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb1006cca0 con 0x7fbb08099be0 2023-10-26T06:50:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.664+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0813c160 con 0x7fbb08099be0 2023-10-26T06:50:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.664+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb08158e00 con 0x7fbb08099be0 2023-10-26T06:50:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.664+0000 7fbaf7fff700 1 -- 172.21.15.89:0/3831355439 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb100705f0 con 0x7fbb08099be0 2023-10-26T06:50:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.666+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc0052e0 con 0x7fbb08099be0 2023-10-26T06:50:45.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.678+0000 7fbaf7fff700 1 -- 172.21.15.89:0/3831355439 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbb10071430 con 0x7fbb08099be0 2023-10-26T06:50:45.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.681+0000 7fbaf7fff700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbaf80a16b0 0x7fbaf80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:45.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.682+0000 7fbaf7fff700 1 -- 172.21.15.89:0/3831355439 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbb100fc8f0 con 0x7fbb08099be0 2023-10-26T06:50:45.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.683+0000 7fbb0effd700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbaf80a16b0 0x7fbaf80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:45.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.685+0000 7fbb0effd700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbaf80a16b0 0x7fbaf80a3b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009fd0 tx=0x7fbb00000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:45.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:45.695+0000 7fbaf7fff700 1 -- 172.21.15.89:0/3831355439 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb100fbfc0 con 0x7fbb08099be0 2023-10-26T06:50:46.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.225+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbafc005150 con 0x7fbb08099be0 2023-10-26T06:50:46.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.228+0000 7fbaf7fff700 1 -- 172.21.15.89:0/3831355439 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fbb100fbfc0 con 0x7fbb08099be0 2023-10-26T06:50:46.230 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:50:46.230 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:50:46.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.233+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbaf80a16b0 msgr2=0x7fbaf80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbaf80a16b0 0x7fbaf80a3b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009fd0 tx=0x7fbb00000bc0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 msgr2=0x7fbb0813b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb0813b380 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb1004fd40 tx=0x7fbb1006ed90 comp rx=0 tx=0).stop 2023-10-26T06:50:46.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 shutdown_connections 2023-10-26T06:50:46.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbaf80a16b0 0x7fbaf80a3b70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb08099be0 0x7fbb0813b380 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb08094e70 0x7fbb08142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 --2- 172.21.15.89:0/3831355439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08095830 0x7fbb081433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 >> 172.21.15.89:0/3831355439 conn(0x7fbb0800a760 msgr2=0x7fbb08092ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:46.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 shutdown_connections 2023-10-26T06:50:46.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.234+0000 7fbb0ffff700 1 -- 172.21.15.89:0/3831355439 wait complete. 2023-10-26T06:50:46.237 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:50:46.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:50:46.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.639+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/417992504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 msgr2=0x7f2b9000cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.639+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/417992504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 0x7f2b9000cda0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2b98066470 tx=0x7f2b98068f80 comp rx=0 tx=0).stop 2023-10-26T06:50:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.640+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/417992504 shutdown_connections 2023-10-26T06:50:46.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.640+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/417992504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 0x7f2b9000cda0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.640+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/417992504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 0x7f2b900a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.640+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/417992504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b900a53d0 0x7f2b900a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.640+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/417992504 >> 172.21.15.89:0/417992504 conn(0x7f2b9001a670 msgr2=0x7f2b9001aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:46.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.640+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/417992504 shutdown_connections 2023-10-26T06:50:46.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.640+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/417992504 wait complete. 2023-10-26T06:50:46.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.643+0000 7f2b9ef9d700 1 Processor -- start 2023-10-26T06:50:46.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.644+0000 7f2b9ef9d700 1 -- start start 2023-10-26T06:50:46.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.647+0000 7f2b9ef9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 0x7f2b9000ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.647+0000 7f2b9ef9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b900a53d0 0x7f2b9000ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.647+0000 7f2b9ef9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 0x7f2b9000b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.647+0000 7f2b9ef9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9000bc00 con 0x7f2b900a53d0 2023-10-26T06:50:46.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.647+0000 7f2b9ef9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9015c770 con 0x7f2b900a4ff0 2023-10-26T06:50:46.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.647+0000 7f2b9ef9d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9015c8b0 con 0x7f2b900aa1c0 2023-10-26T06:50:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.648+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 0x7f2b9000ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.648+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 0x7f2b9000ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36182/0 (socket says 172.21.15.89:36182) 2023-10-26T06:50:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.648+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2322619977 learned_addr learned my addr 172.21.15.89:0/2322619977 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.649+0000 7f2b9e79c700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 0x7f2b9000b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.649+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b900a53d0 0x7f2b9000ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.649+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2322619977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 msgr2=0x7f2b9000b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.649+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 0x7f2b9000b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.649+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b900a53d0 msgr2=0x7f2b9000ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.649+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b900a53d0 0x7f2b9000ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.650+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2322619977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9804ed30 con 0x7f2b900a4ff0 2023-10-26T06:50:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.650+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 0x7f2b9000ccb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400d980 tx=0x7f2b9400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:46.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.652+0000 7f2b8effd700 1 -- 172.21.15.89:0/2322619977 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94009b40 con 0x7f2b900a4ff0 2023-10-26T06:50:46.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.652+0000 7f2b8effd700 1 -- 172.21.15.89:0/2322619977 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b9400de60 con 0x7f2b900a4ff0 2023-10-26T06:50:46.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.652+0000 7f2b8effd700 1 -- 172.21.15.89:0/2322619977 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400f620 con 0x7f2b900a4ff0 2023-10-26T06:50:46.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.653+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9015cb40 con 0x7f2b900a4ff0 2023-10-26T06:50:46.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.654+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9015d000 con 0x7f2b900a4ff0 2023-10-26T06:50:46.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.668+0000 7f2b8effd700 1 -- 172.21.15.89:0/2322619977 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2b94010460 con 0x7f2b900a4ff0 2023-10-26T06:50:46.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.671+0000 7f2b8effd700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b840a16f0 0x7f2b840a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.671+0000 7f2b8effd700 1 -- 172.21.15.89:0/2322619977 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2b940cc530 con 0x7f2b900a4ff0 2023-10-26T06:50:46.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.673+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b840a16f0 0x7f2b840a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.673+0000 7f2bb3b43b80 1 Processor -- start 2023-10-26T06:50:46.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.674+0000 7f2bb3b43b80 1 -- start start 2023-10-26T06:50:46.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.676+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da67d60 0x557e0da68140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.677+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da68680 0x557e0da622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.677+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da67d60 0x557e0da68140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.677+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da67d60 0x557e0da68140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45454/0 (socket says 172.21.15.89:45454) 2023-10-26T06:50:46.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.678+0000 7f2b9d79a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da68680 0x557e0da622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.678+0000 7f2b9d79a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da68680 0x557e0da622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39216/0 (socket says 172.21.15.89:39216) 2023-10-26T06:50:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.678+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2272598524 learned_addr learned my addr 172.21.15.89:0/2272598524 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:46.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.679+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da62c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.679+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0d9d56e0 con 0x557e0da67d60 2023-10-26T06:50:46.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.679+0000 7f2b9e79c700 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da62c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.679+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0d9bbc60 con 0x557e0da62820 2023-10-26T06:50:46.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.679+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0d9b9580 con 0x557e0da68680 2023-10-26T06:50:46.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.680+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b840a16f0 0x7f2b840a3bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005fd0 tx=0x7f2b88005ee0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:46.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.681+0000 7f2b9d79a700 1 -- 172.21.15.89:0/2272598524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 msgr2=0x557e0da62c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.681+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da62c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.681+0000 7f2b9d79a700 1 -- 172.21.15.89:0/2272598524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da67d60 msgr2=0x557e0da68140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.681+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da67d60 0x557e0da68140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.681+0000 7f2b9d79a700 1 -- 172.21.15.89:0/2272598524 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e0d9723f0 con 0x557e0da68680 2023-10-26T06:50:46.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.681+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da68680 0x557e0da622e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800ead0 tx=0x7f2b88019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c604fbbbca50193 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:46.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.682+0000 7f2b83fff700 1 -- 172.21.15.89:0/2272598524 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88022070 con 0x557e0da68680 2023-10-26T06:50:46.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.682+0000 7f2b83fff700 1 -- 172.21.15.89:0/2272598524 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b8800ee90 con 0x557e0da68680 2023-10-26T06:50:46.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.682+0000 7f2b83fff700 1 -- 172.21.15.89:0/2272598524 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88004e90 con 0x557e0da68680 2023-10-26T06:50:46.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.682+0000 7f2b9e79c700 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da62c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:50:46.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.683+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da68680 msgr2=0x557e0da622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.683+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da68680 0x557e0da622e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800ead0 tx=0x7f2b88019000 comp rx=0 tx=0).stop 2023-10-26T06:50:46.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.684+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 shutdown_connections 2023-10-26T06:50:46.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.684+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da67d60 0x557e0da68140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.684+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da62c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.684+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2272598524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da68680 0x557e0da622e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.684+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 >> 172.21.15.89:0/2272598524 conn(0x557e0d9d6720 msgr2=0x557e0da5ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:46.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.685+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 shutdown_connections 2023-10-26T06:50:46.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.685+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2272598524 wait complete. 2023-10-26T06:50:46.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.686+0000 7f2bb3b43b80 1 -- start start 2023-10-26T06:50:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.688+0000 7f2bb3b43b80 1 Processor -- start 2023-10-26T06:50:46.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.690+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da6bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.690+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da6bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.691+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da6bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36194/0 (socket says 172.21.15.89:36194) 2023-10-26T06:50:46.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.691+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2986985980 learned_addr learned my addr 172.21.15.89:0/2986985980 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:46.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.691+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.691+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 0x557e0da6c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.691+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.692+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0d9b4e70 con 0x557e0da68680 2023-10-26T06:50:46.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.692+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0d951710 con 0x557e0da62820 2023-10-26T06:50:46.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.692+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0d9513b0 con 0x557e0da67d60 2023-10-26T06:50:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.693+0000 7f2bb3b43b80 10 client.0 _ll_register_callbacks cb 0x557e0d89ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.693+0000 7f2b9e79c700 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 0x557e0da6c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.693+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2986985980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 msgr2=0x557e0da6bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.693+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da6bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.693+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2986985980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 msgr2=0x557e0da6c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.693+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 0x557e0da6c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.693+0000 7f2b9df9b700 1 -- 172.21.15.89:0/2986985980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8800e7b0 con 0x557e0da62820 2023-10-26T06:50:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.694+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da6bab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b94099480 tx=0x7f2b94099c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9de6e74acd4eabb server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:46.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.694+0000 7f2b817fa700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:50:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.694+0000 7f2b817fa700 1 -- 172.21.15.89:0/2986985980 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94099950 con 0x557e0da62820 2023-10-26T06:50:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.694+0000 7f2b817fa700 1 -- 172.21.15.89:0/2986985980 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2b940a1880 con 0x557e0da62820 2023-10-26T06:50:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.695+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e0d9b9ef0 con 0x557e0da62820 2023-10-26T06:50:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.695+0000 7f2b817fa700 1 -- 172.21.15.89:0/2986985980 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b940989e0 con 0x557e0da62820 2023-10-26T06:50:46.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.695+0000 7f2bb3b43b80 10 client.25874 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.695+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557e0d7dfde0 con 0x557e0da62820 2023-10-26T06:50:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2b817fa700 1 -- 172.21.15.89:0/2986985980 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b94098b40 con 0x557e0da62820 2023-10-26T06:50:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2b817fa700 1 -- 172.21.15.89:0/2986985980 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b94010710 con 0x557e0da62820 2023-10-26T06:50:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2bb3b43b80 10 client.25874 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:46.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2bb3b43b80 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2bb3b43b80 20 client.25874 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0da7beb0 0x557e0da7e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2bb3b43b80 4 client.25874 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:46.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.696+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e0d9b9ef0 con 0x557e0da7beb0 2023-10-26T06:50:46.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.697+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0da7beb0 0x557e0da7e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.698+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0da7beb0 0x557e0da7e310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46cb8acd1ede5579 server_cookie=d43ad6c0f4ea40f9 in_seq=0 out_seq=0 2023-10-26T06:50:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.698+0000 7f2b817fa700 10 client.25874 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.698+0000 7f2b817fa700 1 -- 172.21.15.89:0/2986985980 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e0d9b9ef0 con 0x557e0da7beb0 2023-10-26T06:50:46.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.699+0000 7f2b817fa700 10 client.25874 handle_command_reply: tid=0 2023-10-26T06:50:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.699+0000 7f2bb3b43b80 1 client.25874 shutdown 2023-10-26T06:50:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.699+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 msgr2=0x557e0da6bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.699+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da6bab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b94099480 tx=0x7f2b94099c70 comp rx=0 tx=0).stop 2023-10-26T06:50:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.700+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 shutdown_connections 2023-10-26T06:50:46.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.700+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 0x557e0da6c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.700+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0da7beb0 0x557e0da7e310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.700+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da6bab0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.700+0000 7f2b817fa700 0 client.25874 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:46.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.700+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/2986985980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da6bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.700+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 >> 172.21.15.89:0/2986985980 conn(0x557e0d9d6720 msgr2=0x557e0da612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:46.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.702+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 shutdown_connections 2023-10-26T06:50:46.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.702+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/2986985980 wait complete. 2023-10-26T06:50:46.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.704+0000 7f2bb3b43b80 20 client.25874 trim_cache size 0 max 16384 2023-10-26T06:50:46.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.731+0000 7f2bb3b43b80 1 Processor -- start 2023-10-26T06:50:46.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.732+0000 7f2bb3b43b80 1 -- start start 2023-10-26T06:50:46.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.732+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da62820 0x557e0da622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.732+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da57820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.732+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da68680 0x557e0da73060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.732+0000 7f2bb3b43b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0d835920 con 0x557e0da62820 2023-10-26T06:50:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.732+0000 7f2bb3b43b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0da5e580 con 0x557e0da68680 2023-10-26T06:50:46.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.732+0000 7f2bb3b43b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0da63190 con 0x557e0da67d60 2023-10-26T06:50:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.733+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da62820 0x557e0da622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.733+0000 7f2b9e79c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da68680 0x557e0da73060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.733+0000 7f2b9d79a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da57820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.733+0000 7f2b9e79c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da68680 0x557e0da73060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36196/0 (socket says 172.21.15.89:36196) 2023-10-26T06:50:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.733+0000 7f2b9d79a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da57820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39236/0 (socket says 172.21.15.89:39236) 2023-10-26T06:50:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.733+0000 7f2b9e79c700 1 -- 172.21.15.89:0/1021917127 learned_addr learned my addr 172.21.15.89:0/1021917127 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.734+0000 7f2b9d79a700 1 -- 172.21.15.89:0/1021917127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da68680 msgr2=0x557e0da73060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.734+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/1021917127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da68680 0x557e0da73060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.734+0000 7f2b9d79a700 1 -- 172.21.15.89:0/1021917127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da62820 msgr2=0x557e0da622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.734+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/1021917127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da62820 0x557e0da622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.734+0000 7f2b9d79a700 1 -- 172.21.15.89:0/1021917127 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e0d7c8c60 con 0x557e0da67d60 2023-10-26T06:50:46.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.736+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/1021917127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da57820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x557e0da6d080 tx=0x7f2b88004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d793089a8deaf9c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:46.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.736+0000 7f2b8cff9700 1 -- 172.21.15.89:0/1021917127 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88022040 con 0x557e0da67d60 2023-10-26T06:50:46.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.736+0000 7f2b8cff9700 1 -- 172.21.15.89:0/1021917127 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b88023040 con 0x557e0da67d60 2023-10-26T06:50:46.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.736+0000 7f2b8cff9700 1 -- 172.21.15.89:0/1021917127 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88013b80 con 0x557e0da67d60 2023-10-26T06:50:46.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.737+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/1021917127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 msgr2=0x557e0da57820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.737+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/1021917127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da57820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x557e0da6d080 tx=0x7f2b88004a70 comp rx=0 tx=0).stop 2023-10-26T06:50:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.737+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/1021917127 shutdown_connections 2023-10-26T06:50:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.738+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/1021917127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da62820 0x557e0da622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.738+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/1021917127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da68680 0x557e0da73060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.738+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/1021917127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da57820 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.738+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/1021917127 >> 172.21.15.89:0/1021917127 conn(0x557e0d9d6720 msgr2=0x557e0da6a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:46.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.738+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/1021917127 shutdown_connections 2023-10-26T06:50:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.738+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/1021917127 wait complete. 2023-10-26T06:50:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.738+0000 7f2bb3b43b80 1 -- start start 2023-10-26T06:50:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.739+0000 7f2bb3b43b80 1 Processor -- start 2023-10-26T06:50:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da72f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da71450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2bb3b43b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 0x557e0da71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2bb3b43b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0da72100 con 0x557e0da68680 2023-10-26T06:50:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2bb3b43b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0da72270 con 0x557e0da62820 2023-10-26T06:50:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2bb3b43b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0dad8540 con 0x557e0da67d60 2023-10-26T06:50:46.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2bb3b43b80 10 client.0 _ll_register_callbacks cb 0x557e0d89ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da72f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.740+0000 7f2b9d79a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da71450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.741+0000 7f2b9df9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da72f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36206/0 (socket says 172.21.15.89:36206) 2023-10-26T06:50:46.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.741+0000 7f2b9df9b700 1 -- 172.21.15.89:0/3156426786 learned_addr learned my addr 172.21.15.89:0/3156426786 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:46.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.741+0000 7f2b9d79a700 1 -- 172.21.15.89:0/3156426786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 msgr2=0x557e0da72f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.741+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da72f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.741+0000 7f2b9d79a700 1 -- 172.21.15.89:0/3156426786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 msgr2=0x557e0da71a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:46.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.741+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 0x557e0da71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.741+0000 7f2b9d79a700 1 -- 172.21.15.89:0/3156426786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e0d9723f0 con 0x557e0da67d60 2023-10-26T06:50:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.742+0000 7f2b9d79a700 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da71450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x557e0da6ef70 tx=0x7f2b88004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352744d97595ed31 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.742+0000 7f2b817fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:50:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.742+0000 7f2b817fa700 1 -- 172.21.15.89:0/3156426786 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88022040 con 0x557e0da67d60 2023-10-26T06:50:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.742+0000 7f2b817fa700 1 -- 172.21.15.89:0/3156426786 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2b8802cde0 con 0x557e0da67d60 2023-10-26T06:50:46.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.742+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b8800ee90 con 0x557e0da67d60 2023-10-26T06:50:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.743+0000 7f2b817fa700 1 -- 172.21.15.89:0/3156426786 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b8802c350 con 0x557e0da67d60 2023-10-26T06:50:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.743+0000 7f2bb3b43b80 10 client.25879 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.743+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557e0d806250 con 0x557e0da67d60 2023-10-26T06:50:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.743+0000 7f2b817fa700 1 -- 172.21.15.89:0/3156426786 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b8800ee90 con 0x557e0da67d60 2023-10-26T06:50:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.743+0000 7f2b817fa700 1 -- 172.21.15.89:0/3156426786 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b88011880 con 0x557e0da67d60 2023-10-26T06:50:46.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.744+0000 7f2bb3b43b80 10 client.25879 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.744+0000 7f2bb3b43b80 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.744+0000 7f2bb3b43b80 20 client.25879 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.744+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0dad9800 0x557e0dadbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.744+0000 7f2bb3b43b80 4 client.25879 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:50:46.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.744+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e0da318b0 con 0x557e0dad9800 2023-10-26T06:50:46.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.744+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0dad9800 0x557e0dadbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:46.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.751+0000 7f2b817fa700 10 client.25879 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:46.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.751+0000 7f2b9df9b700 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0dad9800 0x557e0dadbbf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24b71c5050a6c34 server_cookie=9613c19cfdcbb3de in_seq=0 out_seq=0 2023-10-26T06:50:46.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.752+0000 7f2b817fa700 1 -- 172.21.15.89:0/3156426786 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e0da318b0 con 0x557e0dad9800 2023-10-26T06:50:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.752+0000 7f2b817fa700 10 client.25879 handle_command_reply: tid=0 2023-10-26T06:50:46.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.753+0000 7f2bb3b43b80 1 client.25879 shutdown 2023-10-26T06:50:46.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.756+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 msgr2=0x557e0da71450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.759 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.756+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da71450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x557e0da6ef70 tx=0x7f2b88004920 comp rx=0 tx=0).stop 2023-10-26T06:50:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.758+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 shutdown_connections 2023-10-26T06:50:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.758+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e0da68680 0x557e0da71a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.758+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e0dad9800 0x557e0dadbbf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.758+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e0da62820 0x557e0da72f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.758+0000 7f2bb3b43b80 1 --2- 172.21.15.89:0/3156426786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e0da67d60 0x557e0da71450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.760 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.758+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 >> 172.21.15.89:0/3156426786 conn(0x557e0d9d6720 msgr2=0x557e0da56700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:46.761 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.758+0000 7f2b817fa700 0 client.25879 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:46.762 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.762+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 shutdown_connections 2023-10-26T06:50:46.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.762+0000 7f2bb3b43b80 1 -- 172.21.15.89:0/3156426786 wait complete. 2023-10-26T06:50:46.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.763+0000 7f2bb3b43b80 20 client.25879 trim_cache size 0 max 16384 2023-10-26T06:50:46.765 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:50:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b840a16f0 msgr2=0x7f2b840a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b840a16f0 0x7f2b840a3bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005fd0 tx=0x7f2b88005ee0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 msgr2=0x7f2b9000ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 0x7f2b9000ccb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400d980 tx=0x7f2b9400dd40 comp rx=0 tx=0).stop 2023-10-26T06:50:46.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 shutdown_connections 2023-10-26T06:50:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2b840a16f0 0x7f2b840a3bb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b900aa1c0 0x7f2b9000b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b900a4ff0 0x7f2b9000ccb0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400d980 tx=0x7f2b9400dd40 comp rx=0 tx=0).stop 2023-10-26T06:50:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 --2- 172.21.15.89:0/2322619977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b900a53d0 0x7f2b9000ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:46.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.764+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 >> 172.21.15.89:0/2322619977 conn(0x7f2b9001a670 msgr2=0x7f2b90006a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:46.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.765+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 shutdown_connections 2023-10-26T06:50:46.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:46.765+0000 7f2b9ef9d700 1 -- 172.21.15.89:0/2322619977 wait complete. 2023-10-26T06:50:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:47 smithi135 ceph-mon[131403]: pgmap v426: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 69 KiB/s rd, 95 MiB/s wr, 14.70k op/s 2023-10-26T06:50:47.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:47 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3831355439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:50:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:47 smithi089 ceph-mon[95031]: pgmap v426: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 69 KiB/s rd, 95 MiB/s wr, 14.70k op/s 2023-10-26T06:50:47.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:47 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3831355439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:50:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:47 smithi192 ceph-mon[131661]: pgmap v426: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 69 KiB/s rd, 95 MiB/s wr, 14.70k op/s 2023-10-26T06:50:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:47 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3831355439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:50:47.797 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:50:47.797 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:50:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.167+0000 7ff3da9ff700 1 -- 172.21.15.89:0/4199732667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 msgr2=0x7ff3d410d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.167+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/4199732667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 0x7ff3d410d610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00b0a0 tx=0x7ff3cc00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.169+0000 7ff3da9ff700 1 -- 172.21.15.89:0/4199732667 shutdown_connections 2023-10-26T06:50:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.169+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/4199732667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 0x7ff3d410d610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.169+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/4199732667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d4106750 0x7ff3d4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.169+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/4199732667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d4100740 0x7ff3d4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.169+0000 7ff3da9ff700 1 -- 172.21.15.89:0/4199732667 >> 172.21.15.89:0/4199732667 conn(0x7ff3d40fc230 msgr2=0x7ff3d40fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.170+0000 7ff3da9ff700 1 -- 172.21.15.89:0/4199732667 shutdown_connections 2023-10-26T06:50:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.170+0000 7ff3da9ff700 1 -- 172.21.15.89:0/4199732667 wait complete. 2023-10-26T06:50:48.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.171+0000 7ff3da9ff700 1 Processor -- start 2023-10-26T06:50:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.172+0000 7ff3da9ff700 1 -- start start 2023-10-26T06:50:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.172+0000 7ff3da9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d4100740 0x7ff3d41a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.172+0000 7ff3da9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 0x7ff3d41a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.172+0000 7ff3da9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d410c7d0 0x7ff3d410cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.172+0000 7ff3da9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d410d2b0 con 0x7ff3d410c7d0 2023-10-26T06:50:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.172+0000 7ff3da9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d410d420 con 0x7ff3d4100740 2023-10-26T06:50:48.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.172+0000 7ff3da9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d410d5e0 con 0x7ff3d4101100 2023-10-26T06:50:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d410c7d0 0x7ff3d410cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d4100740 0x7ff3d41a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 0x7ff3d41a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d410c7d0 0x7ff3d410cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45504/0 (socket says 172.21.15.89:45504) 2023-10-26T06:50:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d4100740 0x7ff3d41a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36214/0 (socket says 172.21.15.89:36214) 2023-10-26T06:50:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3682344378 learned_addr learned my addr 172.21.15.89:0/3682344378 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3682344378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 msgr2=0x7ff3d41a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 0x7ff3d41a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3682344378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d4100740 msgr2=0x7ff3d41a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d4100740 0x7ff3d41a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.173+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3682344378 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c8009520 con 0x7ff3d410c7d0 2023-10-26T06:50:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.174+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d410c7d0 0x7ff3d410cc50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009fd0 tx=0x7ff3cc004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.174+0000 7ff3d17fa700 1 -- 172.21.15.89:0/3682344378 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00e040 con 0x7ff3d410c7d0 2023-10-26T06:50:48.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.174+0000 7ff3d17fa700 1 -- 172.21.15.89:0/3682344378 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3cc00be00 con 0x7ff3d410c7d0 2023-10-26T06:50:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.174+0000 7ff3d17fa700 1 -- 172.21.15.89:0/3682344378 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00e040 con 0x7ff3d410c7d0 2023-10-26T06:50:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.174+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc009d00 con 0x7ff3d410c7d0 2023-10-26T06:50:48.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.174+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d41aa140 con 0x7ff3d410c7d0 2023-10-26T06:50:48.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.180+0000 7ff3d17fa700 1 -- 172.21.15.89:0/3682344378 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff3cc0043f0 con 0x7ff3d410c7d0 2023-10-26T06:50:48.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.181+0000 7ff3ee24eb80 1 Processor -- start 2023-10-26T06:50:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.181+0000 7ff3ee24eb80 1 -- start start 2023-10-26T06:50:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca971040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca96b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca96bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca8de760 con 0x555bca96b650 2023-10-26T06:50:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca8c4ce0 con 0x555bca971580 2023-10-26T06:50:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca8c2600 con 0x555bca970c60 2023-10-26T06:50:48.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca96b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca96b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36216/0 (socket says 172.21.15.89:36216) 2023-10-26T06:50:48.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.182+0000 7ff3d3fff700 1 -- 172.21.15.89:0/3073427537 learned_addr learned my addr 172.21.15.89:0/3073427537 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.183+0000 7ff3d3fff700 1 -- 172.21.15.89:0/3073427537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 msgr2=0x555bca971040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:50:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.183+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/3073427537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca971040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.183+0000 7ff3d3fff700 1 -- 172.21.15.89:0/3073427537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 msgr2=0x555bca96bad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:50:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.183+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/3073427537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca96bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.183+0000 7ff3d3fff700 1 -- 172.21.15.89:0/3073427537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bca87b470 con 0x555bca971580 2023-10-26T06:50:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.183+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/3073427537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca96b110 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4006ab0 tx=0x7ff3c400bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f7dd1077c70d36 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.183+0000 7ff3ba7fc700 1 -- 172.21.15.89:0/3073427537 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40125f0 con 0x555bca971580 2023-10-26T06:50:48.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.185+0000 7ff3d17fa700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff3bc0a16b0 0x7ff3bc0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.185+0000 7ff3d17fa700 1 -- 172.21.15.89:0/3682344378 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff3cc0cdd80 con 0x7ff3d410c7d0 2023-10-26T06:50:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.186+0000 7ff3ba7fc700 1 -- 172.21.15.89:0/3073427537 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c4012c10 con 0x555bca971580 2023-10-26T06:50:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.186+0000 7ff3ba7fc700 1 -- 172.21.15.89:0/3073427537 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4014d20 con 0x555bca971580 2023-10-26T06:50:48.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3073427537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 msgr2=0x555bca96b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3073427537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca96b110 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4006ab0 tx=0x7ff3c400bb10 comp rx=0 tx=0).stop 2023-10-26T06:50:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3073427537 shutdown_connections 2023-10-26T06:50:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff3bc0a16b0 0x7ff3bc0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3073427537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca96bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3073427537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca971040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3073427537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca96b110 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.187+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3073427537 >> 172.21.15.89:0/3073427537 conn(0x555bca8df7a0 msgr2=0x555bca965be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.188+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3073427537 shutdown_connections 2023-10-26T06:50:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.188+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3073427537 wait complete. 2023-10-26T06:50:48.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.188+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff3bc0a16b0 0x7ff3bc0a3b70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80094f0 tx=0x7ff3c8004750 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.188+0000 7ff3ee24eb80 1 -- start start 2023-10-26T06:50:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.189+0000 7ff3ee24eb80 1 Processor -- start 2023-10-26T06:50:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.189+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca977530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.189+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.189+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca974de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.189+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca8bdef0 con 0x555bca96b650 2023-10-26T06:50:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.189+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca85a790 con 0x555bca971580 2023-10-26T06:50:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.189+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca85a430 con 0x555bca970c60 2023-10-26T06:50:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.190+0000 7ff3ee24eb80 10 client.0 _ll_register_callbacks cb 0x555bca7a4b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:48.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.190+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca977530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.190+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca977530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45508/0 (socket says 172.21.15.89:45508) 2023-10-26T06:50:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.190+0000 7ff3d3fff700 1 -- 172.21.15.89:0/2825124520 learned_addr learned my addr 172.21.15.89:0/2825124520 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.190+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca974de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.190+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.192+0000 7ff3d3fff700 1 -- 172.21.15.89:0/2825124520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 msgr2=0x555bca9747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.192+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.192+0000 7ff3d3fff700 1 -- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 msgr2=0x555bca974de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.192+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca974de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.192+0000 7ff3d3fff700 1 -- 172.21.15.89:0/2825124520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c400e040 con 0x555bca96b650 2023-10-26T06:50:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.193+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca977530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4012fb0 tx=0x7ff3c4012fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a22a13e66bc205 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.193+0000 7ff3b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:50:48.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.193+0000 7ff3b37fe700 1 -- 172.21.15.89:0/2825124520 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40146c0 con 0x555bca96b650 2023-10-26T06:50:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.193+0000 7ff3b37fe700 1 -- 172.21.15.89:0/2825124520 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff3c4012800 con 0x555bca96b650 2023-10-26T06:50:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.193+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bca8c2f70 con 0x555bca96b650 2023-10-26T06:50:48.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.194+0000 7ff3b37fe700 1 -- 172.21.15.89:0/2825124520 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3c40129e0 con 0x555bca96b650 2023-10-26T06:50:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.196+0000 7ff3ee24eb80 10 client.15852 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.196+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555bca6e8e60 con 0x555bca96b650 2023-10-26T06:50:48.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.197+0000 7ff3b37fe700 1 -- 172.21.15.89:0/2825124520 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bca8c2f70 con 0x555bca96b650 2023-10-26T06:50:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.197+0000 7ff3b37fe700 1 -- 172.21.15.89:0/2825124520 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3c40036f0 con 0x555bca96b650 2023-10-26T06:50:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.197+0000 7ff3ee24eb80 10 client.15852 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.198+0000 7ff3ee24eb80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.198+0000 7ff3ee24eb80 20 client.15852 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:48.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.198+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca995da0 0x555bca9981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.198+0000 7ff3ee24eb80 4 client.15852 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.198+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bca93a930 con 0x555bca995da0 2023-10-26T06:50:48.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.198+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca995da0 0x555bca9981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.199+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca995da0 0x555bca9981c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bdbc5ae03d6872 server_cookie=571f5894b9653a09 in_seq=0 out_seq=0 2023-10-26T06:50:48.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.199+0000 7ff3b37fe700 10 client.15852 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.200+0000 7ff3b37fe700 1 -- 172.21.15.89:0/2825124520 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555bca93a930 con 0x555bca995da0 2023-10-26T06:50:48.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.200+0000 7ff3b37fe700 10 client.15852 handle_command_reply: tid=0 2023-10-26T06:50:48.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.201+0000 7ff3ee24eb80 1 client.15852 shutdown 2023-10-26T06:50:48.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.201+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 msgr2=0x555bca977530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.201+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca977530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4012fb0 tx=0x7ff3c4012fe0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 shutdown_connections 2023-10-26T06:50:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca96b650 0x555bca977530 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca995da0 0x555bca9981c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca971580 0x555bca974de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/2825124520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 >> 172.21.15.89:0/2825124520 conn(0x555bca8df7a0 msgr2=0x555bca96a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3b37fe700 0 client.15852 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 shutdown_connections 2023-10-26T06:50:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.202+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/2825124520 wait complete. 2023-10-26T06:50:48.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.204+0000 7ff3ee24eb80 20 client.15852 trim_cache size 0 max 16384 2023-10-26T06:50:48.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.233+0000 7ff3ee24eb80 1 Processor -- start 2023-10-26T06:50:48.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.234+0000 7ff3ee24eb80 1 -- start start 2023-10-26T06:50:48.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.234+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca97bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.234+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.234+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca9735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.234+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca73e9a0 con 0x555bca971580 2023-10-26T06:50:48.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.234+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca9753e0 con 0x555bca96b650 2023-10-26T06:50:48.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.234+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca96b030 con 0x555bca970c60 2023-10-26T06:50:48.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.235+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca97bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.235+0000 7ff3d3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca97bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36232/0 (socket says 172.21.15.89:36232) 2023-10-26T06:50:48.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.235+0000 7ff3d3fff700 1 -- 172.21.15.89:0/609978903 learned_addr learned my addr 172.21.15.89:0/609978903 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.235+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.235+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca9735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.236+0000 7ff3d3fff700 1 -- 172.21.15.89:0/609978903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 msgr2=0x555bca9730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.236+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.236+0000 7ff3d3fff700 1 -- 172.21.15.89:0/609978903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 msgr2=0x555bca9735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.236+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca9735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.236+0000 7ff3d3fff700 1 -- 172.21.15.89:0/609978903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bca965df0 con 0x555bca96b650 2023-10-26T06:50:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.236+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca97bf60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4014580 tx=0x7ff3c400b170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9e49d3518c12c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.236+0000 7ff3baffd700 1 -- 172.21.15.89:0/609978903 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40036f0 con 0x555bca96b650 2023-10-26T06:50:48.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.237+0000 7ff3baffd700 1 -- 172.21.15.89:0/609978903 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c4021070 con 0x555bca96b650 2023-10-26T06:50:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.237+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/609978903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 msgr2=0x555bca97bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.237+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca97bf60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4014580 tx=0x7ff3c400b170 comp rx=0 tx=0).stop 2023-10-26T06:50:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.238+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/609978903 shutdown_connections 2023-10-26T06:50:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.238+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca9735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.238+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca97bf60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.238+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/609978903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.238+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/609978903 >> 172.21.15.89:0/609978903 conn(0x555bca8df7a0 msgr2=0x555bca96a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.238+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/609978903 shutdown_connections 2023-10-26T06:50:48.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.238+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/609978903 wait complete. 2023-10-26T06:50:48.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.239+0000 7ff3ee24eb80 1 -- start start 2023-10-26T06:50:48.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.239+0000 7ff3ee24eb80 1 Processor -- start 2023-10-26T06:50:48.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.240+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca9e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.240+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.240+0000 7ff3ee24eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca97b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.240+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca97b730 con 0x555bca971580 2023-10-26T06:50:48.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.240+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca97b8a0 con 0x555bca96b650 2023-10-26T06:50:48.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.240+0000 7ff3ee24eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bca97ba60 con 0x555bca970c60 2023-10-26T06:50:48.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.240+0000 7ff3ee24eb80 10 client.0 _ll_register_callbacks cb 0x555bca7a4b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:48.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.241+0000 7ff3d8f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca97b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.241+0000 7ff3d8f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca97b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45530/0 (socket says 172.21.15.89:45530) 2023-10-26T06:50:48.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.241+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3673525767 learned_addr learned my addr 172.21.15.89:0/3673525767 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.241+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.241+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca9e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.242+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3673525767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 msgr2=0x555bca9e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.242+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.242+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 msgr2=0x555bca9e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.242+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca9e41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.242+0000 7ff3d8f9c700 1 -- 172.21.15.89:0/3673525767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bca6e8e60 con 0x555bca971580 2023-10-26T06:50:48.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.242+0000 7ff3d37fe700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:50:48.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.242+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca9e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:50:48.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.244+0000 7ff3d8f9c700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca97b050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00b780 tx=0x7ff3cc09a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19feac3655be0792 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.244+0000 7ff3b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:50:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.244+0000 7ff3b37fe700 1 -- 172.21.15.89:0/3673525767 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00e040 con 0x555bca971580 2023-10-26T06:50:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.244+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3c4012c10 con 0x555bca971580 2023-10-26T06:50:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.245+0000 7ff3b37fe700 1 -- 172.21.15.89:0/3673525767 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff3cc09ef80 con 0x555bca971580 2023-10-26T06:50:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.245+0000 7ff3b37fe700 1 -- 172.21.15.89:0/3673525767 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3cc0a3130 con 0x555bca971580 2023-10-26T06:50:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.245+0000 7ff3ee24eb80 10 client.15858 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:48.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.245+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555bca6d1ce0 con 0x555bca971580 2023-10-26T06:50:48.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.246+0000 7ff3b37fe700 1 -- 172.21.15.89:0/3673525767 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c4012c10 con 0x555bca971580 2023-10-26T06:50:48.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.247+0000 7ff3b37fe700 1 -- 172.21.15.89:0/3673525767 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3cc09b090 con 0x555bca971580 2023-10-26T06:50:48.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.247+0000 7ff3ee24eb80 10 client.15858 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:48.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.247+0000 7ff3ee24eb80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:48.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.247+0000 7ff3ee24eb80 20 client.15858 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:48.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.247+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca9ebcd0 0x555bca9ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.247+0000 7ff3ee24eb80 4 client.15858 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:50:48.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.247+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555bca8b6ed0 con 0x555bca9ebcd0 2023-10-26T06:50:48.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.248+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca9ebcd0 0x555bca9ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.249+0000 7ff3d3fff700 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca9ebcd0 0x555bca9ee0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d4f1031ce4ea22 server_cookie=dedd12e00c93cb7d in_seq=0 out_seq=0 2023-10-26T06:50:48.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.249+0000 7ff3b37fe700 10 client.15858 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.250+0000 7ff3b37fe700 1 -- 172.21.15.89:0/3673525767 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555bca8b6ed0 con 0x555bca9ebcd0 2023-10-26T06:50:48.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.250+0000 7ff3b37fe700 10 client.15858 handle_command_reply: tid=0 2023-10-26T06:50:48.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.250+0000 7ff3ee24eb80 1 client.15858 shutdown 2023-10-26T06:50:48.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.250+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 msgr2=0x555bca97b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca97b050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00b780 tx=0x7ff3cc09a840 comp rx=0 tx=0).stop 2023-10-26T06:50:48.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 shutdown_connections 2023-10-26T06:50:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bca971580 0x555bca97b050 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555bca9ebcd0 0x555bca9ee0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bca96b650 0x555bca9e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 --2- 172.21.15.89:0/3673525767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bca970c60 0x555bca9e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 >> 172.21.15.89:0/3673525767 conn(0x555bca8df7a0 msgr2=0x555bca96e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3b37fe700 0 client.15858 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 shutdown_connections 2023-10-26T06:50:48.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.251+0000 7ff3ee24eb80 1 -- 172.21.15.89:0/3673525767 wait complete. 2023-10-26T06:50:48.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.252+0000 7ff3ee24eb80 20 client.15858 trim_cache size 0 max 16384 2023-10-26T06:50:48.256 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:50:48.257 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:50:48.257 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "60c2401f-607c-4469-83c2-14a1662b767a", 2023-10-26T06:50:48.257 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:50:48.257 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:50:48.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.253+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff3bc0a16b0 msgr2=0x7ff3bc0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.253+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff3bc0a16b0 0x7ff3bc0a3b70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80094f0 tx=0x7ff3c8004750 comp rx=0 tx=0).stop 2023-10-26T06:50:48.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.253+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d410c7d0 msgr2=0x7ff3d410cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.253+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d410c7d0 0x7ff3d410cc50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009fd0 tx=0x7ff3cc004d30 comp rx=0 tx=0).stop 2023-10-26T06:50:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.253+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 shutdown_connections 2023-10-26T06:50:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.254+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff3bc0a16b0 0x7ff3bc0a3b70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.254+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3d410c7d0 0x7ff3d410cc50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.254+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3d4100740 0x7ff3d41a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.254+0000 7ff3da9ff700 1 --2- 172.21.15.89:0/3682344378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3d4101100 0x7ff3d41a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.254+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 >> 172.21.15.89:0/3682344378 conn(0x7ff3d40fc230 msgr2=0x7ff3d40fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.254+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 shutdown_connections 2023-10-26T06:50:48.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.254+0000 7ff3da9ff700 1 -- 172.21.15.89:0/3682344378 wait complete. 2023-10-26T06:50:48.289 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:50:48.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.617+0000 7f51ff7ff700 1 -- 172.21.15.89:0/3590116117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810ef10 msgr2=0x7f51f81077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.617+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/3590116117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810ef10 0x7f51f81077f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009990 tx=0x7f51f4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.617+0000 7f51ff7ff700 1 -- 172.21.15.89:0/3590116117 shutdown_connections 2023-10-26T06:50:48.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.617+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/3590116117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f8119fd0 0x7f51f811c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.617+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/3590116117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810ef10 0x7f51f81077f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.617+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/3590116117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51f810f380 0x7f51f8107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.617+0000 7f51ff7ff700 1 -- 172.21.15.89:0/3590116117 >> 172.21.15.89:0/3590116117 conn(0x7f51f806b840 msgr2=0x7f51f806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.618+0000 7f51ff7ff700 1 -- 172.21.15.89:0/3590116117 shutdown_connections 2023-10-26T06:50:48.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.618+0000 7f51ff7ff700 1 -- 172.21.15.89:0/3590116117 wait complete. 2023-10-26T06:50:48.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.619+0000 7f51ff7ff700 1 Processor -- start 2023-10-26T06:50:48.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.619+0000 7f51ff7ff700 1 -- start start 2023-10-26T06:50:48.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51ff7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f810ef10 0x7f51f81ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51ff7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810f380 0x7f51f81abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51ff7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51f8119fd0 0x7f51f81a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51ff7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81a6050 con 0x7f51f8119fd0 2023-10-26T06:50:48.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51ff7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81a61c0 con 0x7f51f810ef10 2023-10-26T06:50:48.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51ff7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81a6380 con 0x7f51f810f380 2023-10-26T06:50:48.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51fcd9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810f380 0x7f51f81abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51fd59b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f810ef10 0x7f51f81ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51fd59b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f810ef10 0x7f51f81ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36260/0 (socket says 172.21.15.89:36260) 2023-10-26T06:50:48.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51fd59b700 1 -- 172.21.15.89:0/555176138 learned_addr learned my addr 172.21.15.89:0/555176138 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.620+0000 7f51fdd9c700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51f8119fd0 0x7f51f81a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.621+0000 7f51fd59b700 1 -- 172.21.15.89:0/555176138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810f380 msgr2=0x7f51f81abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.621+0000 7f51fd59b700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810f380 0x7f51f81abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.621+0000 7f51fd59b700 1 -- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51f8119fd0 msgr2=0x7f51f81a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.621+0000 7f51fd59b700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51f8119fd0 0x7f51f81a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.621+0000 7f51fd59b700 1 -- 172.21.15.89:0/555176138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f40095f0 con 0x7f51f810ef10 2023-10-26T06:50:48.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.621+0000 7f51fd59b700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f810ef10 0x7f51f81ab8c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f4005910 tx=0x7f51f4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.621+0000 7f51ea7fc700 1 -- 172.21.15.89:0/555176138 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f401d070 con 0x7f51f810ef10 2023-10-26T06:50:48.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.622+0000 7f51ea7fc700 1 -- 172.21.15.89:0/555176138 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f4022470 con 0x7f51f810ef10 2023-10-26T06:50:48.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.622+0000 7f51ea7fc700 1 -- 172.21.15.89:0/555176138 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4017650 con 0x7f51f810ef10 2023-10-26T06:50:48.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.622+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f81a6600 con 0x7f51f810ef10 2023-10-26T06:50:48.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.622+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f81b26d0 con 0x7f51f810ef10 2023-10-26T06:50:48.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.630+0000 7f51ea7fc700 1 -- 172.21.15.89:0/555176138 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f51f40225e0 con 0x7f51f810ef10 2023-10-26T06:50:48.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.630+0000 7f52131c2b80 1 Processor -- start 2023-10-26T06:50:48.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.631+0000 7f52131c2b80 1 -- start start 2023-10-26T06:50:48.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.632+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be536fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.633+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be531090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.633+0000 7f51ea7fc700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51e40a16f0 0x7f51e40a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.633+0000 7f51fdd9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be536fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.633+0000 7f51fdd9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be536fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39320/0 (socket says 172.21.15.89:39320) 2023-10-26T06:50:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.633+0000 7f51ea7fc700 1 -- 172.21.15.89:0/555176138 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f51f40ce130 con 0x7f51f810ef10 2023-10-26T06:50:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be531a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fdd9c700 1 -- 172.21.15.89:0/4070002796 learned_addr learned my addr 172.21.15.89:0/4070002796 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be4a46e0 con 0x5637be5315d0 2023-10-26T06:50:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be48ac60 con 0x5637be537500 2023-10-26T06:50:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be488580 con 0x5637be536be0 2023-10-26T06:50:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51e40a16f0 0x7f51e40a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fd59b700 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be531090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fdd9c700 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be531a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fdd9c700 1 -- 172.21.15.89:0/4070002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 msgr2=0x5637be531090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fdd9c700 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be531090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fdd9c700 1 -- 172.21.15.89:0/4070002796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 msgr2=0x5637be531a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fdd9c700 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be531a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.634+0000 7f51fdd9c700 1 -- 172.21.15.89:0/4070002796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637be4413f0 con 0x5637be536be0 2023-10-26T06:50:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.635+0000 7f51fdd9c700 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be536fc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51f000a480 tx=0x7f51f0009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b248e3bed7d405 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.635+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51e40a16f0 0x7f51e40a3bb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f51f81a6bb0 tx=0x7f51ec009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.635+0000 7f51e37fe700 1 -- 172.21.15.89:0/4070002796 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0018420 con 0x5637be536be0 2023-10-26T06:50:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.635+0000 7f51e37fe700 1 -- 172.21.15.89:0/4070002796 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f0017880 con 0x5637be536be0 2023-10-26T06:50:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.635+0000 7f51e37fe700 1 -- 172.21.15.89:0/4070002796 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0016730 con 0x5637be536be0 2023-10-26T06:50:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.636+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 msgr2=0x5637be536fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.636+0000 7f52131c2b80 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be536fc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51f000a480 tx=0x7f51f0009650 comp rx=0 tx=0).stop 2023-10-26T06:50:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.637+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 shutdown_connections 2023-10-26T06:50:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.637+0000 7f52131c2b80 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be531a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.637+0000 7f52131c2b80 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be536fc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.637+0000 7f52131c2b80 1 --2- 172.21.15.89:0/4070002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be531090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.637+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 >> 172.21.15.89:0/4070002796 conn(0x5637be4a5720 msgr2=0x5637be52bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.637+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 shutdown_connections 2023-10-26T06:50:48.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.637+0000 7f52131c2b80 1 -- 172.21.15.89:0/4070002796 wait complete. 2023-10-26T06:50:48.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.638+0000 7f52131c2b80 1 -- start start 2023-10-26T06:50:48.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.639+0000 7f52131c2b80 1 Processor -- start 2023-10-26T06:50:48.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.639+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be53d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.639+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be536be0 0x5637be53a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.639+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be537500 0x5637be53aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f52131c2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be483e70 con 0x5637be5315d0 2023-10-26T06:50:48.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f52131c2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be420710 con 0x5637be536be0 2023-10-26T06:50:48.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f52131c2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be4203b0 con 0x5637be537500 2023-10-26T06:50:48.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f52131c2b80 10 client.0 _ll_register_callbacks cb 0x5637be36aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:48.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f51fcd9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be536be0 0x5637be53a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f51fdd9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be537500 0x5637be53aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f51fdd9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be537500 0x5637be53aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39336/0 (socket says 172.21.15.89:39336) 2023-10-26T06:50:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f51fcd9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be536be0 0x5637be53a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:36278/0 (socket says 172.21.15.89:36278) 2023-10-26T06:50:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.640+0000 7f51fdd9c700 1 -- 172.21.15.89:0/52151479 learned_addr learned my addr 172.21.15.89:0/52151479 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.641+0000 7f51fd59b700 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be53d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.641+0000 7f51fcd9a700 1 -- 172.21.15.89:0/52151479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be537500 msgr2=0x5637be53aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.641+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be537500 0x5637be53aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.641+0000 7f51fcd9a700 1 -- 172.21.15.89:0/52151479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 msgr2=0x5637be53d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.641+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be53d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.641+0000 7f51fcd9a700 1 -- 172.21.15.89:0/52151479 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0013040 con 0x5637be536be0 2023-10-26T06:50:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.641+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be536be0 0x5637be53a760 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51f810f780 tx=0x7f51ec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0f5ed645fb4504 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.642+0000 7f51e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:50:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.642+0000 7f51e0ff9700 1 -- 172.21.15.89:0/52151479 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00ee70 con 0x5637be536be0 2023-10-26T06:50:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.642+0000 7f51e0ff9700 1 -- 172.21.15.89:0/52151479 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f51ec004030 con 0x5637be536be0 2023-10-26T06:50:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.643+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637be488ef0 con 0x5637be536be0 2023-10-26T06:50:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.643+0000 7f51e0ff9700 1 -- 172.21.15.89:0/52151479 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51ec0041f0 con 0x5637be536be0 2023-10-26T06:50:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.643+0000 7f52131c2b80 10 client.25904 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.643+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5637be2aede0 con 0x5637be536be0 2023-10-26T06:50:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.644+0000 7f51e0ff9700 1 -- 172.21.15.89:0/52151479 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637be488ef0 con 0x5637be536be0 2023-10-26T06:50:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.644+0000 7f51e0ff9700 1 -- 172.21.15.89:0/52151479 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f51ec01e750 con 0x5637be536be0 2023-10-26T06:50:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.644+0000 7f52131c2b80 10 client.25904 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.644+0000 7f52131c2b80 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.644+0000 7f52131c2b80 20 client.25904 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.644+0000 7f52131c2b80 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be54aee0 0x5637be54d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.645+0000 7f52131c2b80 4 client.25904 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:50:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.645+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637be5008b0 con 0x5637be54aee0 2023-10-26T06:50:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.645+0000 7f51fd59b700 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be54aee0 0x5637be54d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.645+0000 7f51fd59b700 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be54aee0 0x5637be54d300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3fecc676a65f42 server_cookie=ac8b51b5ab183d24 in_seq=0 out_seq=0 2023-10-26T06:50:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.646+0000 7f51e0ff9700 10 client.25904 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.646+0000 7f51e0ff9700 1 -- 172.21.15.89:0/52151479 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5637be5008b0 con 0x5637be54aee0 2023-10-26T06:50:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f51e0ff9700 10 client.25904 handle_command_reply: tid=0 2023-10-26T06:50:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 client.25904 shutdown 2023-10-26T06:50:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be536be0 msgr2=0x5637be53a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be536be0 0x5637be53a760 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51f810f780 tx=0x7f51ec0047c0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 shutdown_connections 2023-10-26T06:50:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be53d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be54aee0 0x5637be54d300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be537500 0x5637be53aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.647+0000 7f52131c2b80 1 --2- 172.21.15.89:0/52151479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be536be0 0x5637be53a760 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.648+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 >> 172.21.15.89:0/52151479 conn(0x5637be4a5720 msgr2=0x5637be530080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.648+0000 7f51e0ff9700 0 client.25904 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.648+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 shutdown_connections 2023-10-26T06:50:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.648+0000 7f52131c2b80 1 -- 172.21.15.89:0/52151479 wait complete. 2023-10-26T06:50:48.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.648+0000 7f52131c2b80 20 client.25904 trim_cache size 0 max 16384 2023-10-26T06:50:48.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 Processor -- start 2023-10-26T06:50:48.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 -- start start 2023-10-26T06:50:48.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be541eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be53d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be53d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be304920 con 0x5637be5315d0 2023-10-26T06:50:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be53b6b0 con 0x5637be537500 2023-10-26T06:50:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.679+0000 7f52131c2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be537350 con 0x5637be536be0 2023-10-26T06:50:48.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.680+0000 7f51fdd9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be53d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.680+0000 7f51fcd9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be53d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.680+0000 7f51fcd9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be53d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39342/0 (socket says 172.21.15.89:39342) 2023-10-26T06:50:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.680+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1286092797 learned_addr learned my addr 172.21.15.89:0/1286092797 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.680+0000 7f51fd59b700 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be541eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.681+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1286092797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 msgr2=0x5637be53d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.681+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be53d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.681+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1286092797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 msgr2=0x5637be541eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.681+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be541eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.681+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1286092797 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637be297c60 con 0x5637be536be0 2023-10-26T06:50:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.681+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be53d010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0097c0 tx=0x7f51ec005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=541b125baa397dd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.682+0000 7f51e3fff700 1 -- 172.21.15.89:0/1286092797 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec035070 con 0x5637be536be0 2023-10-26T06:50:48.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.682+0000 7f51e3fff700 1 -- 172.21.15.89:0/1286092797 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51ec004530 con 0x5637be536be0 2023-10-26T06:50:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.682+0000 7f51e3fff700 1 -- 172.21.15.89:0/1286092797 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec003850 con 0x5637be536be0 2023-10-26T06:50:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 -- 172.21.15.89:0/1286092797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 msgr2=0x5637be53d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be53d010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0097c0 tx=0x7f51ec005cc0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 -- 172.21.15.89:0/1286092797 shutdown_connections 2023-10-26T06:50:48.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be5315d0 0x5637be541eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be53d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1286092797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be536be0 0x5637be53d010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 -- 172.21.15.89:0/1286092797 >> 172.21.15.89:0/1286092797 conn(0x5637be4a5720 msgr2=0x5637be52b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 -- 172.21.15.89:0/1286092797 shutdown_connections 2023-10-26T06:50:48.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.683+0000 7f52131c2b80 1 -- 172.21.15.89:0/1286092797 wait complete. 2023-10-26T06:50:48.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.684+0000 7f52131c2b80 1 -- start start 2023-10-26T06:50:48.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.684+0000 7f52131c2b80 1 Processor -- start 2023-10-26T06:50:48.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be5315d0 0x5637be541ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be536be0 0x5637be53ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f52131c2b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be540490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f52131c2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be5409d0 con 0x5637be536be0 2023-10-26T06:50:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f52131c2b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be540b40 con 0x5637be537500 2023-10-26T06:50:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f52131c2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637be540d00 con 0x5637be5315d0 2023-10-26T06:50:48.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f52131c2b80 10 client.0 _ll_register_callbacks cb 0x5637be36aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:50:48.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f51fcd9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be536be0 0x5637be53ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f51fcd9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be536be0 0x5637be53ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45564/0 (socket says 172.21.15.89:45564) 2023-10-26T06:50:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1120709863 learned_addr learned my addr 172.21.15.89:0/1120709863 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:50:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.685+0000 7f51fd59b700 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be5315d0 0x5637be541ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.686+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1120709863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be5315d0 msgr2=0x5637be541ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.686+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be5315d0 0x5637be541ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.686+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1120709863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 msgr2=0x5637be540490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:50:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.686+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be540490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.686+0000 7f51fcd9a700 1 -- 172.21.15.89:0/1120709863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637be2aede0 con 0x5637be536be0 2023-10-26T06:50:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.686+0000 7f51fcd9a700 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be536be0 0x5637be53ff50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f51ec01ec00 tx=0x7f51ec02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6565132209fb02 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:50:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.686+0000 7f51e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:50:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.687+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1120709863 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec02fc80 con 0x5637be536be0 2023-10-26T06:50:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.687+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1120709863 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f51ec02fde0 con 0x5637be536be0 2023-10-26T06:50:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.687+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51f0017880 con 0x5637be536be0 2023-10-26T06:50:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.687+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1120709863 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51ec005030 con 0x5637be536be0 2023-10-26T06:50:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.687+0000 7f52131c2b80 10 client.15870 fetch_fsmap learned FSMap version 20 2023-10-26T06:50:48.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.687+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5637be2d5250 con 0x5637be536be0 2023-10-26T06:50:48.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1120709863 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f0017880 con 0x5637be536be0 2023-10-26T06:50:48.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1120709863 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f51ec044070 con 0x5637be536be0 2023-10-26T06:50:48.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f52131c2b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:50:48.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f52131c2b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:50:48.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f52131c2b80 20 client.15870 populate_metadata read hostname 'smithi089' 2023-10-26T06:50:48.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be59fd00 0x5637be5a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:50:48.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f52131c2b80 4 client.15870 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:50:48.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.688+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637be47ce50 con 0x5637be59fd00 2023-10-26T06:50:48.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.689+0000 7f51fd59b700 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be59fd00 0x5637be5a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:50:48.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.690+0000 7f51e0ff9700 10 client.15870 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.690+0000 7f51fd59b700 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be59fd00 0x5637be5a20f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fa28bd892a7e7c server_cookie=169f5f5717db6d01 in_seq=0 out_seq=0 2023-10-26T06:50:48.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.690+0000 7f51e0ff9700 1 -- 172.21.15.89:0/1120709863 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5637be47ce50 con 0x5637be59fd00 2023-10-26T06:50:48.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.690+0000 7f51e0ff9700 10 client.15870 handle_command_reply: tid=0 2023-10-26T06:50:48.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.691+0000 7f52131c2b80 1 client.15870 shutdown 2023-10-26T06:50:48.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.691+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be536be0 msgr2=0x5637be53ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.691+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be536be0 0x5637be53ff50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f51ec01ec00 tx=0x7f51ec02f710 comp rx=0 tx=0).stop 2023-10-26T06:50:48.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 shutdown_connections 2023-10-26T06:50:48.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5637be59fd00 0x5637be5a20f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637be537500 0x5637be540490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5637be5315d0 0x5637be541ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 --2- 172.21.15.89:0/1120709863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637be536be0 0x5637be53ff50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 >> 172.21.15.89:0/1120709863 conn(0x5637be4a5720 msgr2=0x5637be5346b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f51e0ff9700 0 client.15870 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:50:48.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 shutdown_connections 2023-10-26T06:50:48.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.692+0000 7f52131c2b80 1 -- 172.21.15.89:0/1120709863 wait complete. 2023-10-26T06:50:48.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.693+0000 7f52131c2b80 20 client.15870 trim_cache size 0 max 16384 2023-10-26T06:50:48.702 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:50:48.702 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:50:48.703 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:50:48.703 INFO:teuthology.orchestra.run.smithi089.stdout: "60c2401f-607c-4469-83c2-14a1662b767a": { 2023-10-26T06:50:48.703 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:50:48.703 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "60c2401f-607c-4469-83c2-14a1662b767a", 2023-10-26T06:50:48.703 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:50:48.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:50:48.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:50:48.704 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:50:48.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.693+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51e40a16f0 msgr2=0x7f51e40a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.693+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51e40a16f0 0x7f51e40a3bb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f51f81a6bb0 tx=0x7f51ec009260 comp rx=0 tx=0).stop 2023-10-26T06:50:48.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f810ef10 msgr2=0x7f51f81ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:50:48.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f810ef10 0x7f51f81ab8c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f51f4005910 tx=0x7f51f4004730 comp rx=0 tx=0).stop 2023-10-26T06:50:48.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 shutdown_connections 2023-10-26T06:50:48.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f51e40a16f0 0x7f51e40a3bb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51f8119fd0 0x7f51f81a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f810ef10 0x7f51f81ab8c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 --2- 172.21.15.89:0/555176138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f810f380 0x7f51f81abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:50:48.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 >> 172.21.15.89:0/555176138 conn(0x7f51f806b840 msgr2=0x7f51f8117fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:50:48.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 shutdown_connections 2023-10-26T06:50:48.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:50:48.694+0000 7f51ff7ff700 1 -- 172.21.15.89:0/555176138 wait complete. 2023-10-26T06:50:48.727 INFO:tasks.cephfs.filesystem:scrub status for tag:60c2401f-607c-4469-83c2-14a1662b767a - {'path': '/', 'tag': '60c2401f-607c-4469-83c2-14a1662b767a', 'options': 'recursive,force'} 2023-10-26T06:50:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:49 smithi089 ceph-mon[95031]: pgmap v427: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 44 KiB/s rd, 89 MiB/s wr, 14.26k op/s 2023-10-26T06:50:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:49 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:50:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:49 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:49 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:50:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:49 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:49.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:49 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:50:49.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:49 smithi192 ceph-mon[131661]: pgmap v427: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 44 KiB/s rd, 89 MiB/s wr, 14.26k op/s 2023-10-26T06:50:49.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:49 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:50:49.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:49 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:49.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:49 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:50:49.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:49 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:49.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:49 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:50:49.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:49 smithi135 ceph-mon[131403]: pgmap v427: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 44 KiB/s rd, 89 MiB/s wr, 14.26k op/s 2023-10-26T06:50:49.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:49 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:50:49.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:49 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:49.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:49 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:50:49.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:49 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:50:49.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:49 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:50:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:51 smithi089 ceph-mon[95031]: pgmap v428: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 993 GiB / 1.0 TiB avail; 44 KiB/s rd, 83 MiB/s wr, 13.55k op/s 2023-10-26T06:50:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:51 smithi192 ceph-mon[131661]: pgmap v428: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 993 GiB / 1.0 TiB avail; 44 KiB/s rd, 83 MiB/s wr, 13.55k op/s 2023-10-26T06:50:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:51 smithi135 ceph-mon[131403]: pgmap v428: 97 pgs: 97 active+clean; 22 GiB data, 79 GiB used, 993 GiB / 1.0 TiB avail; 44 KiB/s rd, 83 MiB/s wr, 13.55k op/s 2023-10-26T06:50:53.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:53 smithi089 ceph-mon[95031]: pgmap v429: 97 pgs: 97 active+clean; 22 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 72 KiB/s rd, 94 MiB/s wr, 15.70k op/s 2023-10-26T06:50:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:53 smithi192 ceph-mon[131661]: pgmap v429: 97 pgs: 97 active+clean; 22 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 72 KiB/s rd, 94 MiB/s wr, 15.70k op/s 2023-10-26T06:50:53.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:53 smithi135 ceph-mon[131403]: pgmap v429: 97 pgs: 97 active+clean; 22 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 72 KiB/s rd, 94 MiB/s wr, 15.70k op/s 2023-10-26T06:50:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:55 smithi089 ceph-mon[95031]: pgmap v430: 97 pgs: 97 active+clean; 23 GiB data, 80 GiB used, 992 GiB / 1.0 TiB avail; 50 KiB/s rd, 85 MiB/s wr, 14.79k op/s 2023-10-26T06:50:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:55 smithi192 ceph-mon[131661]: pgmap v430: 97 pgs: 97 active+clean; 23 GiB data, 80 GiB used, 992 GiB / 1.0 TiB avail; 50 KiB/s rd, 85 MiB/s wr, 14.79k op/s 2023-10-26T06:50:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:55 smithi135 ceph-mon[131403]: pgmap v430: 97 pgs: 97 active+clean; 23 GiB data, 80 GiB used, 992 GiB / 1.0 TiB avail; 50 KiB/s rd, 85 MiB/s wr, 14.79k op/s 2023-10-26T06:50:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:57 smithi089 ceph-mon[95031]: pgmap v431: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 82 KiB/s rd, 85 MiB/s wr, 15.26k op/s 2023-10-26T06:50:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:57 smithi192 ceph-mon[131661]: pgmap v431: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 82 KiB/s rd, 85 MiB/s wr, 15.26k op/s 2023-10-26T06:50:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:57 smithi135 ceph-mon[131403]: pgmap v431: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 82 KiB/s rd, 85 MiB/s wr, 15.26k op/s 2023-10-26T06:50:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:50:59 smithi192 ceph-mon[131661]: pgmap v432: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 62 KiB/s rd, 67 MiB/s wr, 12.04k op/s 2023-10-26T06:50:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:50:59 smithi135 ceph-mon[131403]: pgmap v432: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 62 KiB/s rd, 67 MiB/s wr, 12.04k op/s 2023-10-26T06:50:59.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:50:59 smithi089 ceph-mon[95031]: pgmap v432: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 62 KiB/s rd, 67 MiB/s wr, 12.04k op/s 2023-10-26T06:51:01.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:01 smithi192 ceph-mon[131661]: pgmap v433: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 62 KiB/s rd, 64 MiB/s wr, 11.73k op/s 2023-10-26T06:51:01.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:01 smithi135 ceph-mon[131403]: pgmap v433: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 62 KiB/s rd, 64 MiB/s wr, 11.73k op/s 2023-10-26T06:51:01.736 INFO:tasks.workunit.client.0.smithi089.stdout:Wrote -1 instead of 4096 bytes. 2023-10-26T06:51:01.737 INFO:tasks.workunit.client.0.smithi089.stdout:Probably out of disk space 2023-10-26T06:51:01.737 INFO:tasks.workunit.client.0.smithi089.stderr:write: Resource temporarily unavailable 2023-10-26T06:51:01.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T06:51:01.782 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2023-10-26T06:51:01.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-26T06:51:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:01 smithi089 ceph-mon[95031]: pgmap v433: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 62 KiB/s rd, 64 MiB/s wr, 11.73k op/s 2023-10-26T06:51:03.378 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=384993681abceb8cb8affea10538d607052db4b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2023-10-26T06:51:03.487 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bc459db9ebcf452688ad94ee3f52b7d9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=384993681abceb8cb8affea10538d607052db4b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2023-10-26T06:51:03.491 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-10-26T06:51:03.501 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-10-26T06:51:03.501 INFO:teuthology.task.exec:Executing custom commands... 2023-10-26T06:51:03.501 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T06:51:03.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-10-26T06:51:03.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:03 smithi192 ceph-mon[131661]: pgmap v434: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 86 KiB/s rd, 71 MiB/s wr, 13.24k op/s 2023-10-26T06:51:03.663 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:03 smithi089 ceph-mon[95031]: pgmap v434: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 86 KiB/s rd, 71 MiB/s wr, 13.24k op/s 2023-10-26T06:51:03.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-10-26T06:51:03.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:03 smithi135 ceph-mon[131403]: pgmap v434: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 86 KiB/s rd, 71 MiB/s wr, 13.24k op/s 2023-10-26T06:51:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:05 smithi192 ceph-mon[131661]: pgmap v435: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 64 KiB/s rd, 57 MiB/s wr, 10.93k op/s 2023-10-26T06:51:05.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:05 smithi135 ceph-mon[131403]: pgmap v435: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 64 KiB/s rd, 57 MiB/s wr, 10.93k op/s 2023-10-26T06:51:05.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:05 smithi089 ceph-mon[95031]: pgmap v435: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 64 KiB/s rd, 57 MiB/s wr, 10.93k op/s 2023-10-26T06:51:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:07 smithi192 ceph-mon[131661]: pgmap v436: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 991 GiB / 1.0 TiB avail; 64 KiB/s rd, 60 MiB/s wr, 11.77k op/s 2023-10-26T06:51:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:07 smithi135 ceph-mon[131403]: pgmap v436: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 991 GiB / 1.0 TiB avail; 64 KiB/s rd, 60 MiB/s wr, 11.77k op/s 2023-10-26T06:51:07.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:07 smithi089 ceph-mon[95031]: pgmap v436: 97 pgs: 97 active+clean; 23 GiB data, 81 GiB used, 991 GiB / 1.0 TiB avail; 64 KiB/s rd, 60 MiB/s wr, 11.77k op/s 2023-10-26T06:51:09.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:09 smithi135 ceph-mon[131403]: pgmap v437: 97 pgs: 97 active+clean; 23 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 39 KiB/s rd, 62 MiB/s wr, 12.10k op/s 2023-10-26T06:51:09.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:09 smithi089 ceph-mon[95031]: pgmap v437: 97 pgs: 97 active+clean; 23 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 39 KiB/s rd, 62 MiB/s wr, 12.10k op/s 2023-10-26T06:51:09.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:09 smithi192 ceph-mon[131661]: pgmap v437: 97 pgs: 97 active+clean; 23 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 39 KiB/s rd, 62 MiB/s wr, 12.10k op/s 2023-10-26T06:51:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:11 smithi135 ceph-mon[131403]: pgmap v438: 97 pgs: 97 active+clean; 23 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 39 KiB/s rd, 63 MiB/s wr, 12.15k op/s 2023-10-26T06:51:11.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:11 smithi089 ceph-mon[95031]: pgmap v438: 97 pgs: 97 active+clean; 23 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 39 KiB/s rd, 63 MiB/s wr, 12.15k op/s 2023-10-26T06:51:11.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:11 smithi192 ceph-mon[131661]: pgmap v438: 97 pgs: 97 active+clean; 23 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 39 KiB/s rd, 63 MiB/s wr, 12.15k op/s 2023-10-26T06:51:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:13 smithi135 ceph-mon[131403]: pgmap v439: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 100 KiB/s rd, 84 MiB/s wr, 15.63k op/s 2023-10-26T06:51:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:13 smithi089 ceph-mon[95031]: pgmap v439: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 100 KiB/s rd, 84 MiB/s wr, 15.63k op/s 2023-10-26T06:51:13.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:13 smithi192 ceph-mon[131661]: pgmap v439: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 100 KiB/s rd, 84 MiB/s wr, 15.63k op/s 2023-10-26T06:51:15.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:15 smithi089 ceph-mon[95031]: pgmap v440: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 76 KiB/s rd, 74 MiB/s wr, 13.56k op/s 2023-10-26T06:51:15.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:15 smithi192 ceph-mon[131661]: pgmap v440: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 76 KiB/s rd, 74 MiB/s wr, 13.56k op/s 2023-10-26T06:51:15.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:15 smithi135 ceph-mon[131403]: pgmap v440: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 76 KiB/s rd, 74 MiB/s wr, 13.56k op/s 2023-10-26T06:51:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:17 smithi089 ceph-mon[95031]: pgmap v441: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 70 KiB/s rd, 77 MiB/s wr, 14.26k op/s 2023-10-26T06:51:17.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:17 smithi192 ceph-mon[131661]: pgmap v441: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 70 KiB/s rd, 77 MiB/s wr, 14.26k op/s 2023-10-26T06:51:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:17 smithi135 ceph-mon[131403]: pgmap v441: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 70 KiB/s rd, 77 MiB/s wr, 14.26k op/s 2023-10-26T06:51:18.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:51:19.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 -- 172.21.15.89:0/2420256784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64841055a0 msgr2=0x7f6484105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 --2- 172.21.15.89:0/2420256784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64841055a0 0x7f6484105980 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f64800099c0 tx=0x7f6480009cd0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 -- 172.21.15.89:0/2420256784 shutdown_connections 2023-10-26T06:51:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 --2- 172.21.15.89:0/2420256784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64841055a0 0x7f6484105980 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 --2- 172.21.15.89:0/2420256784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64840ffea0 0x7f648410c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 --2- 172.21.15.89:0/2420256784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64840ff4e0 0x7f64840ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 -- 172.21.15.89:0/2420256784 >> 172.21.15.89:0/2420256784 conn(0x7f64840fafb0 msgr2=0x7f64840fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:19.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 -- 172.21.15.89:0/2420256784 shutdown_connections 2023-10-26T06:51:19.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.126+0000 7f648a59d700 1 -- 172.21.15.89:0/2420256784 wait complete. 2023-10-26T06:51:19.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.128+0000 7f648a59d700 1 Processor -- start 2023-10-26T06:51:19.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.130+0000 7f648a59d700 1 -- start start 2023-10-26T06:51:19.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648a59d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64840ff4e0 0x7f6484071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648a59d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64840ffea0 0x7f64840722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648a59d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64841055a0 0x7f648406ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648a59d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648406b650 con 0x7f64840ffea0 2023-10-26T06:51:19.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648a59d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648406b7c0 con 0x7f64840ff4e0 2023-10-26T06:51:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648959b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64840ff4e0 0x7f6484071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648959b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64840ff4e0 0x7f6484071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47318/0 (socket says 172.21.15.89:47318) 2023-10-26T06:51:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f648959b700 1 -- 172.21.15.89:0/1674602776 learned_addr learned my addr 172.21.15.89:0/1674602776 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:19.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.131+0000 7f6489d9c700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64841055a0 0x7f648406ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.132+0000 7f6488d9a700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64840ffea0 0x7f64840722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.132+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648406b980 con 0x7f64841055a0 2023-10-26T06:51:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.132+0000 7f6488d9a700 1 -- 172.21.15.89:0/1674602776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64841055a0 msgr2=0x7f648406ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.132+0000 7f6488d9a700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64841055a0 0x7f648406ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.132+0000 7f6488d9a700 1 -- 172.21.15.89:0/1674602776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64840ff4e0 msgr2=0x7f6484071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.132+0000 7f6488d9a700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64840ff4e0 0x7f6484071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.133+0000 7f6488d9a700 1 -- 172.21.15.89:0/1674602776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800095f0 con 0x7f64840ffea0 2023-10-26T06:51:19.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.133+0000 7f6488d9a700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64840ffea0 0x7f64840722f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f647400b450 tx=0x7f647400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.134+0000 7f647a7fc700 1 -- 172.21.15.89:0/1674602776 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474012b80 con 0x7f64840ffea0 2023-10-26T06:51:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.134+0000 7f647a7fc700 1 -- 172.21.15.89:0/1674602776 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f647400be50 con 0x7f64840ffea0 2023-10-26T06:51:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.134+0000 7f647a7fc700 1 -- 172.21.15.89:0/1674602776 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474019560 con 0x7f64840ffea0 2023-10-26T06:51:19.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.134+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648406bc60 con 0x7f64840ffea0 2023-10-26T06:51:19.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.134+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648406fbd0 con 0x7f64840ffea0 2023-10-26T06:51:19.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.140+0000 7f647a7fc700 1 -- 172.21.15.89:0/1674602776 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f647401b400 con 0x7f64840ffea0 2023-10-26T06:51:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.141+0000 7f649f038b80 1 Processor -- start 2023-10-26T06:51:19.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.141+0000 7f649f038b80 1 -- start start 2023-10-26T06:51:19.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.142+0000 7f649f038b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.142+0000 7f649f038b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.142+0000 7f649f038b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.142+0000 7f649f038b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec110426e0 con 0x55ec110d4d60 2023-10-26T06:51:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.142+0000 7f649f038b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11028c60 con 0x55ec110d5680 2023-10-26T06:51:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.142+0000 7f649f038b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11026580 con 0x55ec110cf820 2023-10-26T06:51:19.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47322/0 (socket says 172.21.15.89:47322) 2023-10-26T06:51:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 -- 172.21.15.89:0/3939310972 learned_addr learned my addr 172.21.15.89:0/3939310972 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:19.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 -- 172.21.15.89:0/3939310972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 msgr2=0x55ec110cfc80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:51:19.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 --2- 172.21.15.89:0/3939310972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110cfc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 -- 172.21.15.89:0/3939310972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 msgr2=0x55ec110d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:19.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 --2- 172.21.15.89:0/3939310972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.143+0000 7f648959b700 1 -- 172.21.15.89:0/3939310972 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec10fdf3f0 con 0x55ec110d5680 2023-10-26T06:51:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.144+0000 7f648959b700 1 --2- 172.21.15.89:0/3939310972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110cf2e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6480005f30 tx=0x7f6480005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1318079450d383aa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.144+0000 7f647a7fc700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64700a1670 0x7f64700a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.144+0000 7f647a7fc700 1 -- 172.21.15.89:0/1674602776 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f64740d0420 con 0x7f64840ffea0 2023-10-26T06:51:19.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.146+0000 7f646f7fe700 1 -- 172.21.15.89:0/3939310972 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800054a0 con 0x55ec110d5680 2023-10-26T06:51:19.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.146+0000 7f646f7fe700 1 -- 172.21.15.89:0/3939310972 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648001e040 con 0x55ec110d5680 2023-10-26T06:51:19.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.146+0000 7f646f7fe700 1 -- 172.21.15.89:0/3939310972 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480003ca0 con 0x55ec110d5680 2023-10-26T06:51:19.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.146+0000 7f6489d9c700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64700a1670 0x7f64700a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.147+0000 7f649f038b80 1 -- 172.21.15.89:0/3939310972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 msgr2=0x55ec110cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.147+0000 7f649f038b80 1 --2- 172.21.15.89:0/3939310972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110cf2e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6480005f30 tx=0x7f6480005050 comp rx=0 tx=0).stop 2023-10-26T06:51:19.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.147+0000 7f649f038b80 1 -- 172.21.15.89:0/3939310972 shutdown_connections 2023-10-26T06:51:19.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.147+0000 7f649f038b80 1 --2- 172.21.15.89:0/3939310972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.147+0000 7f649f038b80 1 --2- 172.21.15.89:0/3939310972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.147+0000 7f649f038b80 1 --2- 172.21.15.89:0/3939310972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110cf2e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.147+0000 7f649f038b80 1 -- 172.21.15.89:0/3939310972 >> 172.21.15.89:0/3939310972 conn(0x55ec11043720 msgr2=0x55ec110c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:19.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.148+0000 7f649f038b80 1 -- 172.21.15.89:0/3939310972 shutdown_connections 2023-10-26T06:51:19.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.148+0000 7f6489d9c700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64700a1670 0x7f64700a3b30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f647c00f520 tx=0x7f647c009920 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.148+0000 7f649f038b80 1 -- 172.21.15.89:0/3939310972 wait complete. 2023-10-26T06:51:19.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.149+0000 7f649f038b80 1 -- start start 2023-10-26T06:51:19.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 1 Processor -- start 2023-10-26T06:51:19.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110cf820 0x55ec110d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110d4d60 0x55ec110d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d5680 0x55ec110d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11021e70 con 0x55ec110d5680 2023-10-26T06:51:19.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec10fbe710 con 0x55ec110cf820 2023-10-26T06:51:19.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec10fbe3b0 con 0x55ec110d4d60 2023-10-26T06:51:19.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.151+0000 7f649f038b80 10 client.0 _ll_register_callbacks cb 0x55ec10f08a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:19.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.152+0000 7f6488d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110d4d60 0x55ec110d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.152+0000 7f6488d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110d4d60 0x55ec110d8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41298/0 (socket says 172.21.15.89:41298) 2023-10-26T06:51:19.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.152+0000 7f6488d9a700 1 -- 172.21.15.89:0/2008438937 learned_addr learned my addr 172.21.15.89:0/2008438937 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:19.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.153+0000 7f6488d9a700 1 -- 172.21.15.89:0/2008438937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110cf820 msgr2=0x55ec110d8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:19.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.153+0000 7f6488d9a700 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110cf820 0x55ec110d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.154+0000 7f6488d9a700 1 -- 172.21.15.89:0/2008438937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d5680 msgr2=0x55ec110d94d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:51:19.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.154+0000 7f6488d9a700 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d5680 0x55ec110d94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.154+0000 7f6488d9a700 1 -- 172.21.15.89:0/2008438937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648410cc60 con 0x55ec110d4d60 2023-10-26T06:51:19.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.154+0000 7f6488d9a700 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110d4d60 0x55ec110d8f90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64740ce1b0 tx=0x7f647409d090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df82578c4e830c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.154+0000 7f646cff9700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:51:19.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.157+0000 7f646cff9700 1 -- 172.21.15.89:0/2008438937 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647409d550 con 0x55ec110d4d60 2023-10-26T06:51:19.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.157+0000 7f646cff9700 1 -- 172.21.15.89:0/2008438937 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f64740a16a0 con 0x55ec110d4d60 2023-10-26T06:51:19.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.157+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec11026ef0 con 0x55ec110d4d60 2023-10-26T06:51:19.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.158+0000 7f646cff9700 1 -- 172.21.15.89:0/2008438937 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64740a58e0 con 0x55ec110d4d60 2023-10-26T06:51:19.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.158+0000 7f649f038b80 10 client.25918 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:19.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.158+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ec10e4cde0 con 0x55ec110d4d60 2023-10-26T06:51:19.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.158+0000 7f646cff9700 1 -- 172.21.15.89:0/2008438937 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec11026ef0 con 0x55ec110d4d60 2023-10-26T06:51:19.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.159+0000 7f646cff9700 1 -- 172.21.15.89:0/2008438937 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f647409cb80 con 0x55ec110d4d60 2023-10-26T06:51:19.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.159+0000 7f649f038b80 10 client.25918 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:19.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.159+0000 7f649f038b80 10 client.25918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:19.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.159+0000 7f649f038b80 20 client.25918 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:19.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.159+0000 7f649f038b80 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec110f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.159+0000 7f649f038b80 4 client.25918 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:19.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.159+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec1109e8b0 con 0x55ec110f1950 2023-10-26T06:51:19.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.162+0000 7f648959b700 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec110f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.168+0000 7f646cff9700 10 client.25918 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:19.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.168+0000 7f648959b700 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec110f3d70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e66f51d19326adf server_cookie=49cb1911c28a85d9 in_seq=0 out_seq=0 2023-10-26T06:51:19.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.169+0000 7f646cff9700 1 -- 172.21.15.89:0/2008438937 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec1109e8b0 con 0x55ec110f1950 2023-10-26T06:51:19.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.169+0000 7f646cff9700 10 client.25918 handle_command_reply: tid=0 2023-10-26T06:51:19.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.170+0000 7f649f038b80 1 client.25918 shutdown 2023-10-26T06:51:19.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.170+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110d4d60 msgr2=0x55ec110d8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.170+0000 7f649f038b80 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110d4d60 0x55ec110d8f90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64740ce1b0 tx=0x7f647409d090 comp rx=0 tx=0).stop 2023-10-26T06:51:19.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.171+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 shutdown_connections 2023-10-26T06:51:19.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.171+0000 7f649f038b80 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d5680 0x55ec110d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.171+0000 7f649f038b80 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec110f3d70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.171+0000 7f649f038b80 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110cf820 0x55ec110d8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.171+0000 7f646cff9700 0 client.25918 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:19.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.171+0000 7f649f038b80 1 --2- 172.21.15.89:0/2008438937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110d4d60 0x55ec110d8f90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.171+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 >> 172.21.15.89:0/2008438937 conn(0x55ec11043720 msgr2=0x55ec110ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:19.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.172+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 shutdown_connections 2023-10-26T06:51:19.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.173+0000 7f649f038b80 1 -- 172.21.15.89:0/2008438937 wait complete. 2023-10-26T06:51:19.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.173+0000 7f649f038b80 20 client.25918 trim_cache size 0 max 16384 2023-10-26T06:51:19.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.211+0000 7f649f038b80 1 Processor -- start 2023-10-26T06:51:19.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.211+0000 7f649f038b80 1 -- start start 2023-10-26T06:51:19.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.212+0000 7f649f038b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.212+0000 7f649f038b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.212+0000 7f649f038b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.212+0000 7f649f038b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec10ea2920 con 0x55ec110d4d60 2023-10-26T06:51:19.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.212+0000 7f649f038b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11026ef0 con 0x55ec110d5680 2023-10-26T06:51:19.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.212+0000 7f649f038b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec110d01e0 con 0x55ec110cf820 2023-10-26T06:51:19.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.213+0000 7f6488d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.213+0000 7f6488d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38550/0 (socket says 172.21.15.89:38550) 2023-10-26T06:51:19.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.213+0000 7f648959b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.213+0000 7f648959b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:41300/0 (socket says 172.21.15.89:41300) 2023-10-26T06:51:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.213+0000 7f648959b700 1 -- 172.21.15.89:0/3413907148 learned_addr learned my addr 172.21.15.89:0/3413907148 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.214+0000 7f648959b700 1 -- 172.21.15.89:0/3413907148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 msgr2=0x55ec110d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.214+0000 7f648959b700 1 --2- 172.21.15.89:0/3413907148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.214+0000 7f648959b700 1 -- 172.21.15.89:0/3413907148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 msgr2=0x55ec110e8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.214+0000 7f648959b700 1 --2- 172.21.15.89:0/3413907148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.214+0000 7f648959b700 1 -- 172.21.15.89:0/3413907148 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec10e35c60 con 0x55ec110cf820 2023-10-26T06:51:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.215+0000 7f648959b700 1 --2- 172.21.15.89:0/3413907148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110d4360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6480009990 tx=0x7f64800037e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f615c2bac4f3c99d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.217+0000 7f646ffff700 1 -- 172.21.15.89:0/3413907148 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480005520 con 0x55ec110cf820 2023-10-26T06:51:19.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.218+0000 7f646ffff700 1 -- 172.21.15.89:0/3413907148 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6480005680 con 0x55ec110cf820 2023-10-26T06:51:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.218+0000 7f646ffff700 1 -- 172.21.15.89:0/3413907148 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480005520 con 0x55ec110cf820 2023-10-26T06:51:19.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 -- 172.21.15.89:0/3413907148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 msgr2=0x55ec110d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 --2- 172.21.15.89:0/3413907148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110d4360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6480009990 tx=0x7f64800037e0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 -- 172.21.15.89:0/3413907148 shutdown_connections 2023-10-26T06:51:19.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 --2- 172.21.15.89:0/3413907148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec110d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 --2- 172.21.15.89:0/3413907148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110d4360 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 --2- 172.21.15.89:0/3413907148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 -- 172.21.15.89:0/3413907148 >> 172.21.15.89:0/3413907148 conn(0x55ec11043720 msgr2=0x55ec110d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:19.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 -- 172.21.15.89:0/3413907148 shutdown_connections 2023-10-26T06:51:19.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.219+0000 7f649f038b80 1 -- 172.21.15.89:0/3413907148 wait complete. 2023-10-26T06:51:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.220+0000 7f649f038b80 1 -- start start 2023-10-26T06:51:19.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.221+0000 7f649f038b80 1 Processor -- start 2023-10-26T06:51:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.222+0000 7f649f038b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.222+0000 7f649f038b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.222+0000 7f649f038b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec11144780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.222+0000 7f649f038b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11144cc0 con 0x55ec110d4d60 2023-10-26T06:51:19.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.222+0000 7f649f038b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11144e00 con 0x55ec110d5680 2023-10-26T06:51:19.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.222+0000 7f649f038b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11144f40 con 0x55ec110cf820 2023-10-26T06:51:19.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.222+0000 7f649f038b80 10 client.0 _ll_register_callbacks cb 0x55ec10f08a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.223+0000 7f6489d9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec11144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.223+0000 7f6489d9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec11144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47356/0 (socket says 172.21.15.89:47356) 2023-10-26T06:51:19.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.223+0000 7f6489d9c700 1 -- 172.21.15.89:0/2141591693 learned_addr learned my addr 172.21.15.89:0/2141591693 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.223+0000 7f648959b700 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.223+0000 7f6488d9a700 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.226+0000 7f6488d9a700 1 -- 172.21.15.89:0/2141591693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 msgr2=0x55ec110e66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:19.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.226+0000 7f6488d9a700 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110e66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.226+0000 7f6488d9a700 1 -- 172.21.15.89:0/2141591693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 msgr2=0x55ec11144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:19.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.226+0000 7f6488d9a700 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec11144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.226+0000 7f6488d9a700 1 -- 172.21.15.89:0/2141591693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec10fdf3f0 con 0x55ec110d4d60 2023-10-26T06:51:19.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.226+0000 7f646cff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:51:19.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.226+0000 7f6488d9a700 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e6c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6474009fd0 tx=0x7f6474004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1961ac89d186d994 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.228+0000 7f646cff9700 1 -- 172.21.15.89:0/2141591693 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64740a16a0 con 0x55ec110d4d60 2023-10-26T06:51:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.229+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f648001e040 con 0x55ec110d4d60 2023-10-26T06:51:19.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.229+0000 7f646cff9700 1 -- 172.21.15.89:0/2141591693 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f647409d550 con 0x55ec110d4d60 2023-10-26T06:51:19.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.230+0000 7f646cff9700 1 -- 172.21.15.89:0/2141591693 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64740a16a0 con 0x55ec110d4d60 2023-10-26T06:51:19.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.230+0000 7f649f038b80 10 client.15882 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:19.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.230+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55ec10e73250 con 0x55ec110d4d60 2023-10-26T06:51:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.231+0000 7f646cff9700 1 -- 172.21.15.89:0/2141591693 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648001e040 con 0x55ec110d4d60 2023-10-26T06:51:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.231+0000 7f646cff9700 1 -- 172.21.15.89:0/2141591693 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6474018b40 con 0x55ec110d4d60 2023-10-26T06:51:19.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.231+0000 7f649f038b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.231+0000 7f649f038b80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.231+0000 7f649f038b80 20 client.15882 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.231+0000 7f649f038b80 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec11148200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.231+0000 7f649f038b80 4 client.15882 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:51:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.232+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec1101ae50 con 0x55ec110f1950 2023-10-26T06:51:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.232+0000 7f648959b700 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec11148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.233+0000 7f648959b700 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec11148200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3d039a2c7e8d96 server_cookie=ca1bbb3e1cd839b9 in_seq=0 out_seq=0 2023-10-26T06:51:19.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.233+0000 7f646cff9700 10 client.15882 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:19.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.236+0000 7f646cff9700 1 -- 172.21.15.89:0/2141591693 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ec1101ae50 con 0x55ec110f1950 2023-10-26T06:51:19.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.236+0000 7f646cff9700 10 client.15882 handle_command_reply: tid=0 2023-10-26T06:51:19.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.236+0000 7f649f038b80 1 client.15882 shutdown 2023-10-26T06:51:19.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 msgr2=0x55ec110e6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e6c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6474009fd0 tx=0x7f6474004750 comp rx=0 tx=0).stop 2023-10-26T06:51:19.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 shutdown_connections 2023-10-26T06:51:19.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55ec110f1950 0x55ec11148200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec110d5680 0x55ec11144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ec110cf820 0x55ec110e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 --2- 172.21.15.89:0/2141591693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec110d4d60 0x55ec110e6c00 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6474009fd0 tx=0x7f6474004750 comp rx=0 tx=0).stop 2023-10-26T06:51:19.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 >> 172.21.15.89:0/2141591693 conn(0x55ec11043720 msgr2=0x55ec110d9330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:19.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.237+0000 7f646cff9700 0 client.15882 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:19.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.238+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 shutdown_connections 2023-10-26T06:51:19.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.238+0000 7f649f038b80 1 -- 172.21.15.89:0/2141591693 wait complete. 2023-10-26T06:51:19.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.238+0000 7f649f038b80 20 client.15882 trim_cache size 0 max 16384 2023-10-26T06:51:19.245 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:51:19.245 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:51:19.246 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:51:19.246 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:51:19.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64700a1670 msgr2=0x7f64700a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64700a1670 0x7f64700a3b30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f647c00f520 tx=0x7f647c009920 comp rx=0 tx=0).stop 2023-10-26T06:51:19.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64840ffea0 msgr2=0x7f64840722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64840ffea0 0x7f64840722f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f647400b450 tx=0x7f647400b760 comp rx=0 tx=0).stop 2023-10-26T06:51:19.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 shutdown_connections 2023-10-26T06:51:19.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64700a1670 0x7f64700a3b30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64841055a0 0x7f648406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64840ff4e0 0x7f6484071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 --2- 172.21.15.89:0/1674602776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64840ffea0 0x7f64840722f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.240+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 >> 172.21.15.89:0/1674602776 conn(0x7f64840fafb0 msgr2=0x7f64840fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:19.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.242+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 shutdown_connections 2023-10-26T06:51:19.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.242+0000 7f648a59d700 1 -- 172.21.15.89:0/1674602776 wait complete. 2023-10-26T06:51:19.280 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:51:19.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:51:19.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:19 smithi089 ceph-mon[95031]: pgmap v442: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 69 KiB/s rd, 79 MiB/s wr, 14.63k op/s 2023-10-26T06:51:19.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1683359780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 msgr2=0x7fdaa4107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1683359780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa4107bd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fda94009990 tx=0x7fda94009ca0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1683359780 shutdown_connections 2023-10-26T06:51:19.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1683359780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa4107bd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1683359780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4113940 0x7fdaa4115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1683359780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaa4108110 0x7fdaa4113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1683359780 >> 172.21.15.89:0/1683359780 conn(0x7fdaa406b7d0 msgr2=0x7fdaa406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:19.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.664+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1683359780 shutdown_connections 2023-10-26T06:51:19.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.665+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1683359780 wait complete. 2023-10-26T06:51:19.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.667+0000 7fdaa89ff700 1 Processor -- start 2023-10-26T06:51:19.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.667+0000 7fdaa89ff700 1 -- start start 2023-10-26T06:51:19.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa89ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa89ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4108110 0x7fdaa419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa89ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaa4113940 0x7fdaa41a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa89ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41a4370 con 0x7fdaa41077f0 2023-10-26T06:51:19.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa89ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41a44e0 con 0x7fdaa4108110 2023-10-26T06:51:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa89ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41a46a0 con 0x7fdaa4113940 2023-10-26T06:51:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa419f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38586/0 (socket says 172.21.15.89:38586) 2023-10-26T06:51:19.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.668+0000 7fdaa259c700 1 -- 172.21.15.89:0/1012945456 learned_addr learned my addr 172.21.15.89:0/1012945456 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.669+0000 7fdaa1d9b700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4108110 0x7fdaa419f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.669+0000 7fdaa2d9d700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaa4113940 0x7fdaa41a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.669+0000 7fdaa259c700 1 -- 172.21.15.89:0/1012945456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaa4113940 msgr2=0x7fdaa41a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.669+0000 7fdaa259c700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaa4113940 0x7fdaa41a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.669+0000 7fdaa259c700 1 -- 172.21.15.89:0/1012945456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4108110 msgr2=0x7fdaa419f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:19.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.669+0000 7fdaa259c700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4108110 0x7fdaa419f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:19.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.669+0000 7fdaa259c700 1 -- 172.21.15.89:0/1012945456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda940095f0 con 0x7fdaa41077f0 2023-10-26T06:51:19.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.670+0000 7fdaa259c700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa419f450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b5c0 tx=0x7fda94005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.671+0000 7fda937fe700 1 -- 172.21.15.89:0/1012945456 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9401d070 con 0x7fdaa41077f0 2023-10-26T06:51:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.672+0000 7fda937fe700 1 -- 172.21.15.89:0/1012945456 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda9400fbb0 con 0x7fdaa41077f0 2023-10-26T06:51:19.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.672+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa41a4920 con 0x7fdaa41077f0 2023-10-26T06:51:19.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.672+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa41093f0 con 0x7fdaa41077f0 2023-10-26T06:51:19.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.672+0000 7fda937fe700 1 -- 172.21.15.89:0/1012945456 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94022cb0 con 0x7fdaa41077f0 2023-10-26T06:51:19.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.673+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda840052e0 con 0x7fdaa41077f0 2023-10-26T06:51:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.678+0000 7fda937fe700 1 -- 172.21.15.89:0/1012945456 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fda94022470 con 0x7fdaa41077f0 2023-10-26T06:51:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.680+0000 7fda937fe700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fda8c0a1740 0x7fda8c0a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:19.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.680+0000 7fda937fe700 1 -- 172.21.15.89:0/1012945456 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fda940cea90 con 0x7fdaa41077f0 2023-10-26T06:51:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.681+0000 7fdaa1d9b700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fda8c0a1740 0x7fda8c0a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:19.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.682+0000 7fdaa1d9b700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fda8c0a1740 0x7fda8c0a3c00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda98006fd0 tx=0x7fda98008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:19.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:19.687+0000 7fda937fe700 1 -- 172.21.15.89:0/1012945456 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fda94097ea0 con 0x7fdaa41077f0 2023-10-26T06:51:19.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:19 smithi192 ceph-mon[131661]: pgmap v442: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 69 KiB/s rd, 79 MiB/s wr, 14.63k op/s 2023-10-26T06:51:19.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:19 smithi135 ceph-mon[131403]: pgmap v442: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 69 KiB/s rd, 79 MiB/s wr, 14.63k op/s 2023-10-26T06:51:20.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.177+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda84005860 con 0x7fdaa41077f0 2023-10-26T06:51:20.183 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:51:20.183 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:51:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.180+0000 7fda937fe700 1 -- 172.21.15.89:0/1012945456 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fda94027020 con 0x7fdaa41077f0 2023-10-26T06:51:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.185+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fda8c0a1740 msgr2=0x7fda8c0a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fda8c0a1740 0x7fda8c0a3c00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda98006fd0 tx=0x7fda98008040 comp rx=0 tx=0).stop 2023-10-26T06:51:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 msgr2=0x7fdaa419f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa419f450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b5c0 tx=0x7fda94005740 comp rx=0 tx=0).stop 2023-10-26T06:51:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 shutdown_connections 2023-10-26T06:51:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fda8c0a1740 0x7fda8c0a3c00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaa41077f0 0x7fdaa419f450 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaa4113940 0x7fdaa41a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 --2- 172.21.15.89:0/1012945456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaa4108110 0x7fdaa419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 >> 172.21.15.89:0/1012945456 conn(0x7fdaa406b7d0 msgr2=0x7fdaa406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 shutdown_connections 2023-10-26T06:51:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.186+0000 7fdaa89ff700 1 -- 172.21.15.89:0/1012945456 wait complete. 2023-10-26T06:51:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:51:20.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:51:20.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.594+0000 7f4dab59d700 1 -- 172.21.15.89:0/558889933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c005150 msgr2=0x7f4d9c0a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.594+0000 7f4dab59d700 1 --2- 172.21.15.89:0/558889933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c005150 0x7f4d9c0a2be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4da4066470 tx=0x7f4da4068f00 comp rx=0 tx=0).stop 2023-10-26T06:51:20.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.595+0000 7f4dab59d700 1 -- 172.21.15.89:0/558889933 shutdown_connections 2023-10-26T06:51:20.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.595+0000 7f4dab59d700 1 --2- 172.21.15.89:0/558889933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c005150 0x7f4d9c0a2be0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.595+0000 7f4dab59d700 1 --2- 172.21.15.89:0/558889933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c09be60 0x7f4d9c09c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.595+0000 7f4dab59d700 1 --2- 172.21.15.89:0/558889933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c004790 0x7f4d9c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.595+0000 7f4dab59d700 1 -- 172.21.15.89:0/558889933 >> 172.21.15.89:0/558889933 conn(0x7f4d9c00a790 msgr2=0x7f4d9c00aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:20.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.595+0000 7f4dab59d700 1 -- 172.21.15.89:0/558889933 shutdown_connections 2023-10-26T06:51:20.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.595+0000 7f4dab59d700 1 -- 172.21.15.89:0/558889933 wait complete. 2023-10-26T06:51:20.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.597+0000 7f4dab59d700 1 Processor -- start 2023-10-26T06:51:20.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.597+0000 7f4dab59d700 1 -- start start 2023-10-26T06:51:20.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.597+0000 7f4dab59d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c004790 0x7f4d9c132db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.597+0000 7f4dab59d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c005150 0x7f4d9c1332f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4dab59d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c09be60 0x7f4d9c0a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4dab59d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c0a12d0 con 0x7f4d9c005150 2023-10-26T06:51:20.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4dab59d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c0a1440 con 0x7f4d9c004790 2023-10-26T06:51:20.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daa59b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c004790 0x7f4d9c132db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4dab59d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c0a1600 con 0x7f4d9c09be60 2023-10-26T06:51:20.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daa59b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c004790 0x7f4d9c132db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47840/0 (socket says 172.21.15.89:47840) 2023-10-26T06:51:20.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daa59b700 1 -- 172.21.15.89:0/2449033757 learned_addr learned my addr 172.21.15.89:0/2449033757 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:20.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daad9c700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c09be60 0x7f4d9c0a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daad9c700 1 -- 172.21.15.89:0/2449033757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c004790 msgr2=0x7f4d9c132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daad9c700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c004790 0x7f4d9c132db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daad9c700 1 -- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c005150 msgr2=0x7f4d9c1332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daad9c700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c005150 0x7f4d9c1332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daad9c700 1 -- 172.21.15.89:0/2449033757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da404ed10 con 0x7f4d9c09be60 2023-10-26T06:51:20.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.598+0000 7f4daa59b700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c004790 0x7f4d9c132db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:51:20.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.599+0000 7f4daad9c700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c09be60 0x7f4d9c0a0b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c005710 tx=0x7f4da4073fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.599+0000 7f4d9b7fe700 1 -- 172.21.15.89:0/2449033757 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da4079040 con 0x7f4d9c09be60 2023-10-26T06:51:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.599+0000 7f4d9b7fe700 1 -- 172.21.15.89:0/2449033757 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4da40745e0 con 0x7f4d9c09be60 2023-10-26T06:51:20.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.599+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c0a1880 con 0x7f4d9c09be60 2023-10-26T06:51:20.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.599+0000 7f4d9b7fe700 1 -- 172.21.15.89:0/2449033757 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da407e680 con 0x7f4d9c09be60 2023-10-26T06:51:20.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.599+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c12ea30 con 0x7f4d9c09be60 2023-10-26T06:51:20.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.607+0000 7f4d9b7fe700 1 -- 172.21.15.89:0/2449033757 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4da4074750 con 0x7f4d9c09be60 2023-10-26T06:51:20.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.608+0000 7f4dc00cbb80 1 Processor -- start 2023-10-26T06:51:20.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.611+0000 7f4d9b7fe700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4d900a16f0 0x7f4d900a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.611+0000 7f4d9b7fe700 1 -- 172.21.15.89:0/2449033757 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4da4137680 con 0x7f4d9c09be60 2023-10-26T06:51:20.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.611+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4d900a16f0 0x7f4d900a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.612+0000 7f4dc00cbb80 1 -- start start 2023-10-26T06:51:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.612+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c63140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.612+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c5d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.612+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c5dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.612+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43bd06e0 con 0x563d43c62d60 2023-10-26T06:51:20.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.612+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43bb6c60 con 0x563d43c63680 2023-10-26T06:51:20.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.612+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43bb4580 con 0x563d43c5d820 2023-10-26T06:51:20.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.613+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c5d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.613+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c5d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47842/0 (socket says 172.21.15.89:47842) 2023-10-26T06:51:20.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.613+0000 7f4da9d9a700 1 -- 172.21.15.89:0/1103479812 learned_addr learned my addr 172.21.15.89:0/1103479812 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:20.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.613+0000 7f4daa59b700 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c63140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.613+0000 7f4daad9c700 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c5dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.614+0000 7f4daa59b700 1 -- 172.21.15.89:0/1103479812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 msgr2=0x563d43c5dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.614+0000 7f4daa59b700 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c5dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.614+0000 7f4daa59b700 1 -- 172.21.15.89:0/1103479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 msgr2=0x563d43c5d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.614+0000 7f4daa59b700 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c5d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.614+0000 7f4daa59b700 1 -- 172.21.15.89:0/1103479812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d43b6d3f0 con 0x563d43c62d60 2023-10-26T06:51:20.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.614+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c5d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:20.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.614+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4d900a16f0 0x7f4d900a3bb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005950 tx=0x7f4da00058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:20.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.615+0000 7f4daa59b700 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c63140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4d94005950 tx=0x7f4d94011820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20470eb1310f7692 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:20.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.616+0000 7f4d98ff9700 1 -- 172.21.15.89:0/1103479812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94004230 con 0x563d43c62d60 2023-10-26T06:51:20.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.617+0000 7f4d98ff9700 1 -- 172.21.15.89:0/1103479812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d94004cf0 con 0x563d43c62d60 2023-10-26T06:51:20.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.617+0000 7f4d98ff9700 1 -- 172.21.15.89:0/1103479812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94005670 con 0x563d43c62d60 2023-10-26T06:51:20.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.619+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/1103479812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 msgr2=0x563d43c63140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.619+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c63140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4d94005950 tx=0x7f4d94011820 comp rx=0 tx=0).stop 2023-10-26T06:51:20.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.619+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/1103479812 shutdown_connections 2023-10-26T06:51:20.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.619+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c63140 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.619+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c5dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.619+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/1103479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c5d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.619+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/1103479812 >> 172.21.15.89:0/1103479812 conn(0x563d43bd1720 msgr2=0x563d43c57cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:20.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.620+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/1103479812 shutdown_connections 2023-10-26T06:51:20.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.620+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/1103479812 wait complete. 2023-10-26T06:51:20.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.621+0000 7f4dc00cbb80 1 -- start start 2023-10-26T06:51:20.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.621+0000 7f4dc00cbb80 1 Processor -- start 2023-10-26T06:51:20.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.622+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c5d820 0x563d43c696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.622+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c62d60 0x563d43c66a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.622+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c63680 0x563d43c67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.622+0000 7f4daa59b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c5d820 0x563d43c696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.622+0000 7f4daa59b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c5d820 0x563d43c696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47856/0 (socket says 172.21.15.89:47856) 2023-10-26T06:51:20.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c62d60 0x563d43c66a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c62d60 0x563d43c66a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46486/0 (socket says 172.21.15.89:46486) 2023-10-26T06:51:20.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43bafe70 con 0x563d43c63680 2023-10-26T06:51:20.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43b4c710 con 0x563d43c5d820 2023-10-26T06:51:20.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43b4c3b0 con 0x563d43c62d60 2023-10-26T06:51:20.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4daa59b700 1 -- 172.21.15.89:0/2895544037 learned_addr learned my addr 172.21.15.89:0/2895544037 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:20.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4dc00cbb80 10 client.0 _ll_register_callbacks cb 0x563d43a96a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:20.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4daa59b700 1 -- 172.21.15.89:0/2895544037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c62d60 msgr2=0x563d43c66a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4daa59b700 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c62d60 0x563d43c66a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4daa59b700 1 -- 172.21.15.89:0/2895544037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c63680 msgr2=0x563d43c67120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4daa59b700 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c63680 0x563d43c67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.623+0000 7f4daa59b700 1 -- 172.21.15.89:0/2895544037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d94009d80 con 0x563d43c5d820 2023-10-26T06:51:20.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.625+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c62d60 0x563d43c66a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-26T06:51:20.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.625+0000 7f4daa59b700 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c5d820 0x563d43c696f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009ca0 tx=0x7f4d94005090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514454c475226842 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:20.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.625+0000 7f4d8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:51:20.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.625+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2895544037 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94005680 con 0x563d43c5d820 2023-10-26T06:51:20.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.625+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2895544037 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4d94011e40 con 0x563d43c5d820 2023-10-26T06:51:20.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.626+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d43bb4ef0 con 0x563d43c5d820 2023-10-26T06:51:20.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.626+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2895544037 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d9402e450 con 0x563d43c5d820 2023-10-26T06:51:20.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.626+0000 7f4dc00cbb80 10 client.25931 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:20.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.626+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563d439dade0 con 0x563d43c5d820 2023-10-26T06:51:20.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.626+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2895544037 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d43bb4ef0 con 0x563d43c5d820 2023-10-26T06:51:20.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.630+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2895544037 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d940195c0 con 0x563d43c5d820 2023-10-26T06:51:20.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.630+0000 7f4dc00cbb80 10 client.25931 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:20.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.630+0000 7f4dc00cbb80 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.630+0000 7f4dc00cbb80 20 client.25931 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.630+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43c770f0 0x563d43c79530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.630+0000 7f4dc00cbb80 4 client.25931 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:20.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.630+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d43c2c8b0 con 0x563d43c770f0 2023-10-26T06:51:20.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.631+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43c770f0 0x563d43c79530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.632+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43c770f0 0x563d43c79530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c3546ef4ae9fd1 server_cookie=88ee90cba9105553 in_seq=0 out_seq=0 2023-10-26T06:51:20.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.632+0000 7f4d8dffb700 10 client.25931 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.634+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2895544037 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d43c2c8b0 con 0x563d43c770f0 2023-10-26T06:51:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.634+0000 7f4d8dffb700 10 client.25931 handle_command_reply: tid=0 2023-10-26T06:51:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 client.25931 shutdown 2023-10-26T06:51:20.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c5d820 msgr2=0x563d43c696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c5d820 0x563d43c696f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009ca0 tx=0x7f4d94005090 comp rx=0 tx=0).stop 2023-10-26T06:51:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 shutdown_connections 2023-10-26T06:51:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c63680 0x563d43c67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43c770f0 0x563d43c79530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c5d820 0x563d43c696f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2895544037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c62d60 0x563d43c66a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4d8dffb700 0 client.25931 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:20.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.635+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 >> 172.21.15.89:0/2895544037 conn(0x563d43bd1720 msgr2=0x563d43c5c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:20.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.636+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 shutdown_connections 2023-10-26T06:51:20.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.637+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2895544037 wait complete. 2023-10-26T06:51:20.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.638+0000 7f4dc00cbb80 20 client.25931 trim_cache size 0 max 16384 2023-10-26T06:51:20.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.674+0000 7f4dc00cbb80 1 Processor -- start 2023-10-26T06:51:20.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.675+0000 7f4dc00cbb80 1 -- start start 2023-10-26T06:51:20.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.675+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.675+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.675+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c69220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.675+0000 7f4daa59b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.676+0000 7f4daa59b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c794e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46496/0 (socket says 172.21.15.89:46496) 2023-10-26T06:51:20.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.676+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.676+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51370/0 (socket says 172.21.15.89:51370) 2023-10-26T06:51:20.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4daa59b700 1 -- 172.21.15.89:0/3116512014 learned_addr learned my addr 172.21.15.89:0/3116512014 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:20.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43a30920 con 0x563d43c62d60 2023-10-26T06:51:20.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43c6e670 con 0x563d43c63680 2023-10-26T06:51:20.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43c65000 con 0x563d43c5d820 2023-10-26T06:51:20.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4daad9c700 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c69220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4daa59b700 1 -- 172.21.15.89:0/3116512014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 msgr2=0x563d43c69220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4daa59b700 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c69220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4daa59b700 1 -- 172.21.15.89:0/3116512014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 msgr2=0x563d43c6e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.677+0000 7f4daa59b700 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.678+0000 7f4daa59b700 1 -- 172.21.15.89:0/3116512014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d439c3c60 con 0x563d43c5d820 2023-10-26T06:51:20.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.678+0000 7f4daa59b700 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c794e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000c00 tx=0x7f4d94009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482eb004c193447 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:20.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.678+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:20.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.678+0000 7f4d997fa700 1 -- 172.21.15.89:0/3116512014 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d940045f0 con 0x563d43c5d820 2023-10-26T06:51:20.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.678+0000 7f4d997fa700 1 -- 172.21.15.89:0/3116512014 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d94019c40 con 0x563d43c5d820 2023-10-26T06:51:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.678+0000 7f4d997fa700 1 -- 172.21.15.89:0/3116512014 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9402ec50 con 0x563d43c5d820 2023-10-26T06:51:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.679+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 msgr2=0x563d43c794e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.679+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c794e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000c00 tx=0x7f4d94009310 comp rx=0 tx=0).stop 2023-10-26T06:51:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.680+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 shutdown_connections 2023-10-26T06:51:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.680+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43c69220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.680+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c794e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.680+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/3116512014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.680+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 >> 172.21.15.89:0/3116512014 conn(0x563d43bd1720 msgr2=0x563d43c597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.680+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 shutdown_connections 2023-10-26T06:51:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.681+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/3116512014 wait complete. 2023-10-26T06:51:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.681+0000 7f4dc00cbb80 1 -- start start 2023-10-26T06:51:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.682+0000 7f4dc00cbb80 1 Processor -- start 2023-10-26T06:51:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.684+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.684+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.684+0000 7f4dc00cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43cc9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.685+0000 7f4daa59b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.685+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43c6cf20 con 0x563d43c62d60 2023-10-26T06:51:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.685+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43cca4c0 con 0x563d43c63680 2023-10-26T06:51:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.685+0000 7f4dc00cbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d43cca600 con 0x563d43c5d820 2023-10-26T06:51:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.685+0000 7f4dc00cbb80 10 client.0 _ll_register_callbacks cb 0x563d43a96a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.685+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.685+0000 7f4da9d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51380/0 (socket says 172.21.15.89:51380) 2023-10-26T06:51:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.686+0000 7f4da9d9a700 1 -- 172.21.15.89:0/2424685285 learned_addr learned my addr 172.21.15.89:0/2424685285 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.686+0000 7f4da9d9a700 1 -- 172.21.15.89:0/2424685285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 msgr2=0x563d43c6c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.686+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c6c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.686+0000 7f4da9d9a700 1 -- 172.21.15.89:0/2424685285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 msgr2=0x563d43cc9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.686+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43cc9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.686+0000 7f4da9d9a700 1 -- 172.21.15.89:0/2424685285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d439dade0 con 0x563d43c62d60 2023-10-26T06:51:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.688+0000 7f4d8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:51:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.688+0000 7f4da9d9a700 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6c5e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4da0018000 tx=0x7f4da0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d2e1e76c907629 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:20.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.689+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2424685285 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0022070 con 0x563d43c62d60 2023-10-26T06:51:20.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.689+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d94004cf0 con 0x563d43c62d60 2023-10-26T06:51:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.689+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2424685285 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4da0024430 con 0x563d43c62d60 2023-10-26T06:51:20.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.689+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2424685285 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da0004830 con 0x563d43c62d60 2023-10-26T06:51:20.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.690+0000 7f4dc00cbb80 10 client.15900 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:20.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.690+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563d43a01250 con 0x563d43c62d60 2023-10-26T06:51:20.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2424685285 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d94004cf0 con 0x563d43c62d60 2023-10-26T06:51:20.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2424685285 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4da0005590 con 0x563d43c62d60 2023-10-26T06:51:20.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4dc00cbb80 10 client.15900 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:20.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4dc00cbb80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4dc00cbb80 20 client.15900 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43ccbea0 0x563d43cce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4dc00cbb80 4 client.15900 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:51:20.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.691+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d43ba8e50 con 0x563d43ccbea0 2023-10-26T06:51:20.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.692+0000 7f4daa59b700 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43ccbea0 0x563d43cce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:20.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.693+0000 7f4d8dffb700 10 client.15900 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:20.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.693+0000 7f4daa59b700 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43ccbea0 0x563d43cce2c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a1b2eae74ec488 server_cookie=26f43e0dbe5b83a8 in_seq=0 out_seq=0 2023-10-26T06:51:20.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.693+0000 7f4d8dffb700 1 -- 172.21.15.89:0/2424685285 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d43ba8e50 con 0x563d43ccbea0 2023-10-26T06:51:20.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.693+0000 7f4d8dffb700 10 client.15900 handle_command_reply: tid=0 2023-10-26T06:51:20.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.693+0000 7f4dc00cbb80 1 client.15900 shutdown 2023-10-26T06:51:20.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.693+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 msgr2=0x563d43c6c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.693+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6c5e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4da0018000 tx=0x7f4da0009260 comp rx=0 tx=0).stop 2023-10-26T06:51:20.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.694+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 shutdown_connections 2023-10-26T06:51:20.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.694+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563d43ccbea0 0x563d43cce2c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.694+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d43c63680 0x563d43cc9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.694+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563d43c5d820 0x563d43c6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.694+0000 7f4dc00cbb80 1 --2- 172.21.15.89:0/2424685285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d43c62d60 0x563d43c6c5e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.694+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 >> 172.21.15.89:0/2424685285 conn(0x563d43bd1720 msgr2=0x563d43c608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:20.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.694+0000 7f4d8dffb700 0 client.15900 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:20.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.695+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 shutdown_connections 2023-10-26T06:51:20.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.695+0000 7f4dc00cbb80 1 -- 172.21.15.89:0/2424685285 wait complete. 2023-10-26T06:51:20.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.695+0000 7f4dc00cbb80 20 client.15900 trim_cache size 0 max 16384 2023-10-26T06:51:20.720 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:51:20.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.696+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4d900a16f0 msgr2=0x7f4d900a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.696+0000 7f4dab59d700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4d900a16f0 0x7f4d900a3bb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005950 tx=0x7f4da00058e0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.696+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c09be60 msgr2=0x7f4d9c0a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:20.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.696+0000 7f4dab59d700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c09be60 0x7f4d9c0a0b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c005710 tx=0x7f4da4073fd0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.696+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 shutdown_connections 2023-10-26T06:51:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.697+0000 7f4dab59d700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4d900a16f0 0x7f4d900a3bb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.697+0000 7f4dab59d700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d9c09be60 0x7f4d9c0a0b50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.697+0000 7f4dab59d700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d9c004790 0x7f4d9c132db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.697+0000 7f4dab59d700 1 --2- 172.21.15.89:0/2449033757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d9c005150 0x7f4d9c1332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:20.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.697+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 >> 172.21.15.89:0/2449033757 conn(0x7f4d9c00a790 msgr2=0x7f4d9c095a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:20.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.697+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 shutdown_connections 2023-10-26T06:51:20.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:20.697+0000 7f4dab59d700 1 -- 172.21.15.89:0/2449033757 wait complete. 2023-10-26T06:51:20.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:20 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1012945456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:51:20.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:20 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1012945456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:51:20.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:20 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1012945456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:51:21.731 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:51:21.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:51:21.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:21 smithi089 ceph-mon[95031]: pgmap v443: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 62 KiB/s rd, 61 MiB/s wr, 11.52k op/s 2023-10-26T06:51:21.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:21 smithi192 ceph-mon[131661]: pgmap v443: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 62 KiB/s rd, 61 MiB/s wr, 11.52k op/s 2023-10-26T06:51:21.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:21 smithi135 ceph-mon[131403]: pgmap v443: 97 pgs: 97 active+clean; 24 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 62 KiB/s rd, 61 MiB/s wr, 11.52k op/s 2023-10-26T06:51:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 -- 172.21.15.89:0/3606245074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8067890 msgr2=0x7f48f8067d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 --2- 172.21.15.89:0/3606245074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8067890 0x7f48f8067d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f48f000b0a0 tx=0x7f48f000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 -- 172.21.15.89:0/3606245074 shutdown_connections 2023-10-26T06:51:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 --2- 172.21.15.89:0/3606245074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8067890 0x7f48f8067d10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 --2- 172.21.15.89:0/3606245074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f81056d0 0x7f48f8103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 --2- 172.21.15.89:0/3606245074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48f8103690 0x7f48f8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 -- 172.21.15.89:0/3606245074 >> 172.21.15.89:0/3606245074 conn(0x7f48f80767f0 msgr2=0x7f48f8076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 -- 172.21.15.89:0/3606245074 shutdown_connections 2023-10-26T06:51:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.096+0000 7f49005ff700 1 -- 172.21.15.89:0/3606245074 wait complete. 2023-10-26T06:51:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.098+0000 7f49005ff700 1 Processor -- start 2023-10-26T06:51:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.098+0000 7f49005ff700 1 -- start start 2023-10-26T06:51:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f49005ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f8067890 0x7f48f819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f49005ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48f8103690 0x7f48f819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f49005ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f81056d0 0x7f48f81a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f49005ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81a2200 con 0x7f48f8103690 2023-10-26T06:51:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f49005ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81a2370 con 0x7f48f8067890 2023-10-26T06:51:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f49005ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81a2530 con 0x7f48f81056d0 2023-10-26T06:51:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f48fe39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f8067890 0x7f48f819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f48fe39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f8067890 0x7f48f819d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47898/0 (socket says 172.21.15.89:47898) 2023-10-26T06:51:22.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f48fe39b700 1 -- 172.21.15.89:0/3789845187 learned_addr learned my addr 172.21.15.89:0/3789845187 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.099+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48f8103690 0x7f48f819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48feb9c700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f81056d0 0x7f48f81a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48fe39b700 1 -- 172.21.15.89:0/3789845187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f81056d0 msgr2=0x7f48f81a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48fe39b700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f81056d0 0x7f48f81a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48fe39b700 1 -- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48f8103690 msgr2=0x7f48f819d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48fe39b700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48f8103690 0x7f48f819d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48fe39b700 1 -- 172.21.15.89:0/3789845187 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0009d00 con 0x7f48f8067890 2023-10-26T06:51:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48feb9c700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f81056d0 0x7f48f81a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.100+0000 7f48fe39b700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f8067890 0x7f48f819d2e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48f400d950 tx=0x7f48f400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.101+0000 7f48ef7fe700 1 -- 172.21.15.89:0/3789845187 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4009b40 con 0x7f48f8067890 2023-10-26T06:51:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.101+0000 7f48ef7fe700 1 -- 172.21.15.89:0/3789845187 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f4009ca0 con 0x7f48f8067890 2023-10-26T06:51:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.101+0000 7f48ef7fe700 1 -- 172.21.15.89:0/3789845187 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f400f620 con 0x7f48f8067890 2023-10-26T06:51:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.103+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f81a2810 con 0x7f48f8067890 2023-10-26T06:51:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.103+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f81a2a00 con 0x7f48f8067890 2023-10-26T06:51:22.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.116+0000 7f48ef7fe700 1 -- 172.21.15.89:0/3789845187 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f48f4010460 con 0x7f48f8067890 2023-10-26T06:51:22.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.118+0000 7f48ef7fe700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f48e40a16f0 0x7f48e40a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.118+0000 7f48ef7fe700 1 -- 172.21.15.89:0/3789845187 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f48f40ccee0 con 0x7f48f8067890 2023-10-26T06:51:22.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.118+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f48e40a16f0 0x7f48e40a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.120+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f48e40a16f0 0x7f48e40a3bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48e8005fd0 tx=0x7f48e8005f60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 Processor -- start 2023-10-26T06:51:22.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 -- start start 2023-10-26T06:51:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3556040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f3550110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3550650 0x55d0f3550ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f34c3760 con 0x55d0f3555c60 2023-10-26T06:51:22.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f34a9ce0 con 0x55d0f3550650 2023-10-26T06:51:22.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.122+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f34a7600 con 0x55d0f3556580 2023-10-26T06:51:22.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48fdb9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f3550110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48feb9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3550650 0x55d0f3550ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48feb9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3550650 0x55d0f3550ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47910/0 (socket says 172.21.15.89:47910) 2023-10-26T06:51:22.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48fdb9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f3550110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46526/0 (socket says 172.21.15.89:46526) 2023-10-26T06:51:22.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48fdb9a700 1 -- 172.21.15.89:0/131460753 learned_addr learned my addr 172.21.15.89:0/131460753 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48fdb9a700 1 -- 172.21.15.89:0/131460753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3550650 msgr2=0x55d0f3550ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3550650 0x55d0f3550ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48fdb9a700 1 -- 172.21.15.89:0/131460753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 msgr2=0x55d0f3556040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.123+0000 7f48fe39b700 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3556040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.124+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3556040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.124+0000 7f48fdb9a700 1 -- 172.21.15.89:0/131460753 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f3460470 con 0x55d0f3556580 2023-10-26T06:51:22.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.124+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f3550110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48e800fc60 tx=0x7f48e8012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a6bc21ca7a2bda server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.125+0000 7f48ecbfe700 1 -- 172.21.15.89:0/131460753 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8011ad0 con 0x55d0f3556580 2023-10-26T06:51:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.125+0000 7f48ecbfe700 1 -- 172.21.15.89:0/131460753 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48e8011c30 con 0x55d0f3556580 2023-10-26T06:51:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.125+0000 7f48ecbfe700 1 -- 172.21.15.89:0/131460753 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8011ad0 con 0x55d0f3556580 2023-10-26T06:51:22.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.126+0000 7f4913f6fb80 1 -- 172.21.15.89:0/131460753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 msgr2=0x55d0f3550110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.126+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f3550110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48e800fc60 tx=0x7f48e8012000 comp rx=0 tx=0).stop 2023-10-26T06:51:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.126+0000 7f4913f6fb80 1 -- 172.21.15.89:0/131460753 shutdown_connections 2023-10-26T06:51:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.126+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3556040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.126+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3550650 0x55d0f3550ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.126+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/131460753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f3550110 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.126+0000 7f4913f6fb80 1 -- 172.21.15.89:0/131460753 >> 172.21.15.89:0/131460753 conn(0x55d0f34c47a0 msgr2=0x55d0f354abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.128+0000 7f4913f6fb80 1 -- 172.21.15.89:0/131460753 shutdown_connections 2023-10-26T06:51:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.128+0000 7f4913f6fb80 1 -- 172.21.15.89:0/131460753 wait complete. 2023-10-26T06:51:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.129+0000 7f4913f6fb80 1 -- start start 2023-10-26T06:51:22.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.130+0000 7f4913f6fb80 1 Processor -- start 2023-10-26T06:51:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3550650 0x55d0f3559840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3559d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f355a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f34a2ef0 con 0x55d0f3555c60 2023-10-26T06:51:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f343f790 con 0x55d0f3556580 2023-10-26T06:51:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f343f430 con 0x55d0f3550650 2023-10-26T06:51:22.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f4913f6fb80 10 client.0 _ll_register_callbacks cb 0x55d0f3389b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f48fe39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3550650 0x55d0f3559840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f48fe39b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3550650 0x55d0f3559840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46538/0 (socket says 172.21.15.89:46538) 2023-10-26T06:51:22.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.131+0000 7f48fe39b700 1 -- 172.21.15.89:0/2395208856 learned_addr learned my addr 172.21.15.89:0/2395208856 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.132+0000 7f48fe39b700 1 -- 172.21.15.89:0/2395208856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 msgr2=0x55d0f355a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.132+0000 7f48fe39b700 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f355a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.132+0000 7f48fe39b700 1 -- 172.21.15.89:0/2395208856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 msgr2=0x55d0f3559d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.132+0000 7f48fe39b700 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3559d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.132+0000 7f48fe39b700 1 -- 172.21.15.89:0/2395208856 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e800f940 con 0x55d0f3550650 2023-10-26T06:51:22.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.133+0000 7f48fe39b700 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3550650 0x55d0f3559840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48f4099ba0 tx=0x7f48f4099800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0aa6978a53edeeb server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.133+0000 7f48e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:51:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.133+0000 7f48e1ffb700 1 -- 172.21.15.89:0/2395208856 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40caaa0 con 0x55d0f3550650 2023-10-26T06:51:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.133+0000 7f48e1ffb700 1 -- 172.21.15.89:0/2395208856 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f48f409dff0 con 0x55d0f3550650 2023-10-26T06:51:22.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.133+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0f34a7f70 con 0x55d0f3550650 2023-10-26T06:51:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f48e1ffb700 1 -- 172.21.15.89:0/2395208856 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f40a2230 con 0x55d0f3550650 2023-10-26T06:51:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 10 client.25951 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d0f32cde60 con 0x55d0f3550650 2023-10-26T06:51:22.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f48e1ffb700 1 -- 172.21.15.89:0/2395208856 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0f34a7f70 con 0x55d0f3550650 2023-10-26T06:51:22.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f48e1ffb700 1 -- 172.21.15.89:0/2395208856 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f48f4099390 con 0x55d0f3550650 2023-10-26T06:51:22.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 10 client.25951 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:22.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:22.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 20 client.25951 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:22.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f357ae70 0x55d0f357d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 4 client.25951 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:22.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.134+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0f351f930 con 0x55d0f357ae70 2023-10-26T06:51:22.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.135+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f357ae70 0x55d0f357d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.136+0000 7f48e1ffb700 10 client.25951 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.136+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f357ae70 0x55d0f357d290 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6df89fd603db69d server_cookie=4886ec9e021690aa in_seq=0 out_seq=0 2023-10-26T06:51:22.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.136+0000 7f48e1ffb700 1 -- 172.21.15.89:0/2395208856 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d0f351f930 con 0x55d0f357ae70 2023-10-26T06:51:22.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.136+0000 7f48e1ffb700 10 client.25951 handle_command_reply: tid=0 2023-10-26T06:51:22.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.138+0000 7f4913f6fb80 1 client.25951 shutdown 2023-10-26T06:51:22.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.138+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3550650 msgr2=0x55d0f3559840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.138+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3550650 0x55d0f3559840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48f4099ba0 tx=0x7f48f4099800 comp rx=0 tx=0).stop 2023-10-26T06:51:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.139+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 shutdown_connections 2023-10-26T06:51:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.139+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f357ae70 0x55d0f357d290 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.139+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f355a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.139+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3550650 0x55d0f3559840 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48f4099ba0 tx=0x7f48f4099800 comp rx=0 tx=0).stop 2023-10-26T06:51:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.139+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/2395208856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3559d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.139+0000 7f48e1ffb700 0 client.25951 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.139+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 >> 172.21.15.89:0/2395208856 conn(0x55d0f34c47a0 msgr2=0x55d0f354f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.140+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 shutdown_connections 2023-10-26T06:51:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.140+0000 7f4913f6fb80 1 -- 172.21.15.89:0/2395208856 wait complete. 2023-10-26T06:51:22.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.140+0000 7f4913f6fb80 20 client.25951 trim_cache size 0 max 16384 2023-10-26T06:51:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.175+0000 7f4913f6fb80 1 Processor -- start 2023-10-26T06:51:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.175+0000 7f4913f6fb80 1 -- start start 2023-10-26T06:51:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.176+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3571f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.176+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f355a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.176+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3558840 0x55d0f35536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.176+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f33239a0 con 0x55d0f3555c60 2023-10-26T06:51:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.176+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f34a7f70 con 0x55d0f3558840 2023-10-26T06:51:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.176+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f35784d0 con 0x55d0f3556580 2023-10-26T06:51:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.177+0000 7f48feb9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3558840 0x55d0f35536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.177+0000 7f48feb9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3558840 0x55d0f35536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47924/0 (socket says 172.21.15.89:47924) 2023-10-26T06:51:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.177+0000 7f48feb9c700 1 -- 172.21.15.89:0/156851553 learned_addr learned my addr 172.21.15.89:0/156851553 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.177+0000 7f48fe39b700 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3571f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.179+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f355a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.179+0000 7f48feb9c700 1 -- 172.21.15.89:0/156851553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 msgr2=0x55d0f355a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.179+0000 7f48feb9c700 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f355a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.179+0000 7f48feb9c700 1 -- 172.21.15.89:0/156851553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 msgr2=0x55d0f3571f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.179+0000 7f48feb9c700 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3571f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.179+0000 7f48feb9c700 1 -- 172.21.15.89:0/156851553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f354adf0 con 0x55d0f3558840 2023-10-26T06:51:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.179+0000 7f48feb9c700 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3558840 0x55d0f35536c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48f000b0a0 tx=0x7f48f0007cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d61c3eea5e022 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.180+0000 7f48ed3ff700 1 -- 172.21.15.89:0/156851553 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000e040 con 0x55d0f3558840 2023-10-26T06:51:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.180+0000 7f48ed3ff700 1 -- 172.21.15.89:0/156851553 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f0020070 con 0x55d0f3558840 2023-10-26T06:51:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.180+0000 7f48ed3ff700 1 -- 172.21.15.89:0/156851553 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0005040 con 0x55d0f3558840 2023-10-26T06:51:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 -- 172.21.15.89:0/156851553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3558840 msgr2=0x55d0f35536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3558840 0x55d0f35536c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48f000b0a0 tx=0x7f48f0007cc0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 -- 172.21.15.89:0/156851553 shutdown_connections 2023-10-26T06:51:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f3571f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3558840 0x55d0f35536c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/156851553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3556580 0x55d0f355a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 -- 172.21.15.89:0/156851553 >> 172.21.15.89:0/156851553 conn(0x55d0f34c47a0 msgr2=0x55d0f3558130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.181+0000 7f4913f6fb80 1 -- 172.21.15.89:0/156851553 shutdown_connections 2023-10-26T06:51:22.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.182+0000 7f4913f6fb80 1 -- 172.21.15.89:0/156851553 wait complete. 2023-10-26T06:51:22.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.182+0000 7f4913f6fb80 1 -- start start 2023-10-26T06:51:22.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.182+0000 7f4913f6fb80 1 Processor -- start 2023-10-26T06:51:22.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.183+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f35c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.183+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f35c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.183+0000 7f4913f6fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3558840 0x55d0f35c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.183+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f35cfce0 con 0x55d0f3555c60 2023-10-26T06:51:22.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.183+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f35cfe20 con 0x55d0f3556580 2023-10-26T06:51:22.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.183+0000 7f4913f6fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f35cff90 con 0x55d0f3558840 2023-10-26T06:51:22.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.183+0000 7f4913f6fb80 10 client.0 _ll_register_callbacks cb 0x55d0f3389b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:22.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.184+0000 7f48feb9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3558840 0x55d0f35c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.184+0000 7f48fdb9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f35c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.184+0000 7f48fdb9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f35c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47934/0 (socket says 172.21.15.89:47934) 2023-10-26T06:51:22.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.184+0000 7f48fdb9a700 1 -- 172.21.15.89:0/250991704 learned_addr learned my addr 172.21.15.89:0/250991704 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.184+0000 7f48fe39b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f35c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.185+0000 7f48fdb9a700 1 -- 172.21.15.89:0/250991704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3558840 msgr2=0x55d0f35c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.185+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3558840 0x55d0f35c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.185+0000 7f48fdb9a700 1 -- 172.21.15.89:0/250991704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 msgr2=0x55d0f35c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.185+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f35c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.185+0000 7f48fdb9a700 1 -- 172.21.15.89:0/250991704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f8114580 con 0x55d0f3556580 2023-10-26T06:51:22.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.186+0000 7f48fdb9a700 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f35c51a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48e800fc60 tx=0x7f48e8011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208f31ffaffad70d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.186+0000 7f48e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:51:22.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.186+0000 7f48e1ffb700 1 -- 172.21.15.89:0/250991704 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e802d070 con 0x55d0f3556580 2023-10-26T06:51:22.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.186+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48e8011c30 con 0x55d0f3556580 2023-10-26T06:51:22.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.186+0000 7f48e1ffb700 1 -- 172.21.15.89:0/250991704 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f48e8005250 con 0x55d0f3556580 2023-10-26T06:51:22.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.186+0000 7f48e1ffb700 1 -- 172.21.15.89:0/250991704 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48e8031730 con 0x55d0f3556580 2023-10-26T06:51:22.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.186+0000 7f4913f6fb80 10 client.25949 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:22.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d0f32b6ce0 con 0x55d0f3556580 2023-10-26T06:51:22.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f48e1ffb700 1 -- 172.21.15.89:0/250991704 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48e803f020 con 0x55d0f3556580 2023-10-26T06:51:22.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f48e1ffb700 1 -- 172.21.15.89:0/250991704 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f48e801e5e0 con 0x55d0f3556580 2023-10-26T06:51:22.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f4913f6fb80 10 client.25949 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:22.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f4913f6fb80 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:22.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f4913f6fb80 20 client.25949 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:22.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f35d16f0 0x55d0f35d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f4913f6fb80 4 client.25949 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:51:22.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.187+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d0f349bed0 con 0x55d0f35d16f0 2023-10-26T06:51:22.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.188+0000 7f48fe39b700 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f35d16f0 0x55d0f35d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.190+0000 7f48fe39b700 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f35d16f0 0x55d0f35d3b10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22845f23b6f51d1 server_cookie=fe8757e7551eb6a3 in_seq=0 out_seq=0 2023-10-26T06:51:22.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.190+0000 7f48e1ffb700 10 client.25949 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.192+0000 7f48e1ffb700 1 -- 172.21.15.89:0/250991704 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d0f349bed0 con 0x55d0f35d16f0 2023-10-26T06:51:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.192+0000 7f48e1ffb700 10 client.25949 handle_command_reply: tid=0 2023-10-26T06:51:22.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.192+0000 7f4913f6fb80 1 client.25949 shutdown 2023-10-26T06:51:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.192+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 msgr2=0x55d0f35c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.192+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f35c51a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48e800fc60 tx=0x7f48e8011a80 comp rx=0 tx=0).stop 2023-10-26T06:51:22.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 shutdown_connections 2023-10-26T06:51:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d0f3555c60 0x55d0f35c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d0f35d16f0 0x55d0f35d3b10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d0f3558840 0x55d0f35c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 --2- 172.21.15.89:0/250991704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f3556580 0x55d0f35c51a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 >> 172.21.15.89:0/250991704 conn(0x55d0f34c47a0 msgr2=0x55d0f35536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f48e1ffb700 0 client.25949 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 shutdown_connections 2023-10-26T06:51:22.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.193+0000 7f4913f6fb80 1 -- 172.21.15.89:0/250991704 wait complete. 2023-10-26T06:51:22.206 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:51:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:51:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "daf4e3a6-69bf-436f-9994-c00f7f95edd8", 2023-10-26T06:51:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:51:22.207 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:51:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.194+0000 7f4913f6fb80 20 client.25949 trim_cache size 0 max 16384 2023-10-26T06:51:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f48e40a16f0 msgr2=0x7f48e40a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f48e40a16f0 0x7f48e40a3bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48e8005fd0 tx=0x7f48e8005f60 comp rx=0 tx=0).stop 2023-10-26T06:51:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f8067890 msgr2=0x7f48f819d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f8067890 0x7f48f819d2e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48f400d950 tx=0x7f48f400dd10 comp rx=0 tx=0).stop 2023-10-26T06:51:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 shutdown_connections 2023-10-26T06:51:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f48e40a16f0 0x7f48e40a3bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f81056d0 0x7f48f81a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48f8067890 0x7f48f819d2e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 --2- 172.21.15.89:0/3789845187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48f8103690 0x7f48f819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 >> 172.21.15.89:0/3789845187 conn(0x7f48f80767f0 msgr2=0x7f48f80fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 shutdown_connections 2023-10-26T06:51:22.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.195+0000 7f49005ff700 1 -- 172.21.15.89:0/3789845187 wait complete. 2023-10-26T06:51:22.233 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:51:22.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.589+0000 7fa6443ff700 1 -- 172.21.15.89:0/3248091645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c1010e0 msgr2=0x7fa63c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.589+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3248091645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c1010e0 0x7fa63c10d5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa634009f90 tx=0x7fa63400b330 comp rx=0 tx=0).stop 2023-10-26T06:51:22.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.590+0000 7fa6443ff700 1 -- 172.21.15.89:0/3248091645 shutdown_connections 2023-10-26T06:51:22.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.590+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3248091645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c1010e0 0x7fa63c10d5f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.590+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3248091645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa63c106730 0x7fa63c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.590+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3248091645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63c100720 0x7fa63c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.590+0000 7fa6443ff700 1 -- 172.21.15.89:0/3248091645 >> 172.21.15.89:0/3248091645 conn(0x7fa63c0fc1b0 msgr2=0x7fa63c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.590+0000 7fa6443ff700 1 -- 172.21.15.89:0/3248091645 shutdown_connections 2023-10-26T06:51:22.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.590+0000 7fa6443ff700 1 -- 172.21.15.89:0/3248091645 wait complete. 2023-10-26T06:51:22.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.592+0000 7fa6443ff700 1 Processor -- start 2023-10-26T06:51:22.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.592+0000 7fa6443ff700 1 -- start start 2023-10-26T06:51:22.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.592+0000 7fa6443ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c100720 0x7fa63c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa6443ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa63c1010e0 0x7fa63c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa6443ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63c106730 0x7fa63c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa6443ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c19cb60 con 0x7fa63c106730 2023-10-26T06:51:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa6443ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c19ccd0 con 0x7fa63c100720 2023-10-26T06:51:22.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa6443ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c19ce90 con 0x7fa63c1010e0 2023-10-26T06:51:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa64219b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c100720 0x7fa63c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa64219b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c100720 0x7fa63c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47950/0 (socket says 172.21.15.89:47950) 2023-10-26T06:51:22.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa64219b700 1 -- 172.21.15.89:0/3385915370 learned_addr learned my addr 172.21.15.89:0/3385915370 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa64199a700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa63c1010e0 0x7fa63c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.593+0000 7fa64299c700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63c106730 0x7fa63c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.594+0000 7fa64299c700 1 -- 172.21.15.89:0/3385915370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa63c1010e0 msgr2=0x7fa63c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.594+0000 7fa64299c700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa63c1010e0 0x7fa63c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.594+0000 7fa64299c700 1 -- 172.21.15.89:0/3385915370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c100720 msgr2=0x7fa63c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.594+0000 7fa64299c700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c100720 0x7fa63c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.594+0000 7fa64299c700 1 -- 172.21.15.89:0/3385915370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa634009c40 con 0x7fa63c106730 2023-10-26T06:51:22.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.594+0000 7fa64299c700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63c106730 0x7fa63c19c560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b6c0 tx=0x7fa6340094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.595+0000 7fa6337fe700 1 -- 172.21.15.89:0/3385915370 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400e040 con 0x7fa63c106730 2023-10-26T06:51:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.595+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c19d110 con 0x7fa63c106730 2023-10-26T06:51:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.595+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c1a5ad0 con 0x7fa63c106730 2023-10-26T06:51:22.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.596+0000 7fa6337fe700 1 -- 172.21.15.89:0/3385915370 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa634004e70 con 0x7fa63c106730 2023-10-26T06:51:22.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.596+0000 7fa6337fe700 1 -- 172.21.15.89:0/3385915370 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634005040 con 0x7fa63c106730 2023-10-26T06:51:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.602+0000 7fa657c3bb80 1 Processor -- start 2023-10-26T06:51:22.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.602+0000 7fa6337fe700 1 -- 172.21.15.89:0/3385915370 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa6340051a0 con 0x7fa63c106730 2023-10-26T06:51:22.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.605+0000 7fa6337fe700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6280a16f0 0x7fa6280a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.605+0000 7fa6337fe700 1 -- 172.21.15.89:0/3385915370 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa6340cdf30 con 0x7fa63c106730 2023-10-26T06:51:22.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.606+0000 7fa657c3bb80 1 -- start start 2023-10-26T06:51:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.606+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b17fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.606+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b18500 0x55e618b12090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.606+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b12a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.606+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618a856e0 con 0x55e618b18500 2023-10-26T06:51:22.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.606+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618a6bc60 con 0x55e618b125d0 2023-10-26T06:51:22.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.606+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618a69580 con 0x55e618b17be0 2023-10-26T06:51:22.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.607+0000 7fa64219b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b17fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.607+0000 7fa64219b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b17fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46572/0 (socket says 172.21.15.89:46572) 2023-10-26T06:51:22.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.607+0000 7fa64219b700 1 -- 172.21.15.89:0/1033698396 learned_addr learned my addr 172.21.15.89:0/1033698396 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.607+0000 7fa64299c700 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b12a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.607+0000 7fa64299c700 1 -- 172.21.15.89:0/1033698396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 msgr2=0x55e618b17fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.607+0000 7fa64299c700 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b17fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.607+0000 7fa64299c700 1 -- 172.21.15.89:0/1033698396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b18500 msgr2=0x55e618b12090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:51:22.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.608+0000 7fa64199a700 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b18500 0x55e618b12090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.608+0000 7fa64299c700 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b18500 0x55e618b12090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.608+0000 7fa64299c700 1 -- 172.21.15.89:0/1033698396 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e618a223f0 con 0x55e618b125d0 2023-10-26T06:51:22.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.608+0000 7fa64299c700 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b12a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6340cb660 tx=0x7fa634097340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392c8703cff6d11a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.609+0000 7fa630bfe700 1 -- 172.21.15.89:0/1033698396 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63409f0d0 con 0x55e618b125d0 2023-10-26T06:51:22.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.609+0000 7fa630bfe700 1 -- 172.21.15.89:0/1033698396 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6340a3280 con 0x55e618b125d0 2023-10-26T06:51:22.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.609+0000 7fa630bfe700 1 -- 172.21.15.89:0/1033698396 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634019320 con 0x55e618b125d0 2023-10-26T06:51:22.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.609+0000 7fa657c3bb80 1 -- 172.21.15.89:0/1033698396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 msgr2=0x55e618b12a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.609+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b12a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6340cb660 tx=0x7fa634097340 comp rx=0 tx=0).stop 2023-10-26T06:51:22.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa657c3bb80 1 -- 172.21.15.89:0/1033698396 shutdown_connections 2023-10-26T06:51:22.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b12a50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b17fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b18500 0x55e618b12090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa657c3bb80 1 -- 172.21.15.89:0/1033698396 >> 172.21.15.89:0/1033698396 conn(0x55e618a86720 msgr2=0x55e618b0cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa64199a700 1 --2- 172.21.15.89:0/1033698396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b18500 0x55e618b12090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:51:22.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa64199a700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6280a16f0 0x7fa6280a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.610+0000 7fa657c3bb80 1 -- 172.21.15.89:0/1033698396 shutdown_connections 2023-10-26T06:51:22.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.612+0000 7fa657c3bb80 1 -- 172.21.15.89:0/1033698396 wait complete. 2023-10-26T06:51:22.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.614+0000 7fa64199a700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6280a16f0 0x7fa6280a3bb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55e618b0e9f0 tx=0x7fa62c0073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.616+0000 7fa657c3bb80 1 -- start start 2023-10-26T06:51:22.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.618+0000 7fa657c3bb80 1 Processor -- start 2023-10-26T06:51:22.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.622+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b125d0 0x55e618b1b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.622+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 0x55e618b1beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.622+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.622+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618a64e70 con 0x55e618b17be0 2023-10-26T06:51:22.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618a01710 con 0x55e618b1c3f0 2023-10-26T06:51:22.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618a013b0 con 0x55e618b125d0 2023-10-26T06:51:22.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa657c3bb80 10 client.0 _ll_register_callbacks cb 0x55e61894ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:22.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64219b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b125d0 0x55e618b1b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64299c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64219b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b125d0 0x55e618b1b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46588/0 (socket says 172.21.15.89:46588) 2023-10-26T06:51:22.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64219b700 1 -- 172.21.15.89:0/2733385467 learned_addr learned my addr 172.21.15.89:0/2733385467 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64299c700 1 -- 172.21.15.89:0/2733385467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b125d0 msgr2=0x55e618b1b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64299c700 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b125d0 0x55e618b1b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64299c700 1 -- 172.21.15.89:0/2733385467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 msgr2=0x55e618b1beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.623+0000 7fa64299c700 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 0x55e618b1beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.624+0000 7fa64299c700 1 -- 172.21.15.89:0/2733385467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c1a5ad0 con 0x55e618b1c3f0 2023-10-26T06:51:22.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.624+0000 7fa64219b700 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b125d0 0x55e618b1b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:22.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.624+0000 7fa64299c700 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b27db0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa634006090 tx=0x7fa634009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8668b3a06e65b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.624+0000 7fa625ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:51:22.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.624+0000 7fa625ffb700 1 -- 172.21.15.89:0/2733385467 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6340cbaf0 con 0x55e618b1c3f0 2023-10-26T06:51:22.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.624+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e618a69ef0 con 0x55e618b1c3f0 2023-10-26T06:51:22.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.624+0000 7fa625ffb700 1 -- 172.21.15.89:0/2733385467 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa634079e40 con 0x55e618b1c3f0 2023-10-26T06:51:22.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa625ffb700 1 -- 172.21.15.89:0/2733385467 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa634019320 con 0x55e618b1c3f0 2023-10-26T06:51:22.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa657c3bb80 10 client.25961 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:22.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e61888fde0 con 0x55e618b1c3f0 2023-10-26T06:51:22.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa625ffb700 1 -- 172.21.15.89:0/2733385467 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e618a69ef0 con 0x55e618b1c3f0 2023-10-26T06:51:22.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa625ffb700 1 -- 172.21.15.89:0/2733385467 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa63409d030 con 0x55e618b1c3f0 2023-10-26T06:51:22.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa657c3bb80 10 client.25961 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:22.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa657c3bb80 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:22.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.625+0000 7fa657c3bb80 20 client.25961 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:22.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.626+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b2bd20 0x55e618b2e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.626+0000 7fa657c3bb80 4 client.25961 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:22.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.626+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e618ae18b0 con 0x55e618b2bd20 2023-10-26T06:51:22.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.626+0000 7fa64219b700 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b2bd20 0x55e618b2e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.626+0000 7fa64219b700 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b2bd20 0x55e618b2e140 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607c56b9045367b1 server_cookie=d08cd4c1d9982c7d in_seq=0 out_seq=0 2023-10-26T06:51:22.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.627+0000 7fa625ffb700 10 client.25961 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.627+0000 7fa625ffb700 1 -- 172.21.15.89:0/2733385467 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e618ae18b0 con 0x55e618b2bd20 2023-10-26T06:51:22.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.627+0000 7fa625ffb700 10 client.25961 handle_command_reply: tid=0 2023-10-26T06:51:22.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.627+0000 7fa657c3bb80 1 client.25961 shutdown 2023-10-26T06:51:22.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 msgr2=0x55e618b27db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b27db0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa634006090 tx=0x7fa634009920 comp rx=0 tx=0).stop 2023-10-26T06:51:22.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 shutdown_connections 2023-10-26T06:51:22.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b2bd20 0x55e618b2e140 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b27db0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b125d0 0x55e618b1b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2733385467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 0x55e618b1beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 >> 172.21.15.89:0/2733385467 conn(0x55e618a86720 msgr2=0x55e618b11080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa625ffb700 0 client.25961 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 shutdown_connections 2023-10-26T06:51:22.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.629+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2733385467 wait complete. 2023-10-26T06:51:22.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.630+0000 7fa657c3bb80 20 client.25961 trim_cache size 0 max 16384 2023-10-26T06:51:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.657+0000 7fa657c3bb80 1 Processor -- start 2023-10-26T06:51:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.657+0000 7fa657c3bb80 1 -- start start 2023-10-26T06:51:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b125d0 0x55e618b22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b1b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b1be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6188e5920 con 0x55e618b125d0 2023-10-26T06:51:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618b0e3b0 con 0x55e618b1c3f0 2023-10-26T06:51:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618b1cd60 con 0x55e618b17be0 2023-10-26T06:51:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa64219b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b125d0 0x55e618b22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa64219b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b125d0 0x55e618b22cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51474/0 (socket says 172.21.15.89:51474) 2023-10-26T06:51:22.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa64219b700 1 -- 172.21.15.89:0/352397396 learned_addr learned my addr 172.21.15.89:0/352397396 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa64299c700 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b1be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.658+0000 7fa64199a700 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b1b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.659+0000 7fa64299c700 1 -- 172.21.15.89:0/352397396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 msgr2=0x55e618b1b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.659+0000 7fa64299c700 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b1b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.659+0000 7fa64299c700 1 -- 172.21.15.89:0/352397396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b125d0 msgr2=0x55e618b22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.659+0000 7fa64299c700 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b125d0 0x55e618b22cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.659+0000 7fa64299c700 1 -- 172.21.15.89:0/352397396 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e618878c60 con 0x55e618b1c3f0 2023-10-26T06:51:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.659+0000 7fa64299c700 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b1be70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa63401daf0 tx=0x7fa63409b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=398c745c022574ec server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.659+0000 7fa6313ff700 1 -- 172.21.15.89:0/352397396 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634097300 con 0x55e618b1c3f0 2023-10-26T06:51:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.660+0000 7fa6313ff700 1 -- 172.21.15.89:0/352397396 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6340cbaf0 con 0x55e618b1c3f0 2023-10-26T06:51:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.660+0000 7fa6313ff700 1 -- 172.21.15.89:0/352397396 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400bbf0 con 0x55e618b1c3f0 2023-10-26T06:51:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.660+0000 7fa657c3bb80 1 -- 172.21.15.89:0/352397396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 msgr2=0x55e618b1be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.660+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b1be70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa63401daf0 tx=0x7fa63409b470 comp rx=0 tx=0).stop 2023-10-26T06:51:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.661+0000 7fa657c3bb80 1 -- 172.21.15.89:0/352397396 shutdown_connections 2023-10-26T06:51:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.661+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b125d0 0x55e618b22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.661+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b1c3f0 0x55e618b1be70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.661+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/352397396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b17be0 0x55e618b1b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.661+0000 7fa657c3bb80 1 -- 172.21.15.89:0/352397396 >> 172.21.15.89:0/352397396 conn(0x55e618a86720 msgr2=0x55e618b1aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.662+0000 7fa657c3bb80 1 -- 172.21.15.89:0/352397396 shutdown_connections 2023-10-26T06:51:22.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.662+0000 7fa657c3bb80 1 -- 172.21.15.89:0/352397396 wait complete. 2023-10-26T06:51:22.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.662+0000 7fa657c3bb80 1 -- start start 2023-10-26T06:51:22.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.662+0000 7fa657c3bb80 1 Processor -- start 2023-10-26T06:51:22.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.663+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b22e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.663+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 0x55e618b212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa657c3bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b1c3f0 0x55e618b21990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa64219b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b22e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618b220a0 con 0x55e618b17be0 2023-10-26T06:51:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618b7fb60 con 0x55e618b125d0 2023-10-26T06:51:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa657c3bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e618b7fca0 con 0x55e618b1c3f0 2023-10-26T06:51:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa64219b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b22e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:47990/0 (socket says 172.21.15.89:47990) 2023-10-26T06:51:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa64219b700 1 -- 172.21.15.89:0/2935172509 learned_addr learned my addr 172.21.15.89:0/2935172509 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa657c3bb80 10 client.0 _ll_register_callbacks cb 0x55e61894ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa64299c700 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b1c3f0 0x55e618b21990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.664+0000 7fa64199a700 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 0x55e618b212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.665+0000 7fa64219b700 1 -- 172.21.15.89:0/2935172509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b1c3f0 msgr2=0x55e618b21990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.665+0000 7fa64219b700 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b1c3f0 0x55e618b21990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.665+0000 7fa64219b700 1 -- 172.21.15.89:0/2935172509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 msgr2=0x55e618b212a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.665+0000 7fa64219b700 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 0x55e618b212a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.665+0000 7fa64219b700 1 -- 172.21.15.89:0/2935172509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e61888fde0 con 0x55e618b125d0 2023-10-26T06:51:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.667+0000 7fa64219b700 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b22e70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55e618b2ea50 tx=0x7fa638009a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7064f056a42b7809 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.667+0000 7fa625ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:51:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.667+0000 7fa625ffb700 1 -- 172.21.15.89:0/2935172509 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638018040 con 0x55e618b125d0 2023-10-26T06:51:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.667+0000 7fa625ffb700 1 -- 172.21.15.89:0/2935172509 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa63801f910 con 0x55e618b125d0 2023-10-26T06:51:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.668+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e618b13040 con 0x55e618b125d0 2023-10-26T06:51:22.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.668+0000 7fa625ffb700 1 -- 172.21.15.89:0/2935172509 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa63801fad0 con 0x55e618b125d0 2023-10-26T06:51:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.668+0000 7fa657c3bb80 10 client.25967 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.668+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e6188b6250 con 0x55e618b125d0 2023-10-26T06:51:22.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.668+0000 7fa625ffb700 1 -- 172.21.15.89:0/2935172509 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e618b13040 con 0x55e618b125d0 2023-10-26T06:51:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa625ffb700 1 -- 172.21.15.89:0/2935172509 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa6380134c0 con 0x55e618b125d0 2023-10-26T06:51:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa657c3bb80 10 client.25967 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa657c3bb80 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:22.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa657c3bb80 20 client.25967 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:22.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b89b40 0x55e618b8bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:22.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa657c3bb80 4 client.25967 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:51:22.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fa6340a3280 con 0x55e618b89b40 2023-10-26T06:51:22.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.669+0000 7fa64199a700 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b89b40 0x55e618b8bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:22.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.670+0000 7fa625ffb700 10 client.25967 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.670+0000 7fa64199a700 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b89b40 0x55e618b8bf60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e132e2b922bded9c server_cookie=fe2bf334f62c432c in_seq=0 out_seq=0 2023-10-26T06:51:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.670+0000 7fa625ffb700 1 -- 172.21.15.89:0/2935172509 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x7fa6340a3280 con 0x55e618b89b40 2023-10-26T06:51:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.670+0000 7fa625ffb700 10 client.25967 handle_command_reply: tid=0 2023-10-26T06:51:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.670+0000 7fa657c3bb80 1 client.25967 shutdown 2023-10-26T06:51:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.670+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 msgr2=0x55e618b22e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.670+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b22e70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55e618b2ea50 tx=0x7fa638009a80 comp rx=0 tx=0).stop 2023-10-26T06:51:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 shutdown_connections 2023-10-26T06:51:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e618b89b40 0x55e618b8bf60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e618b1c3f0 0x55e618b21990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e618b125d0 0x55e618b22e70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 --2- 172.21.15.89:0/2935172509 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e618b17be0 0x55e618b212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 >> 172.21.15.89:0/2935172509 conn(0x55e618a86720 msgr2=0x55e618b16aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa625ffb700 0 client.25967 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:22.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 shutdown_connections 2023-10-26T06:51:22.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 1 -- 172.21.15.89:0/2935172509 wait complete. 2023-10-26T06:51:22.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.671+0000 7fa657c3bb80 20 client.25967 trim_cache size 0 max 16384 2023-10-26T06:51:22.688 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:51:22.688 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "scrub active (171 inodes in the stack)", 2023-10-26T06:51:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:51:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "daf4e3a6-69bf-436f-9994-c00f7f95edd8": { 2023-10-26T06:51:22.689 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:51:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "daf4e3a6-69bf-436f-9994-c00f7f95edd8", 2023-10-26T06:51:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:51:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:51:22.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:51:22.691 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:51:22.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6280a16f0 msgr2=0x7fa6280a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6280a16f0 0x7fa6280a3bb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55e618b0e9f0 tx=0x7fa62c0073d0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63c106730 msgr2=0x7fa63c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:22.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63c106730 0x7fa63c19c560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b6c0 tx=0x7fa6340094b0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 shutdown_connections 2023-10-26T06:51:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa6280a16f0 0x7fa6280a3bb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa63c106730 0x7fa63c19c560 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63c100720 0x7fa63c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 --2- 172.21.15.89:0/3385915370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa63c1010e0 0x7fa63c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 >> 172.21.15.89:0/3385915370 conn(0x7fa63c0fc1b0 msgr2=0x7fa63c072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 shutdown_connections 2023-10-26T06:51:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:22.673+0000 7fa6443ff700 1 -- 172.21.15.89:0/3385915370 wait complete. 2023-10-26T06:51:22.702 INFO:tasks.cephfs.filesystem:scrub status for tag:daf4e3a6-69bf-436f-9994-c00f7f95edd8 - {'path': '/', 'tag': 'daf4e3a6-69bf-436f-9994-c00f7f95edd8', 'options': 'recursive,force'} 2023-10-26T06:51:23.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:23 smithi089 ceph-mon[95031]: pgmap v444: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 83 KiB/s rd, 83 MiB/s wr, 16.46k op/s 2023-10-26T06:51:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:23 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:51:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:23 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:23.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:23 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:51:23.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:23 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:23.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:23 smithi192 ceph-mon[131661]: pgmap v444: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 83 KiB/s rd, 83 MiB/s wr, 16.46k op/s 2023-10-26T06:51:23.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:23 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:51:23.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:23 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:23.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:23 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:51:23.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:23 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:23.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:23 smithi135 ceph-mon[131403]: pgmap v444: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 83 KiB/s rd, 83 MiB/s wr, 16.46k op/s 2023-10-26T06:51:23.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:23 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:51:23.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:23 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:23.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:23 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:51:23.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:23 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:24.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:24 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:51:24.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:24 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:51:24.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:24 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:51:24.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:24 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:51:25.060 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:24 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:51:25.060 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:24 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:51:25.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:25 smithi192 ceph-mon[131661]: pgmap v445: 97 pgs: 97 active+clean; 24 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 22 KiB/s rd, 63 MiB/s wr, 13.36k op/s 2023-10-26T06:51:25.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:25 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:51:25.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:25 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:51:25.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:25 smithi135 ceph-mon[131403]: pgmap v445: 97 pgs: 97 active+clean; 24 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 22 KiB/s rd, 63 MiB/s wr, 13.36k op/s 2023-10-26T06:51:25.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:25 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:51:25.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:25 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:51:26.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:25 smithi089 ceph-mon[95031]: pgmap v445: 97 pgs: 97 active+clean; 24 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 22 KiB/s rd, 63 MiB/s wr, 13.36k op/s 2023-10-26T06:51:26.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:25 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:51:26.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:25 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:51:26.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:51:26.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:51:27.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:51:27.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:27 smithi192 ceph-mon[131661]: pgmap v446: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 22 KiB/s rd, 61 MiB/s wr, 13.00k op/s 2023-10-26T06:51:27.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:27 smithi135 ceph-mon[131403]: pgmap v446: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 22 KiB/s rd, 61 MiB/s wr, 13.00k op/s 2023-10-26T06:51:28.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:27 smithi089 ceph-mon[95031]: pgmap v446: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 22 KiB/s rd, 61 MiB/s wr, 13.00k op/s 2023-10-26T06:51:29.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:29 smithi192 ceph-mon[131661]: pgmap v447: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 68 MiB/s wr, 14.52k op/s 2023-10-26T06:51:29.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:29 smithi135 ceph-mon[131403]: pgmap v447: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 68 MiB/s wr, 14.52k op/s 2023-10-26T06:51:30.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:29 smithi089 ceph-mon[95031]: pgmap v447: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 68 MiB/s wr, 14.52k op/s 2023-10-26T06:51:31.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:31 smithi192 ceph-mon[131661]: pgmap v448: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 52 MiB/s wr, 11.14k op/s 2023-10-26T06:51:31.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:31 smithi135 ceph-mon[131403]: pgmap v448: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 52 MiB/s wr, 11.14k op/s 2023-10-26T06:51:32.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:31 smithi089 ceph-mon[95031]: pgmap v448: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 52 MiB/s wr, 11.14k op/s 2023-10-26T06:51:33.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:33 smithi192 ceph-mon[131661]: pgmap v449: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 83 KiB/s rd, 50 MiB/s wr, 10.74k op/s 2023-10-26T06:51:33.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:33 smithi135 ceph-mon[131403]: pgmap v449: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 83 KiB/s rd, 50 MiB/s wr, 10.74k op/s 2023-10-26T06:51:34.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:33 smithi089 ceph-mon[95031]: pgmap v449: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 83 KiB/s rd, 50 MiB/s wr, 10.74k op/s 2023-10-26T06:51:35.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:35 smithi192 ceph-mon[131661]: pgmap v450: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 5.08k op/s 2023-10-26T06:51:35.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:35 smithi135 ceph-mon[131403]: pgmap v450: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 5.08k op/s 2023-10-26T06:51:36.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:35 smithi089 ceph-mon[95031]: pgmap v450: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 61 KiB/s rd, 24 MiB/s wr, 5.08k op/s 2023-10-26T06:51:37.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:37 smithi192 ceph-mon[131661]: pgmap v451: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 3.31k op/s 2023-10-26T06:51:37.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:37 smithi135 ceph-mon[131403]: pgmap v451: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 3.31k op/s 2023-10-26T06:51:38.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:37 smithi089 ceph-mon[95031]: pgmap v451: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 3.31k op/s 2023-10-26T06:51:39.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:39 smithi192 ceph-mon[131661]: pgmap v452: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 15 MiB/s wr, 3.20k op/s 2023-10-26T06:51:39.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:39 smithi135 ceph-mon[131403]: pgmap v452: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 15 MiB/s wr, 3.20k op/s 2023-10-26T06:51:40.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:39 smithi089 ceph-mon[95031]: pgmap v452: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 69 KiB/s rd, 15 MiB/s wr, 3.20k op/s 2023-10-26T06:51:41.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:41 smithi192 ceph-mon[131661]: pgmap v453: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 21 KiB/s rd, 1.3 MiB/s wr, 307 op/s 2023-10-26T06:51:41.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:41 smithi135 ceph-mon[131403]: pgmap v453: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 21 KiB/s rd, 1.3 MiB/s wr, 307 op/s 2023-10-26T06:51:42.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:41 smithi089 ceph-mon[95031]: pgmap v453: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 21 KiB/s rd, 1.3 MiB/s wr, 307 op/s 2023-10-26T06:51:43.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:43 smithi192 ceph-mon[131661]: pgmap v454: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 236 op/s 2023-10-26T06:51:43.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:43 smithi135 ceph-mon[131403]: pgmap v454: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 236 op/s 2023-10-26T06:51:44.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:43 smithi089 ceph-mon[95031]: pgmap v454: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 236 op/s 2023-10-26T06:51:45.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:45 smithi192 ceph-mon[131661]: pgmap v455: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 245 KiB/s wr, 59 op/s 2023-10-26T06:51:45.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:45 smithi135 ceph-mon[131403]: pgmap v455: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 245 KiB/s wr, 59 op/s 2023-10-26T06:51:46.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:45 smithi089 ceph-mon[95031]: pgmap v455: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 245 KiB/s wr, 59 op/s 2023-10-26T06:51:46.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:46 smithi192 ceph-mon[131661]: pgmap v456: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.5 KiB/s wr, 2 op/s 2023-10-26T06:51:46.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:46 smithi135 ceph-mon[131403]: pgmap v456: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.5 KiB/s wr, 2 op/s 2023-10-26T06:51:47.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:46 smithi089 ceph-mon[95031]: pgmap v456: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.5 KiB/s wr, 2 op/s 2023-10-26T06:51:49.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:48 smithi135 ceph-mon[131403]: pgmap v457: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 4 op/s 2023-10-26T06:51:49.289 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:48 smithi089 ceph-mon[95031]: pgmap v457: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 4 op/s 2023-10-26T06:51:49.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:48 smithi192 ceph-mon[131661]: pgmap v457: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 4 op/s 2023-10-26T06:51:51.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:51 smithi089 ceph-mon[95031]: pgmap v458: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:51 smithi192 ceph-mon[131661]: pgmap v458: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:51 smithi135 ceph-mon[131403]: pgmap v458: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:52.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:51:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 -- 172.21.15.89:0/2905599966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 msgr2=0x7f929410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 --2- 172.21.15.89:0/2905599966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 0x7f929410d5f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f928c00b0a0 tx=0x7f928c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 -- 172.21.15.89:0/2905599966 shutdown_connections 2023-10-26T06:51:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 --2- 172.21.15.89:0/2905599966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 0x7f929410d5f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 --2- 172.21.15.89:0/2905599966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9294106730 0x7f9294106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 --2- 172.21.15.89:0/2905599966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 0x7f9294100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 -- 172.21.15.89:0/2905599966 >> 172.21.15.89:0/2905599966 conn(0x7f92940fc1b0 msgr2=0x7f92940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 -- 172.21.15.89:0/2905599966 shutdown_connections 2023-10-26T06:51:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.913+0000 7f9299fff700 1 -- 172.21.15.89:0/2905599966 wait complete. 2023-10-26T06:51:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.914+0000 7f9299fff700 1 Processor -- start 2023-10-26T06:51:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.914+0000 7f9299fff700 1 -- start start 2023-10-26T06:51:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9299fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 0x7f9294198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9299fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 0x7f92941992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9299fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9294106730 0x7f929419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9299fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929419cb50 con 0x7f9294100720 2023-10-26T06:51:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9299fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929419ccc0 con 0x7f9294106730 2023-10-26T06:51:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9299fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929419ce80 con 0x7f92941010e0 2023-10-26T06:51:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 0x7f9294198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 0x7f9294198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49590/0 (socket says 172.21.15.89:49590) 2023-10-26T06:51:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 -- 172.21.15.89:0/4121459755 learned_addr learned my addr 172.21.15.89:0/4121459755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9292ffd700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 0x7f92941992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9293fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9294106730 0x7f929419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 -- 172.21.15.89:0/4121459755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 msgr2=0x7f92941992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 0x7f92941992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 -- 172.21.15.89:0/4121459755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9294106730 msgr2=0x7f929419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9294106730 0x7f929419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f92937fe700 1 -- 172.21.15.89:0/4121459755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c009d00 con 0x7f9294100720 2023-10-26T06:51:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.915+0000 7f9293fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9294106730 0x7f929419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.916+0000 7f92937fe700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 0x7f9294198d70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f928400e9a0 tx=0x7f928400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.916+0000 7f9290ff9700 1 -- 172.21.15.89:0/4121459755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284009790 con 0x7f9294100720 2023-10-26T06:51:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.916+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929419d160 con 0x7f9294100720 2023-10-26T06:51:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.916+0000 7f9290ff9700 1 -- 172.21.15.89:0/4121459755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f928400cbd0 con 0x7f9294100720 2023-10-26T06:51:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.916+0000 7f9290ff9700 1 -- 172.21.15.89:0/4121459755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284018680 con 0x7f9294100720 2023-10-26T06:51:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.916+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92941a5ac0 con 0x7f9294100720 2023-10-26T06:51:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.919+0000 7f9290ff9700 1 -- 172.21.15.89:0/4121459755 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f928400cd40 con 0x7f9294100720 2023-10-26T06:51:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.920+0000 7f92ad834b80 1 Processor -- start 2023-10-26T06:51:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.920+0000 7f9290ff9700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f927c0a15f0 0x7f927c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.920+0000 7f9290ff9700 1 -- 172.21.15.89:0/4121459755 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f9284014070 con 0x7f9294100720 2023-10-26T06:51:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.920+0000 7f92ad834b80 1 -- start start 2023-10-26T06:51:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f9293fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f927c0a15f0 0x7f927c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83218fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d83213090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83213a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92ad834b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d831866e0 con 0x559d83219500 2023-10-26T06:51:52.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92ad834b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8316cc60 con 0x559d832135d0 2023-10-26T06:51:52.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92ad834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8316a580 con 0x559d83218be0 2023-10-26T06:51:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92937fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83218fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f9292ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d83213090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92937fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83218fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38898/0 (socket says 172.21.15.89:38898) 2023-10-26T06:51:52.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92937fe700 1 -- 172.21.15.89:0/40592113 learned_addr learned my addr 172.21.15.89:0/40592113 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:52.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f9293fff700 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83213a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92937fe700 1 -- 172.21.15.89:0/40592113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 msgr2=0x559d83213a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.921+0000 7f92937fe700 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83213a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f92937fe700 1 -- 172.21.15.89:0/40592113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 msgr2=0x559d83213090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f92937fe700 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d83213090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f92937fe700 1 -- 172.21.15.89:0/40592113 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d831233f0 con 0x559d83218be0 2023-10-26T06:51:52.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f9293fff700 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83213a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:51:52.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f9293fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f927c0a15f0 0x7f927c0a3ab0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f928c009610 tx=0x7f928c006230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:52.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f92937fe700 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83218fc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9284096d70 tx=0x7f9284099c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab944989fa819d14 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:52.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f9279ffb700 1 -- 172.21.15.89:0/40592113 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928409a300 con 0x559d83218be0 2023-10-26T06:51:52.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f9279ffb700 1 -- 172.21.15.89:0/40592113 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92840b0070 con 0x559d83218be0 2023-10-26T06:51:52.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.922+0000 7f9279ffb700 1 -- 172.21.15.89:0/40592113 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92840997f0 con 0x559d83218be0 2023-10-26T06:51:52.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 -- 172.21.15.89:0/40592113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 msgr2=0x559d83218fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83218fc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9284096d70 tx=0x7f9284099c60 comp rx=0 tx=0).stop 2023-10-26T06:51:52.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 -- 172.21.15.89:0/40592113 shutdown_connections 2023-10-26T06:51:52.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83213a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83218fc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 --2- 172.21.15.89:0/40592113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d83213090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 -- 172.21.15.89:0/40592113 >> 172.21.15.89:0/40592113 conn(0x559d83187720 msgr2=0x559d8320db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:52.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 -- 172.21.15.89:0/40592113 shutdown_connections 2023-10-26T06:51:52.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 -- 172.21.15.89:0/40592113 wait complete. 2023-10-26T06:51:52.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.923+0000 7f92ad834b80 1 -- start start 2023-10-26T06:51:52.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 1 Processor -- start 2023-10-26T06:51:52.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d832135d0 0x559d8321c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d83218be0 0x559d8321cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d8321d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83165e70 con 0x559d83219500 2023-10-26T06:51:52.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83102710 con 0x559d83218be0 2023-10-26T06:51:52.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d831023b0 con 0x559d832135d0 2023-10-26T06:51:52.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92ad834b80 10 client.0 _ll_register_callbacks cb 0x559d8304ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:52.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f9292ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d83218be0 0x559d8321cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f9293fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d8321d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f9292ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d83218be0 0x559d8321cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58202/0 (socket says 172.21.15.89:58202) 2023-10-26T06:51:52.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f9292ffd700 1 -- 172.21.15.89:0/2337128501 learned_addr learned my addr 172.21.15.89:0/2337128501 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:52.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.924+0000 7f92937fe700 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d832135d0 0x559d8321c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9292ffd700 1 -- 172.21.15.89:0/2337128501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d832135d0 msgr2=0x559d8321c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9292ffd700 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d832135d0 0x559d8321c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9292ffd700 1 -- 172.21.15.89:0/2337128501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 msgr2=0x559d8321d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9292ffd700 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d8321d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9292ffd700 1 -- 172.21.15.89:0/2337128501 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92941a5ac0 con 0x559d83218be0 2023-10-26T06:51:52.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9292ffd700 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d83218be0 0x559d8321cd80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f92880116a0 tx=0x7f92880119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98cff07e942ea85b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:52.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9272ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:51:52.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9272ffd700 1 -- 172.21.15.89:0/2337128501 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288016070 con 0x559d83218be0 2023-10-26T06:51:52.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d8316aef0 con 0x559d83218be0 2023-10-26T06:51:52.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.925+0000 7f9272ffd700 1 -- 172.21.15.89:0/2337128501 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f928800f690 con 0x559d83218be0 2023-10-26T06:51:52.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f9272ffd700 1 -- 172.21.15.89:0/2337128501 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f928800f850 con 0x559d83218be0 2023-10-26T06:51:52.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 10 client.25976 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:52.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559d82f90de0 con 0x559d83218be0 2023-10-26T06:51:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f9272ffd700 1 -- 172.21.15.89:0/2337128501 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d8316aef0 con 0x559d83218be0 2023-10-26T06:51:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f9272ffd700 1 -- 172.21.15.89:0/2337128501 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9288023aa0 con 0x559d83218be0 2023-10-26T06:51:52.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 10 client.25976 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 20 client.25976 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d8322cf90 0x559d8322f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 4 client.25976 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.926+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d831e28b0 con 0x559d8322cf90 2023-10-26T06:51:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.927+0000 7f92937fe700 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d8322cf90 0x559d8322f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.927+0000 7f92937fe700 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d8322cf90 0x559d8322f3b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3962033dbdfe6815 server_cookie=498bab113aea77f3 in_seq=0 out_seq=0 2023-10-26T06:51:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.927+0000 7f9272ffd700 10 client.25976 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f9272ffd700 1 -- 172.21.15.89:0/2337128501 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d831e28b0 con 0x559d8322cf90 2023-10-26T06:51:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f9272ffd700 10 client.25976 handle_command_reply: tid=0 2023-10-26T06:51:52.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 client.25976 shutdown 2023-10-26T06:51:52.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d83218be0 msgr2=0x559d8321cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d83218be0 0x559d8321cd80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f92880116a0 tx=0x7f92880119b0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 shutdown_connections 2023-10-26T06:51:52.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d8321d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d8322cf90 0x559d8322f3b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d832135d0 0x559d8321c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f9272ffd700 0 client.25976 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:52.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2337128501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d83218be0 0x559d8321cd80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 >> 172.21.15.89:0/2337128501 conn(0x559d83187720 msgr2=0x559d83212080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:52.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 shutdown_connections 2023-10-26T06:51:52.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.928+0000 7f92ad834b80 1 -- 172.21.15.89:0/2337128501 wait complete. 2023-10-26T06:51:52.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.929+0000 7f92ad834b80 20 client.25976 trim_cache size 0 max 16384 2023-10-26T06:51:52.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 Processor -- start 2023-10-26T06:51:52.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 -- start start 2023-10-26T06:51:52.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d8321c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83207bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d832178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d82fe6920 con 0x559d83219500 2023-10-26T06:51:52.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8321c660 con 0x559d832135d0 2023-10-26T06:51:52.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f92ad834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8321de60 con 0x559d83218be0 2023-10-26T06:51:52.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.946+0000 7f9293fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d832178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d832178e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49616/0 (socket says 172.21.15.89:49616) 2023-10-26T06:51:52.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 -- 172.21.15.89:0/3693806700 learned_addr learned my addr 172.21.15.89:0/3693806700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:52.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f92937fe700 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d8321c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9292ffd700 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83207bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 -- 172.21.15.89:0/3693806700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 msgr2=0x559d83207bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83207bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 -- 172.21.15.89:0/3693806700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 msgr2=0x559d8321c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d8321c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 -- 172.21.15.89:0/3693806700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d82f79c60 con 0x559d83219500 2023-10-26T06:51:52.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f9293fff700 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d832178e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f928c00bbb0 tx=0x7f928c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c15cec7271eef2a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:52.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.947+0000 7f927a7fc700 1 -- 172.21.15.89:0/3693806700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c004d10 con 0x559d83219500 2023-10-26T06:51:52.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f927a7fc700 1 -- 172.21.15.89:0/3693806700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f928c004e70 con 0x559d83219500 2023-10-26T06:51:52.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f927a7fc700 1 -- 172.21.15.89:0/3693806700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c005900 con 0x559d83219500 2023-10-26T06:51:52.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 -- 172.21.15.89:0/3693806700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 msgr2=0x559d832178e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d832178e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f928c00bbb0 tx=0x7f928c020000 comp rx=0 tx=0).stop 2023-10-26T06:51:52.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 -- 172.21.15.89:0/3693806700 shutdown_connections 2023-10-26T06:51:52.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83219500 0x559d832178e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d8321c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 --2- 172.21.15.89:0/3693806700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83218be0 0x559d83207bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 -- 172.21.15.89:0/3693806700 >> 172.21.15.89:0/3693806700 conn(0x559d83187720 msgr2=0x559d832074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:52.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 -- 172.21.15.89:0/3693806700 shutdown_connections 2023-10-26T06:51:52.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.948+0000 7f92ad834b80 1 -- 172.21.15.89:0/3693806700 wait complete. 2023-10-26T06:51:52.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 -- start start 2023-10-26T06:51:52.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 Processor -- start 2023-10-26T06:51:52.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83222490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83218be0 0x559d832229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83219500 0x559d83222f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83224470 con 0x559d83218be0 2023-10-26T06:51:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83280d90 con 0x559d832135d0 2023-10-26T06:51:52.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d83280ed0 con 0x559d83219500 2023-10-26T06:51:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.949+0000 7f92ad834b80 10 client.0 _ll_register_callbacks cb 0x559d8304ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83222490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f9292ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83218be0 0x559d832229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83222490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58220/0 (socket says 172.21.15.89:58220) 2023-10-26T06:51:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 -- 172.21.15.89:0/2929577155 learned_addr learned my addr 172.21.15.89:0/2929577155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f9293fff700 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83219500 0x559d83222f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 -- 172.21.15.89:0/2929577155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83219500 msgr2=0x559d83222f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83219500 0x559d83222f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 -- 172.21.15.89:0/2929577155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83218be0 msgr2=0x559d832229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83218be0 0x559d832229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 -- 172.21.15.89:0/2929577155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929410def0 con 0x559d832135d0 2023-10-26T06:51:52.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f9292ffd700 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83218be0 0x559d832229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:51:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92937fe700 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83222490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9284098650 tx=0x7f928409a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90749e896f01147 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f9272ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:51:52.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f9272ffd700 1 -- 172.21.15.89:0/2929577155 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928409e450 con 0x559d832135d0 2023-10-26T06:51:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92840b0070 con 0x559d832135d0 2023-10-26T06:51:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.950+0000 7f9272ffd700 1 -- 172.21.15.89:0/2929577155 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f92840b7040 con 0x559d832135d0 2023-10-26T06:51:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f9272ffd700 1 -- 172.21.15.89:0/2929577155 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92840986a0 con 0x559d832135d0 2023-10-26T06:51:52.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 10 client.25982 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:52.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559d82fb7250 con 0x559d832135d0 2023-10-26T06:51:52.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f9272ffd700 1 -- 172.21.15.89:0/2929577155 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92840a6020 con 0x559d832135d0 2023-10-26T06:51:52.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f9272ffd700 1 -- 172.21.15.89:0/2929577155 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f928407c030 con 0x559d832135d0 2023-10-26T06:51:52.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 10 client.25982 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:52.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:52.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 20 client.25982 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:52.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d832824d0 0x559d832848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:52.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 4 client.25982 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:51:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559d8315ee50 con 0x559d832824d0 2023-10-26T06:51:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.951+0000 7f9292ffd700 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d832824d0 0x559d832848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f9272ffd700 10 client.25982 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:52.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f9292ffd700 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d832824d0 0x559d832848f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0a912a2f57749f server_cookie=907c1063c04248a8 in_seq=0 out_seq=0 2023-10-26T06:51:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f9272ffd700 1 -- 172.21.15.89:0/2929577155 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559d8315ee50 con 0x559d832824d0 2023-10-26T06:51:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f9272ffd700 10 client.25982 handle_command_reply: tid=0 2023-10-26T06:51:52.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 client.25982 shutdown 2023-10-26T06:51:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 msgr2=0x559d83222490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83222490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9284098650 tx=0x7f928409a0d0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 shutdown_connections 2023-10-26T06:51:52.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x559d832824d0 0x559d832848f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d83219500 0x559d83222f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f9272ffd700 0 client.25982 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559d832135d0 0x559d83222490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 --2- 172.21.15.89:0/2929577155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559d83218be0 0x559d832229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.952+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 >> 172.21.15.89:0/2929577155 conn(0x559d83187720 msgr2=0x559d832198e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:52.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 shutdown_connections 2023-10-26T06:51:52.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f92ad834b80 1 -- 172.21.15.89:0/2929577155 wait complete. 2023-10-26T06:51:52.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f92ad834b80 20 client.25982 trim_cache size 0 max 16384 2023-10-26T06:51:52.987 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:51:52.987 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:51:52.988 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:51:52.988 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:51:52.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f927c0a15f0 msgr2=0x7f927c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f927c0a15f0 0x7f927c0a3ab0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f928c009610 tx=0x7f928c006230 comp rx=0 tx=0).stop 2023-10-26T06:51:52.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 msgr2=0x7f9294198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:52.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 0x7f9294198d70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f928400e9a0 tx=0x7f928400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 shutdown_connections 2023-10-26T06:51:52.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f927c0a15f0 0x7f927c0a3ab0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9294100720 0x7f9294198d70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9294106730 0x7f929419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 --2- 172.21.15.89:0/4121459755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92941010e0 0x7f92941992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.953+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 >> 172.21.15.89:0/4121459755 conn(0x7f92940fc1b0 msgr2=0x7f9294072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.954+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 shutdown_connections 2023-10-26T06:51:52.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:52.954+0000 7f9299fff700 1 -- 172.21.15.89:0/4121459755 wait complete. 2023-10-26T06:51:52.993 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:51:52.993 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:51:53.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 -- 172.21.15.89:0/1265523997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af41006a0 msgr2=0x7f4af4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 --2- 172.21.15.89:0/1265523997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af41006a0 0x7f4af4100b20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009990 tx=0x7f4adc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 -- 172.21.15.89:0/1265523997 shutdown_connections 2023-10-26T06:51:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 --2- 172.21.15.89:0/1265523997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af4101060 0x7f4af410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 --2- 172.21.15.89:0/1265523997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af41066b0 0x7f4af4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 --2- 172.21.15.89:0/1265523997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af41006a0 0x7f4af4100b20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 -- 172.21.15.89:0/1265523997 >> 172.21.15.89:0/1265523997 conn(0x7f4af40fc1d0 msgr2=0x7f4af40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 -- 172.21.15.89:0/1265523997 shutdown_connections 2023-10-26T06:51:53.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.191+0000 7f4af8bff700 1 -- 172.21.15.89:0/1265523997 wait complete. 2023-10-26T06:51:53.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.192+0000 7f4af8bff700 1 Processor -- start 2023-10-26T06:51:53.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.192+0000 7f4af8bff700 1 -- start start 2023-10-26T06:51:53.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.192+0000 7f4af8bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af41006a0 0x7f4af4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.192+0000 7f4af8bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af4101060 0x7f4af4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.192+0000 7f4af8bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af41066b0 0x7f4af4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.192+0000 7f4af8bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af41a4ec0 con 0x7f4af4101060 2023-10-26T06:51:53.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4af8bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af41a5030 con 0x7f4af41066b0 2023-10-26T06:51:53.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4af8bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af41a51f0 con 0x7f4af41006a0 2023-10-26T06:51:53.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af4101060 0x7f4af4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4af259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af41006a0 0x7f4af4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af4101060 0x7f4af4073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49648/0 (socket says 172.21.15.89:49648) 2023-10-26T06:51:53.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 -- 172.21.15.89:0/3358434440 learned_addr learned my addr 172.21.15.89:0/3358434440 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:53.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4af2d9d700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af41066b0 0x7f4af4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 -- 172.21.15.89:0/3358434440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af41006a0 msgr2=0x7f4af4072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af41006a0 0x7f4af4072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 -- 172.21.15.89:0/3358434440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af41066b0 msgr2=0x7f4af4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af41066b0 0x7f4af4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 -- 172.21.15.89:0/3358434440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc0095f0 con 0x7f4af4101060 2023-10-26T06:51:53.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4af259c700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af41006a0 0x7f4af4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:51:53.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.193+0000 7f4aebfff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af4101060 0x7f4af4073030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4adc005770 tx=0x7f4adc0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.194+0000 7f4aeb7fe700 1 -- 172.21.15.89:0/3358434440 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc01d070 con 0x7f4af4101060 2023-10-26T06:51:53.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.194+0000 7f4aeb7fe700 1 -- 172.21.15.89:0/3358434440 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4adc021410 con 0x7f4af4101060 2023-10-26T06:51:53.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.194+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af41a54d0 con 0x7f4af4101060 2023-10-26T06:51:53.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.194+0000 7f4aeb7fe700 1 -- 172.21.15.89:0/3358434440 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc017650 con 0x7f4af4101060 2023-10-26T06:51:53.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.194+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af41a59e0 con 0x7f4af4101060 2023-10-26T06:51:53.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.195+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af4108cc0 con 0x7f4af4101060 2023-10-26T06:51:53.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.197+0000 7f4aeb7fe700 1 -- 172.21.15.89:0/3358434440 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4adc0217f0 con 0x7f4af4101060 2023-10-26T06:51:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.198+0000 7f4aeb7fe700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.198+0000 7f4aeb7fe700 1 -- 172.21.15.89:0/3358434440 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4adc0ceaa0 con 0x7f4af4101060 2023-10-26T06:51:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.198+0000 7f4af259c700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.199+0000 7f4af259c700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009fd0 tx=0x7f4ae4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.200+0000 7f4aeb7fe700 1 -- 172.21.15.89:0/3358434440 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4adc097e30 con 0x7f4af4101060 2023-10-26T06:51:53.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.450+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4af4073fd0 con 0x7f4af4101060 2023-10-26T06:51:53.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.451+0000 7f4aeb7fe700 1 -- 172.21.15.89:0/3358434440 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f4adc09b450 con 0x7f4af4101060 2023-10-26T06:51:53.454 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:51:53.454 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:51:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ae00a15f0 msgr2=0x7f4ae00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009fd0 tx=0x7f4ae4009260 comp rx=0 tx=0).stop 2023-10-26T06:51:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af4101060 msgr2=0x7f4af4073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af4101060 0x7f4af4073030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4adc005770 tx=0x7f4adc0057a0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 shutdown_connections 2023-10-26T06:51:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ae00a15f0 0x7f4ae00a3ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4af41066b0 0x7f4af4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af41006a0 0x7f4af4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 --2- 172.21.15.89:0/3358434440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4af4101060 0x7f4af4073030 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 >> 172.21.15.89:0/3358434440 conn(0x7f4af40fc1d0 msgr2=0x7f4af40fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.453+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 shutdown_connections 2023-10-26T06:51:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.454+0000 7f4af8bff700 1 -- 172.21.15.89:0/3358434440 wait complete. 2023-10-26T06:51:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:51:53.472 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:53 smithi089 ceph-mon[95031]: pgmap v459: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:53.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:51:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:53 smithi192 ceph-mon[131661]: pgmap v459: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:53.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 -- 172.21.15.89:0/798281064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 msgr2=0x7ffabc106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 --2- 172.21.15.89:0/798281064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc106b10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009990 tx=0x7ffaac009ca0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 -- 172.21.15.89:0/798281064 shutdown_connections 2023-10-26T06:51:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 --2- 172.21.15.89:0/798281064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc106b10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 --2- 172.21.15.89:0/798281064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffabc1010e0 0x7ffabc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 --2- 172.21.15.89:0/798281064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffabc100720 0x7ffabc100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 -- 172.21.15.89:0/798281064 >> 172.21.15.89:0/798281064 conn(0x7ffabc0fc1b0 msgr2=0x7ffabc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 -- 172.21.15.89:0/798281064 shutdown_connections 2023-10-26T06:51:53.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.674+0000 7ffac15ff700 1 -- 172.21.15.89:0/798281064 wait complete. 2023-10-26T06:51:53.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.675+0000 7ffac15ff700 1 Processor -- start 2023-10-26T06:51:53.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.675+0000 7ffac15ff700 1 -- start start 2023-10-26T06:51:53.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffac15ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffabc100720 0x7ffabc19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffac15ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffabc1010e0 0x7ffabc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffac15ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffac15ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc199200 con 0x7ffabc106730 2023-10-26T06:51:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffac15ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc199370 con 0x7ffabc100720 2023-10-26T06:51:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffac15ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc199530 con 0x7ffabc1010e0 2023-10-26T06:51:53.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffabb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffabb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49670/0 (socket says 172.21.15.89:49670) 2023-10-26T06:51:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffabb7fe700 1 -- 172.21.15.89:0/2264958087 learned_addr learned my addr 172.21.15.89:0/2264958087 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffabaffd700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffabc100720 0x7ffabc19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffabc1010e0 0x7ffabc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffabb7fe700 1 -- 172.21.15.89:0/2264958087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffabc1010e0 msgr2=0x7ffabc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffabb7fe700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffabc1010e0 0x7ffabc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.676+0000 7ffabb7fe700 1 -- 172.21.15.89:0/2264958087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffabc100720 msgr2=0x7ffabc19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffabb7fe700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffabc100720 0x7ffabc19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffabb7fe700 1 -- 172.21.15.89:0/2264958087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0095f0 con 0x7ffabc106730 2023-10-26T06:51:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffabb7fe700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc198ba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffab400ef30 tx=0x7ffab400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffaa3fff700 1 -- 172.21.15.89:0/2264958087 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4018070 con 0x7ffabc106730 2023-10-26T06:51:53.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffaa3fff700 1 -- 172.21.15.89:0/2264958087 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab400f040 con 0x7ffabc106730 2023-10-26T06:51:53.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc1997b0 con 0x7ffabc106730 2023-10-26T06:51:53.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffaa3fff700 1 -- 172.21.15.89:0/2264958087 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab400cd00 con 0x7ffabc106730 2023-10-26T06:51:53.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.677+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc1a5a60 con 0x7ffabc106730 2023-10-26T06:51:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.680+0000 7ffaa3fff700 1 -- 172.21.15.89:0/2264958087 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ffab4003a00 con 0x7ffabc106730 2023-10-26T06:51:53.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.682+0000 7ffad4f29b80 1 Processor -- start 2023-10-26T06:51:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.682+0000 7ffad4f29b80 1 -- start start 2023-10-26T06:51:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c21832fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c2182d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2182da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c217a06e0 con 0x555c21832be0 2023-10-26T06:51:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c21786c60 con 0x555c2182d5d0 2023-10-26T06:51:53.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c21784580 con 0x555c21833500 2023-10-26T06:51:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffaa3fff700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffaa3fff700 1 -- 172.21.15.89:0/2264958087 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ffab40ccbf0 con 0x7ffabc106730 2023-10-26T06:51:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c21832fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffabb7fe700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffabaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c2182d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffabaffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c2182d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38978/0 (socket says 172.21.15.89:38978) 2023-10-26T06:51:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffabaffd700 1 -- 172.21.15.89:0/2632037572 learned_addr learned my addr 172.21.15.89:0/2632037572 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:53.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.683+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2182da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffabaffd700 1 -- 172.21.15.89:0/2632037572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 msgr2=0x555c2182da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffabaffd700 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2182da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffabaffd700 1 -- 172.21.15.89:0/2632037572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 msgr2=0x555c21832fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffabaffd700 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c21832fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffabaffd700 1 -- 172.21.15.89:0/2632037572 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c2173d3f0 con 0x555c21833500 2023-10-26T06:51:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2182da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:51:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffabb7fe700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffab40a1a10 tx=0x7ffab4009ed0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffabaffd700 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c2182d090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b610 tx=0x7ffaac00b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785e56bb11a9cf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffaa17fa700 1 -- 172.21.15.89:0/2632037572 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01d070 con 0x555c21833500 2023-10-26T06:51:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffaa17fa700 1 -- 172.21.15.89:0/2632037572 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaac01e040 con 0x555c21833500 2023-10-26T06:51:53.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.684+0000 7ffaa17fa700 1 -- 172.21.15.89:0/2632037572 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac004840 con 0x555c21833500 2023-10-26T06:51:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2632037572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 msgr2=0x555c2182d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c2182d090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b610 tx=0x7ffaac00b640 comp rx=0 tx=0).stop 2023-10-26T06:51:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2632037572 shutdown_connections 2023-10-26T06:51:53.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c21832fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2182da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2632037572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c2182d090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2632037572 >> 172.21.15.89:0/2632037572 conn(0x555c217a1720 msgr2=0x555c21827b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2632037572 shutdown_connections 2023-10-26T06:51:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.685+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2632037572 wait complete. 2023-10-26T06:51:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 -- start start 2023-10-26T06:51:53.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 Processor -- start 2023-10-26T06:51:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c21839520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c218367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c21836ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2177fe70 con 0x555c21832be0 2023-10-26T06:51:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2171c710 con 0x555c2182d5d0 2023-10-26T06:51:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2171c3b0 con 0x555c21833500 2023-10-26T06:51:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.686+0000 7ffad4f29b80 10 client.0 _ll_register_callbacks cb 0x555c21666a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:53.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c218367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c21839520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c21839520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:58272/0 (socket says 172.21.15.89:58272) 2023-10-26T06:51:53.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c218367a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49686/0 (socket says 172.21.15.89:49686) 2023-10-26T06:51:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffaba7fc700 1 -- 172.21.15.89:0/1236199805 learned_addr learned my addr 172.21.15.89:0/1236199805 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c21836ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 -- 172.21.15.89:0/1236199805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 msgr2=0x555c21836ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c21836ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 -- 172.21.15.89:0/1236199805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 msgr2=0x555c21839520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c21839520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 -- 172.21.15.89:0/1236199805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc10def0 con 0x555c21832be0 2023-10-26T06:51:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffabaffd700 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c218367a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffaac000c00 tx=0x7ffaac00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f13d0907f769993 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.687+0000 7ffa9eff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:51:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffa9eff5700 1 -- 172.21.15.89:0/1236199805 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01d030 con 0x555c21832be0 2023-10-26T06:51:53.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c21784ef0 con 0x555c21832be0 2023-10-26T06:51:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffa9eff5700 1 -- 172.21.15.89:0/1236199805 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ffaac005980 con 0x555c21832be0 2023-10-26T06:51:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffa9eff5700 1 -- 172.21.15.89:0/1236199805 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaac004910 con 0x555c21832be0 2023-10-26T06:51:53.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 10 client.15957 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555c215aade0 con 0x555c21832be0 2023-10-26T06:51:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffa9eff5700 1 -- 172.21.15.89:0/1236199805 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c21784ef0 con 0x555c21832be0 2023-10-26T06:51:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffa9eff5700 1 -- 172.21.15.89:0/1236199805 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffaac00f8b0 con 0x555c21832be0 2023-10-26T06:51:53.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 10 client.15957 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:53.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:53.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 20 client.15957 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:53.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c218468e0 0x555c21848d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 4 client.15957 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.688+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c217fc8b0 con 0x555c218468e0 2023-10-26T06:51:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.689+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c218468e0 0x555c21848d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.689+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c218468e0 0x555c21848d00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f1e44237da45 server_cookie=8b58528079c1f51f in_seq=0 out_seq=0 2023-10-26T06:51:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.689+0000 7ffa9eff5700 10 client.15957 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffa9eff5700 1 -- 172.21.15.89:0/1236199805 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c217fc8b0 con 0x555c218468e0 2023-10-26T06:51:53.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffa9eff5700 10 client.15957 handle_command_reply: tid=0 2023-10-26T06:51:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 client.15957 shutdown 2023-10-26T06:51:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 msgr2=0x555c218367a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c218367a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffaac000c00 tx=0x7ffaac00b9e0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 shutdown_connections 2023-10-26T06:51:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c218468e0 0x555c21848d00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21833500 0x555c21836ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c21839520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/1236199805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c218367a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffa9eff5700 0 client.15957 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 >> 172.21.15.89:0/1236199805 conn(0x555c217a1720 msgr2=0x555c2182c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 shutdown_connections 2023-10-26T06:51:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.690+0000 7ffad4f29b80 1 -- 172.21.15.89:0/1236199805 wait complete. 2023-10-26T06:51:53.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.691+0000 7ffad4f29b80 20 client.15957 trim_cache size 0 max 16384 2023-10-26T06:51:53.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:53 smithi135 ceph-mon[131403]: pgmap v459: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 Processor -- start 2023-10-26T06:51:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 -- start start 2023-10-26T06:51:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2183deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21832be0 0x555c21839050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21833500 0x555c21839590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c21600920 con 0x555c21833500 2023-10-26T06:51:53.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c21839c90 con 0x555c2182d5d0 2023-10-26T06:51:53.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c217fc8b0 con 0x555c21832be0 2023-10-26T06:51:53.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21833500 0x555c21839590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21833500 0x555c21839590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49702/0 (socket says 172.21.15.89:49702) 2023-10-26T06:51:53.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffaba7fc700 1 -- 172.21.15.89:0/855576877 learned_addr learned my addr 172.21.15.89:0/855576877 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:53.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2183deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.704+0000 7ffabaffd700 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21832be0 0x555c21839050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaba7fc700 1 -- 172.21.15.89:0/855576877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21832be0 msgr2=0x555c21839050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21832be0 0x555c21839050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaba7fc700 1 -- 172.21.15.89:0/855576877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 msgr2=0x555c2183deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2183deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaba7fc700 1 -- 172.21.15.89:0/855576877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c21593c60 con 0x555c21833500 2023-10-26T06:51:53.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21833500 0x555c21839590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x555c218495a0 tx=0x7ffab0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ee562096f9ed41 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaa1ffb700 1 -- 172.21.15.89:0/855576877 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0027070 con 0x555c21833500 2023-10-26T06:51:53.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaa1ffb700 1 -- 172.21.15.89:0/855576877 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab0018040 con 0x555c21833500 2023-10-26T06:51:53.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.705+0000 7ffaa1ffb700 1 -- 172.21.15.89:0/855576877 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0003680 con 0x555c21833500 2023-10-26T06:51:53.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 -- 172.21.15.89:0/855576877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21833500 msgr2=0x555c21839590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21833500 0x555c21839590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x555c218495a0 tx=0x7ffab0004750 comp rx=0 tx=0).stop 2023-10-26T06:51:53.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 -- 172.21.15.89:0/855576877 shutdown_connections 2023-10-26T06:51:53.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21833500 0x555c21839590 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c2182d5d0 0x555c2183deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/855576877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c21832be0 0x555c21839050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 -- 172.21.15.89:0/855576877 >> 172.21.15.89:0/855576877 conn(0x555c217a1720 msgr2=0x555c2182d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 -- 172.21.15.89:0/855576877 shutdown_connections 2023-10-26T06:51:53.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 -- 172.21.15.89:0/855576877 wait complete. 2023-10-26T06:51:53.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 -- start start 2023-10-26T06:51:53.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.706+0000 7ffad4f29b80 1 Processor -- start 2023-10-26T06:51:53.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c2182d5d0 0x555c2183df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c2183c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffad4f29b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c21833500 0x555c2183c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2183d0b0 con 0x555c21832be0 2023-10-26T06:51:53.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2183d220 con 0x555c21833500 2023-10-26T06:51:53.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffad4f29b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2189a6e0 con 0x555c2182d5d0 2023-10-26T06:51:53.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffad4f29b80 10 client.0 _ll_register_callbacks cb 0x555c21666a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:53.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c2183c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c2183c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49716/0 (socket says 172.21.15.89:49716) 2023-10-26T06:51:53.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 -- 172.21.15.89:0/2688976146 learned_addr learned my addr 172.21.15.89:0/2688976146 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:53.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c2182d5d0 0x555c2183df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c21833500 0x555c2183c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 -- 172.21.15.89:0/2688976146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c2182d5d0 msgr2=0x555c2183df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c2182d5d0 0x555c2183df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 -- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c21833500 msgr2=0x555c2183c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:53.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c21833500 0x555c2183c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.707+0000 7ffabaffd700 1 -- 172.21.15.89:0/2688976146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab00223d0 con 0x555c21832be0 2023-10-26T06:51:53.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffabaffd700 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c2183c490 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x555c21837ce0 tx=0x7ffaac005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b27e74ee4e7ffff server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:53.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffa9eff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:51:53.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffa9eff5700 1 -- 172.21.15.89:0/2688976146 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01d040 con 0x555c21832be0 2023-10-26T06:51:53.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffaac01e040 con 0x555c21832be0 2023-10-26T06:51:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffa9eff5700 1 -- 172.21.15.89:0/2688976146 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ffaac00f660 con 0x555c21832be0 2023-10-26T06:51:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffa9eff5700 1 -- 172.21.15.89:0/2688976146 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaac00be90 con 0x555c21832be0 2023-10-26T06:51:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffad4f29b80 10 client.15963 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:53.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555c215d1250 con 0x555c21832be0 2023-10-26T06:51:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffa9eff5700 1 -- 172.21.15.89:0/2688976146 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaac01e040 con 0x555c21832be0 2023-10-26T06:51:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffa9eff5700 1 -- 172.21.15.89:0/2688976146 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffaac02c780 con 0x555c21832be0 2023-10-26T06:51:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffad4f29b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:53.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffad4f29b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.708+0000 7ffad4f29b80 20 client.15963 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.709+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c2189b990 0x555c2189ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.709+0000 7ffad4f29b80 4 client.15963 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:51:53.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.709+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c21778e50 con 0x555c2189b990 2023-10-26T06:51:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.709+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c2189b990 0x555c2189ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.709+0000 7ffaba7fc700 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c2189b990 0x555c2189ddb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ae89a828e871ef server_cookie=4b6d083018b641d7 in_seq=0 out_seq=0 2023-10-26T06:51:53.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.709+0000 7ffa9eff5700 10 client.15963 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffa9eff5700 1 -- 172.21.15.89:0/2688976146 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c21778e50 con 0x555c2189b990 2023-10-26T06:51:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffa9eff5700 10 client.15963 handle_command_reply: tid=0 2023-10-26T06:51:53.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 client.15963 shutdown 2023-10-26T06:51:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 msgr2=0x555c2183c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c2183c490 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x555c21837ce0 tx=0x7ffaac005af0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 shutdown_connections 2023-10-26T06:51:53.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555c2189b990 0x555c2189ddb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c21833500 0x555c2183c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c2182d5d0 0x555c2183df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 --2- 172.21.15.89:0/2688976146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555c21832be0 0x555c2183c490 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffa9eff5700 0 client.15963 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 >> 172.21.15.89:0/2688976146 conn(0x555c217a1720 msgr2=0x555c218306b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 shutdown_connections 2023-10-26T06:51:53.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 1 -- 172.21.15.89:0/2688976146 wait complete. 2023-10-26T06:51:53.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.710+0000 7ffad4f29b80 20 client.15963 trim_cache size 0 max 16384 2023-10-26T06:51:53.747 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:51:53.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffaa40a15f0 msgr2=0x7ffaa40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffab40a1a10 tx=0x7ffab4009ed0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 msgr2=0x7ffabc198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc198ba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffab400ef30 tx=0x7ffab400c3a0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 shutdown_connections 2023-10-26T06:51:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ffaa40a15f0 0x7ffaa40a3ab0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffabc106730 0x7ffabc198ba0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffabc100720 0x7ffabc19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 --2- 172.21.15.89:0/2264958087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffabc1010e0 0x7ffabc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 >> 172.21.15.89:0/2264958087 conn(0x7ffabc0fc1b0 msgr2=0x7ffabc10bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 shutdown_connections 2023-10-26T06:51:53.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:53.711+0000 7ffac15ff700 1 -- 172.21.15.89:0/2264958087 wait complete. 2023-10-26T06:51:54.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:54 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3358434440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:51:54.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:54 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3358434440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:51:54.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:54 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3358434440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:51:54.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:51:54.753 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:51:54.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.957+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/1419984269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e81010e0 msgr2=0x7fa5e810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:54.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.957+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/1419984269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e81010e0 0x7fa5e810d5f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000b0a0 tx=0x7fa5e000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.957+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/1419984269 shutdown_connections 2023-10-26T06:51:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.957+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/1419984269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e81010e0 0x7fa5e810d5f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.957+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/1419984269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e8106730 0x7fa5e8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.957+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/1419984269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5e8100720 0x7fa5e8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.957+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/1419984269 >> 172.21.15.89:0/1419984269 conn(0x7fa5e80fc1b0 msgr2=0x7fa5e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:54.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.958+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/1419984269 shutdown_connections 2023-10-26T06:51:54.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.958+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/1419984269 wait complete. 2023-10-26T06:51:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 Processor -- start 2023-10-26T06:51:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 -- start start 2023-10-26T06:51:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e8100720 0x7fa5e819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5e81010e0 0x7fa5e819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e8106730 0x7fa5e8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8199190 con 0x7fa5e8100720 2023-10-26T06:51:54.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8199300 con 0x7fa5e8106730 2023-10-26T06:51:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5ed5ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e81994c0 con 0x7fa5e81010e0 2023-10-26T06:51:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e8100720 0x7fa5e819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e8100720 0x7fa5e819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49736/0 (socket says 172.21.15.89:49736) 2023-10-26T06:51:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5deffd700 1 -- 172.21.15.89:0/3866300521 learned_addr learned my addr 172.21.15.89:0/3866300521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5e81010e0 0x7fa5e819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.959+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e8106730 0x7fa5e8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5deffd700 1 -- 172.21.15.89:0/3866300521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5e81010e0 msgr2=0x7fa5e819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:54.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5deffd700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5e81010e0 0x7fa5e819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5deffd700 1 -- 172.21.15.89:0/3866300521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e8106730 msgr2=0x7fa5e8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:54.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5deffd700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e8106730 0x7fa5e8198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5deffd700 1 -- 172.21.15.89:0/3866300521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e0009d00 con 0x7fa5e8100720 2023-10-26T06:51:54.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5deffd700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e8100720 0x7fa5e819e9a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9a0 tx=0x7fa5d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:54.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5e4ff9700 1 -- 172.21.15.89:0/3866300521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8009790 con 0x7fa5e8100720 2023-10-26T06:51:54.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5e4ff9700 1 -- 172.21.15.89:0/3866300521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5d8010430 con 0x7fa5e8100720 2023-10-26T06:51:54.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e81997a0 con 0x7fa5e8100720 2023-10-26T06:51:54.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5e4ff9700 1 -- 172.21.15.89:0/3866300521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8018680 con 0x7fa5e8100720 2023-10-26T06:51:54.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.960+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e81a5a40 con 0x7fa5e8100720 2023-10-26T06:51:54.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.963+0000 7fa5e4ff9700 1 -- 172.21.15.89:0/3866300521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa5d80105a0 con 0x7fa5e8100720 2023-10-26T06:51:54.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.964+0000 7fa600f51b80 1 Processor -- start 2023-10-26T06:51:54.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa600f51b80 1 -- start start 2023-10-26T06:51:54.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa5e4ff9700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa5d40a15f0 0x7fa5d40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f2580 0x5573484ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa5e4ff9700 1 -- 172.21.15.89:0/3866300521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa5d8014070 con 0x7fa5e8100720 2023-10-26T06:51:54.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa600f51b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734845f760 con 0x5573484f1c60 2023-10-26T06:51:54.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa600f51b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557348445ce0 con 0x5573484ec650 2023-10-26T06:51:54.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa600f51b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557348443600 con 0x5573484f2580 2023-10-26T06:51:54.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa5d40a15f0 0x7fa5d40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa5e6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f2580 0x5573484ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa5deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa5deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49738/0 (socket says 172.21.15.89:49738) 2023-10-26T06:51:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.965+0000 7fa5deffd700 1 -- 172.21.15.89:0/2401301411 learned_addr learned my addr 172.21.15.89:0/2401301411 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:54.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/2401301411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 msgr2=0x5573484ecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484ecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/2401301411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 msgr2=0x5573484f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:54.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/2401301411 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573483fc470 con 0x5573484f2580 2023-10-26T06:51:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484ecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:51:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5deffd700 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:51:54.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa5d40a15f0 0x7fa5d40a3ab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009610 tx=0x7fa5e0007ce0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f2580 0x5573484ec110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009c80 tx=0x7fa5d0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=519bf3a964c585ce server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5dd7fa700 1 -- 172.21.15.89:0/2401301411 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0011cb0 con 0x5573484f2580 2023-10-26T06:51:54.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5dd7fa700 1 -- 172.21.15.89:0/2401301411 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5d0011e10 con 0x5573484f2580 2023-10-26T06:51:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.966+0000 7fa5dd7fa700 1 -- 172.21.15.89:0/2401301411 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0005280 con 0x5573484f2580 2023-10-26T06:51:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 -- 172.21.15.89:0/2401301411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f2580 msgr2=0x5573484ec110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f2580 0x5573484ec110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009c80 tx=0x7fa5d0009f90 comp rx=0 tx=0).stop 2023-10-26T06:51:54.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 -- 172.21.15.89:0/2401301411 shutdown_connections 2023-10-26T06:51:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484ecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2401301411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f2580 0x5573484ec110 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 -- 172.21.15.89:0/2401301411 >> 172.21.15.89:0/2401301411 conn(0x5573484607a0 msgr2=0x5573484e6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 -- 172.21.15.89:0/2401301411 shutdown_connections 2023-10-26T06:51:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.967+0000 7fa600f51b80 1 -- 172.21.15.89:0/2401301411 wait complete. 2023-10-26T06:51:54.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 -- start start 2023-10-26T06:51:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 Processor -- start 2023-10-26T06:51:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734843eef0 con 0x5573484f2580 2023-10-26T06:51:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573483db790 con 0x5573484ec650 2023-10-26T06:51:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573483db430 con 0x5573484f1c60 2023-10-26T06:51:54.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.968+0000 7fa600f51b80 10 client.0 _ll_register_callbacks cb 0x557348325b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:54.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5deffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:39032/0 (socket says 172.21.15.89:39032) 2023-10-26T06:51:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49740/0 (socket says 172.21.15.89:49740) 2023-10-26T06:51:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/2221323512 learned_addr learned my addr 172.21.15.89:0/2221323512 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:54.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 msgr2=0x5573484f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/2221323512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 msgr2=0x5573484f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:54.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/2221323512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e81a5a40 con 0x5573484f1c60 2023-10-26T06:51:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5deffd700 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:51:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484f5820 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0011dc0 tx=0x7fa5d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1939e5156af6943e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:54.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.969+0000 7fa5ce7fc700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:51:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/2221323512 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0004260 con 0x5573484f1c60 2023-10-26T06:51:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557348443f70 con 0x5573484f1c60 2023-10-26T06:51:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/2221323512 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa5d00043c0 con 0x5573484f1c60 2023-10-26T06:51:54.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/2221323512 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5d0004580 con 0x5573484f1c60 2023-10-26T06:51:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa600f51b80 10 client.26029 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa5d00098e0 con 0x5573484f1c60 2023-10-26T06:51:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/2221323512 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557348443f70 con 0x5573484f1c60 2023-10-26T06:51:54.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/2221323512 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5d0005370 con 0x5573484f1c60 2023-10-26T06:51:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa600f51b80 10 client.26029 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa600f51b80 10 client.26029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:54.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa600f51b80 20 client.26029 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.970+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557348505e70 0x557348508290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.971+0000 7fa600f51b80 4 client.26029 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.971+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573484bb930 con 0x557348505e70 2023-10-26T06:51:54.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.971+0000 7fa5deffd700 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557348505e70 0x557348508290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.971+0000 7fa5ce7fc700 10 client.26029 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.971+0000 7fa5deffd700 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557348505e70 0x557348508290 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecb568cb411ef78 server_cookie=e8d39b7495bb4e22 in_seq=0 out_seq=0 2023-10-26T06:51:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/2221323512 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5573484bb930 con 0x557348505e70 2023-10-26T06:51:54.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa5ce7fc700 10 client.26029 handle_command_reply: tid=0 2023-10-26T06:51:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 client.26029 shutdown 2023-10-26T06:51:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 msgr2=0x5573484f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484f5820 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0011dc0 tx=0x7fa5d0004750 comp rx=0 tx=0).stop 2023-10-26T06:51:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 shutdown_connections 2023-10-26T06:51:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557348505e70 0x557348508290 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 --2- 172.21.15.89:0/2221323512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484f5820 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 >> 172.21.15.89:0/2221323512 conn(0x5573484607a0 msgr2=0x5573484eb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa5ce7fc700 0 client.26029 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:55.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.972+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 shutdown_connections 2023-10-26T06:51:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.973+0000 7fa600f51b80 1 -- 172.21.15.89:0/2221323512 wait complete. 2023-10-26T06:51:55.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.973+0000 7fa600f51b80 20 client.26029 trim_cache size 0 max 16384 2023-10-26T06:51:55.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 Processor -- start 2023-10-26T06:51:55.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 -- start start 2023-10-26T06:51:55.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484efe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573482bf9a0 con 0x5573484f2580 2023-10-26T06:51:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573484f8d10 con 0x5573484ec650 2023-10-26T06:51:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa600f51b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557348505be0 con 0x5573484f1c60 2023-10-26T06:51:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.990+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49744/0 (socket says 172.21.15.89:49744) 2023-10-26T06:51:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 -- 172.21.15.89:0/3591675450 learned_addr learned my addr 172.21.15.89:0/3591675450 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5deffd700 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 -- 172.21.15.89:0/3591675450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 msgr2=0x5573484fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484fcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 -- 172.21.15.89:0/3591675450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 msgr2=0x5573484efe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484efe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 -- 172.21.15.89:0/3591675450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55734850cea0 con 0x5573484f2580 2023-10-26T06:51:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5deffd700 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:51:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f0960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000b070 tx=0x7fa5e0006230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed57af60fa1b1aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.991+0000 7fa5ddffb700 1 -- 172.21.15.89:0/3591675450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000e040 con 0x5573484f2580 2023-10-26T06:51:55.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa5ddffb700 1 -- 172.21.15.89:0/3591675450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5e0007960 con 0x5573484f2580 2023-10-26T06:51:55.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa5ddffb700 1 -- 172.21.15.89:0/3591675450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e001edf0 con 0x5573484f2580 2023-10-26T06:51:55.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 -- 172.21.15.89:0/3591675450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 msgr2=0x5573484f0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f0960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000b070 tx=0x7fa5e0006230 comp rx=0 tx=0).stop 2023-10-26T06:51:55.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 -- 172.21.15.89:0/3591675450 shutdown_connections 2023-10-26T06:51:55.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f2580 0x5573484f0960 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484ec650 0x5573484efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 --2- 172.21.15.89:0/3591675450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484f1c60 0x5573484fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 -- 172.21.15.89:0/3591675450 >> 172.21.15.89:0/3591675450 conn(0x5573484607a0 msgr2=0x5573484ef730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 -- 172.21.15.89:0/3591675450 shutdown_connections 2023-10-26T06:51:55.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.992+0000 7fa600f51b80 1 -- 172.21.15.89:0/3591675450 wait complete. 2023-10-26T06:51:55.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 -- start start 2023-10-26T06:51:55.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 Processor -- start 2023-10-26T06:51:55.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484ec650 0x5573484fc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484f2580 0x557348559c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573484fd0c0 con 0x5573484f1c60 2023-10-26T06:51:55.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734855a200 con 0x5573484f2580 2023-10-26T06:51:55.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734855a3c0 con 0x5573484ec650 2023-10-26T06:51:55.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa600f51b80 10 client.0 _ll_register_callbacks cb 0x7fa5d80965c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:55.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa5e6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa5e6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484fc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49756/0 (socket says 172.21.15.89:49756) 2023-10-26T06:51:55.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.993+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/1244054488 learned_addr learned my addr 172.21.15.89:0/1244054488 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5e77fe700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484f2580 0x557348559c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5deffd700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484ec650 0x5573484fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/1244054488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484ec650 msgr2=0x5573484fc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484ec650 0x5573484fc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484f2580 msgr2=0x557348559c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484f2580 0x557348559c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5e6ffd700 1 -- 172.21.15.89:0/1244054488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e81997a0 con 0x5573484f1c60 2023-10-26T06:51:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5deffd700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484ec650 0x5573484fc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5e6ffd700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484fc940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009c80 tx=0x7fa5d0022640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43ad1df126983b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5ce7fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:51:55.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/1244054488 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0004470 con 0x5573484f1c60 2023-10-26T06:51:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/1244054488 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fa5d0000e40 con 0x5573484f1c60 2023-10-26T06:51:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5d0011e10 con 0x5573484f1c60 2023-10-26T06:51:55.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.994+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/1244054488 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5d00045d0 con 0x5573484f1c60 2023-10-26T06:51:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 10 client.15981 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa5e810def0 con 0x5573484f1c60 2023-10-26T06:51:55.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/1244054488 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5d0011e10 con 0x5573484f1c60 2023-10-26T06:51:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/1244054488 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5d00051a0 con 0x5573484f1c60 2023-10-26T06:51:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:55.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 20 client.15981 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734855b850 0x55734855dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 4 client.15981 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:51:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557348437ed0 con 0x55734855b850 2023-10-26T06:51:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.995+0000 7fa5deffd700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734855b850 0x55734855dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.996+0000 7fa5ce7fc700 10 client.15981 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:55.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.996+0000 7fa5deffd700 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734855b850 0x55734855dc40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a462b4cc0a90e7 server_cookie=5bf78ff3443a9bd2 in_seq=0 out_seq=0 2023-10-26T06:51:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa5ce7fc700 1 -- 172.21.15.89:0/1244054488 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557348437ed0 con 0x55734855b850 2023-10-26T06:51:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa5ce7fc700 10 client.15981 handle_command_reply: tid=0 2023-10-26T06:51:55.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 client.15981 shutdown 2023-10-26T06:51:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 msgr2=0x5573484fc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484fc940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009c80 tx=0x7fa5d0022640 comp rx=0 tx=0).stop 2023-10-26T06:51:55.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 shutdown_connections 2023-10-26T06:51:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55734855b850 0x55734855dc40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573484f2580 0x557348559c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573484ec650 0x5573484fc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa5ce7fc700 0 client.15981 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 --2- 172.21.15.89:0/1244054488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573484f1c60 0x5573484fc940 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 >> 172.21.15.89:0/1244054488 conn(0x5573484607a0 msgr2=0x5573484e0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 shutdown_connections 2023-10-26T06:51:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.997+0000 7fa600f51b80 1 -- 172.21.15.89:0/1244054488 wait complete. 2023-10-26T06:51:55.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa600f51b80 20 client.15981 trim_cache size 0 max 16384 2023-10-26T06:51:55.034 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:51:55.034 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:51:55.034 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "84ca476c-78b5-4165-bcb7-c7efdf53ca0f", 2023-10-26T06:51:55.034 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:51:55.035 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:51:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa5d40a15f0 msgr2=0x7fa5d40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa5d40a15f0 0x7fa5d40a3ab0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009610 tx=0x7fa5e0007ce0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e8100720 msgr2=0x7fa5e819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e8100720 0x7fa5e819e9a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9a0 tx=0x7fa5d800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 shutdown_connections 2023-10-26T06:51:55.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa5d40a15f0 0x7fa5d40a3ab0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5e8100720 0x7fa5e819e9a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e8106730 0x7fa5e8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 --2- 172.21.15.89:0/3866300521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5e81010e0 0x7fa5e819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 >> 172.21.15.89:0/3866300521 conn(0x7fa5e80fc1b0 msgr2=0x7fa5e80fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 shutdown_connections 2023-10-26T06:51:55.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:54.998+0000 7fa5ed5ff700 1 -- 172.21.15.89:0/3866300521 wait complete. 2023-10-26T06:51:55.039 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:51:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 -- 172.21.15.89:0/2961489762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd01010d0 msgr2=0x7f2fd010d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/2961489762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd01010d0 0x7f2fd010d5e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b0a0 tx=0x7f2fc800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 -- 172.21.15.89:0/2961489762 shutdown_connections 2023-10-26T06:51:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/2961489762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd01010d0 0x7f2fd010d5e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/2961489762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fd0106720 0x7f2fd0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/2961489762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd0100710 0x7f2fd0100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 -- 172.21.15.89:0/2961489762 >> 172.21.15.89:0/2961489762 conn(0x7f2fd00fc200 msgr2=0x7f2fd00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 -- 172.21.15.89:0/2961489762 shutdown_connections 2023-10-26T06:51:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.243+0000 7f2fd79ff700 1 -- 172.21.15.89:0/2961489762 wait complete. 2023-10-26T06:51:55.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.244+0000 7f2fd79ff700 1 Processor -- start 2023-10-26T06:51:55.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.244+0000 7f2fd79ff700 1 -- start start 2023-10-26T06:51:55.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd79ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd0100710 0x7f2fd0194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd79ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fd01010d0 0x7f2fd0194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd79ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd0106720 0x7f2fd0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd79ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0199920 con 0x7f2fd0100710 2023-10-26T06:51:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd79ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0199a90 con 0x7f2fd0106720 2023-10-26T06:51:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd79ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0199c50 con 0x7f2fd01010d0 2023-10-26T06:51:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd0100710 0x7f2fd0194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd0100710 0x7f2fd0194a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49780/0 (socket says 172.21.15.89:49780) 2023-10-26T06:51:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 -- 172.21.15.89:0/3278104376 learned_addr learned my addr 172.21.15.89:0/3278104376 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd0106720 0x7f2fd0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fd01010d0 0x7f2fd0194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 -- 172.21.15.89:0/3278104376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fd01010d0 msgr2=0x7f2fd0194fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fd01010d0 0x7f2fd0194fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 -- 172.21.15.89:0/3278104376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd0106720 msgr2=0x7f2fd0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd0106720 0x7f2fd0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.245+0000 7f2fd579b700 1 -- 172.21.15.89:0/3278104376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009d00 con 0x7f2fd0100710 2023-10-26T06:51:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.246+0000 7f2fd579b700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd0100710 0x7f2fd0194a90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e9a0 tx=0x7f2fcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.246+0000 7f2fc67fc700 1 -- 172.21.15.89:0/3278104376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc009790 con 0x7f2fd0100710 2023-10-26T06:51:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.246+0000 7f2fc67fc700 1 -- 172.21.15.89:0/3278104376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fcc010430 con 0x7f2fd0100710 2023-10-26T06:51:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.246+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd0199f30 con 0x7f2fd0100710 2023-10-26T06:51:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.246+0000 7f2fc67fc700 1 -- 172.21.15.89:0/3278104376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc018680 con 0x7f2fd0100710 2023-10-26T06:51:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.246+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd01a17e0 con 0x7f2fd0100710 2023-10-26T06:51:55.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.249+0000 7f2fc67fc700 1 -- 172.21.15.89:0/3278104376 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2fcc0105a0 con 0x7f2fd0100710 2023-10-26T06:51:55.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.250+0000 7f2feb396b80 1 Processor -- start 2023-10-26T06:51:55.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.250+0000 7f2fc67fc700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.250+0000 7f2feb396b80 1 -- start start 2023-10-26T06:51:55.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.250+0000 7f2fc67fc700 1 -- 172.21.15.89:0/3278104376 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2fcc014070 con 0x7f2fd0100710 2023-10-26T06:51:55.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa8341fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa833ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2feb396b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa82af6e0 con 0x558fa8341be0 2023-10-26T06:51:55.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2feb396b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa8295c60 con 0x558fa833c5d0 2023-10-26T06:51:55.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2feb396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa8293580 con 0x558fa8342500 2023-10-26T06:51:55.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa8341fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd4f9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa8341fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49782/0 (socket says 172.21.15.89:49782) 2023-10-26T06:51:55.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 -- 172.21.15.89:0/1762212174 learned_addr learned my addr 172.21.15.89:0/1762212174 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:55.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa833ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 -- 172.21.15.89:0/1762212174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 msgr2=0x558fa833c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 -- 172.21.15.89:0/1762212174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 msgr2=0x558fa833ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa833ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd579b700 1 -- 172.21.15.89:0/1762212174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa824c3f0 con 0x558fa8341be0 2023-10-26T06:51:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.251+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa833ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.252+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b780 tx=0x7f2fc8006230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.252+0000 7f2fd579b700 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa8341fc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0a2010 tx=0x7f2fcc099b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2984a00fde3f2283 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.252+0000 7f2fbb7fe700 1 -- 172.21.15.89:0/1762212174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc09e300 con 0x558fa8341be0 2023-10-26T06:51:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.252+0000 7f2fbb7fe700 1 -- 172.21.15.89:0/1762212174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fcc0a2540 con 0x558fa8341be0 2023-10-26T06:51:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.252+0000 7f2fbb7fe700 1 -- 172.21.15.89:0/1762212174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0996a0 con 0x558fa8341be0 2023-10-26T06:51:55.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 -- 172.21.15.89:0/1762212174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 msgr2=0x558fa8341fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa8341fc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0a2010 tx=0x7f2fcc099b90 comp rx=0 tx=0).stop 2023-10-26T06:51:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 -- 172.21.15.89:0/1762212174 shutdown_connections 2023-10-26T06:51:55.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa8341fc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa833ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1762212174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 -- 172.21.15.89:0/1762212174 >> 172.21.15.89:0/1762212174 conn(0x558fa82b0720 msgr2=0x558fa8336b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 -- 172.21.15.89:0/1762212174 shutdown_connections 2023-10-26T06:51:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 -- 172.21.15.89:0/1762212174 wait complete. 2023-10-26T06:51:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 -- start start 2023-10-26T06:51:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.253+0000 7f2feb396b80 1 Processor -- start 2023-10-26T06:51:55.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa83484c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa83457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8345ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2feb396b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa828ee70 con 0x558fa8341be0 2023-10-26T06:51:55.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2feb396b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa822b710 con 0x558fa8342500 2023-10-26T06:51:55.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2feb396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa822b3b0 con 0x558fa833c5d0 2023-10-26T06:51:55.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2feb396b80 10 client.0 _ll_register_callbacks cb 0x558fa8175a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:55.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2fd4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa83457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2fd4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa83457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49788/0 (socket says 172.21.15.89:49788) 2023-10-26T06:51:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/4198817447 learned_addr learned my addr 172.21.15.89:0/4198817447 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2fd579b700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa83484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.254+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8345ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/4198817447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 msgr2=0x558fa83484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa83484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 msgr2=0x558fa8345ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8345ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/4198817447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd01a17e0 con 0x558fa8341be0 2023-10-26T06:51:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fd579b700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa83484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:51:55.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa83457b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005950 tx=0x7f2fc000d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751cd7fb9cdde192 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:51:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/4198817447 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0012040 con 0x558fa8341be0 2023-10-26T06:51:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/4198817447 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2fc0004830 con 0x558fa8341be0 2023-10-26T06:51:55.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fa8293ef0 con 0x558fa8341be0 2023-10-26T06:51:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/4198817447 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc00049f0 con 0x558fa8341be0 2023-10-26T06:51:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2feb396b80 10 client.15993 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.255+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558fa80b9de0 con 0x558fa8341be0 2023-10-26T06:51:55.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/4198817447 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fa8293ef0 con 0x558fa8341be0 2023-10-26T06:51:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/4198817447 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fc0021840 con 0x558fa8341be0 2023-10-26T06:51:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2feb396b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:55.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2feb396b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2feb396b80 20 client.15993 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2feb396b80 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa8355ff0 0x558fa8358410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2feb396b80 4 client.15993 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:51:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fa830b8b0 con 0x558fa8355ff0 2023-10-26T06:51:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.256+0000 7f2fd579b700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa8355ff0 0x558fa8358410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.257+0000 7f2fb8ff9700 10 client.15993 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.257+0000 7f2fd579b700 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa8355ff0 0x558fa8358410 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8568c375ac4ff4da server_cookie=878e1d2de8054fe3 in_seq=0 out_seq=0 2023-10-26T06:51:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.257+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/4198817447 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fa830b8b0 con 0x558fa8355ff0 2023-10-26T06:51:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.257+0000 7f2fb8ff9700 10 client.15993 handle_command_reply: tid=0 2023-10-26T06:51:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 client.15993 shutdown 2023-10-26T06:51:55.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 msgr2=0x558fa83457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa83457b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005950 tx=0x7f2fc000d910 comp rx=0 tx=0).stop 2023-10-26T06:51:55.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 shutdown_connections 2023-10-26T06:51:55.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa8355ff0 0x558fa8358410 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8345ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2fb8ff9700 0 client.15993 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:55.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa83484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 --2- 172.21.15.89:0/4198817447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa83457b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 >> 172.21.15.89:0/4198817447 conn(0x558fa82b0720 msgr2=0x558fa833b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 shutdown_connections 2023-10-26T06:51:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 1 -- 172.21.15.89:0/4198817447 wait complete. 2023-10-26T06:51:55.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.258+0000 7f2feb396b80 20 client.15993 trim_cache size 0 max 16384 2023-10-26T06:51:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.275+0000 7f2feb396b80 1 Processor -- start 2023-10-26T06:51:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.275+0000 7f2feb396b80 1 -- start start 2023-10-26T06:51:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.275+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa833fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.275+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8340920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2feb396b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa810f920 con 0x558fa8341be0 2023-10-26T06:51:55.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2feb396b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa8333290 con 0x558fa8342500 2023-10-26T06:51:55.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2feb396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa83383b0 con 0x558fa833c5d0 2023-10-26T06:51:55.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49794/0 (socket says 172.21.15.89:49794) 2023-10-26T06:51:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/1217415583 learned_addr learned my addr 172.21.15.89:0/1217415583 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd579b700 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa833fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8340920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/1217415583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 msgr2=0x558fa833fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa833fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/1217415583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 msgr2=0x558fa8340920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8340920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/1217415583 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fa80a2c60 con 0x558fa8341be0 2023-10-26T06:51:55.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8340920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:51:55.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.276+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834ceb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005d90 tx=0x7f2fc0021ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800f9f3182fca7b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2fbbfff700 1 -- 172.21.15.89:0/1217415583 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0012040 con 0x558fa8341be0 2023-10-26T06:51:55.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2fbbfff700 1 -- 172.21.15.89:0/1217415583 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc00218c0 con 0x558fa8341be0 2023-10-26T06:51:55.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2fbbfff700 1 -- 172.21.15.89:0/1217415583 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00038e0 con 0x558fa8341be0 2023-10-26T06:51:55.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 -- 172.21.15.89:0/1217415583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 msgr2=0x558fa834ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834ceb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005d90 tx=0x7f2fc0021ef0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 -- 172.21.15.89:0/1217415583 shutdown_connections 2023-10-26T06:51:55.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa8342500 0x558fa8340920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa833c5d0 0x558fa833fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 --2- 172.21.15.89:0/1217415583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834ceb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 -- 172.21.15.89:0/1217415583 >> 172.21.15.89:0/1217415583 conn(0x558fa82b0720 msgr2=0x558fa833f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 -- 172.21.15.89:0/1217415583 shutdown_connections 2023-10-26T06:51:55.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.277+0000 7f2feb396b80 1 -- 172.21.15.89:0/1217415583 wait complete. 2023-10-26T06:51:55.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 -- start start 2023-10-26T06:51:55.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 Processor -- start 2023-10-26T06:51:55.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa834cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa833ba50 con 0x558fa8341be0 2023-10-26T06:51:55.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa833bbc0 con 0x558fa833c5d0 2023-10-26T06:51:55.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.278+0000 7f2feb396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fa833bd80 con 0x558fa8342500 2023-10-26T06:51:55.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2feb396b80 10 client.0 _ll_register_callbacks cb 0x7f2fc000ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:51:55.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49804/0 (socket says 172.21.15.89:49804) 2023-10-26T06:51:55.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/2849192342 learned_addr learned my addr 172.21.15.89:0/2849192342 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:51:55.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd579b700 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa834cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd5f9c700 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/2849192342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 msgr2=0x558fa833b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 msgr2=0x558fa834cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:51:55.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa834cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 -- 172.21.15.89:0/2849192342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fa80b9de0 con 0x558fa8341be0 2023-10-26T06:51:55.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fd4f9a700 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834d0a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0021ba0 tx=0x7f2fc0005170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0fc9d83adae404 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:51:55.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.279+0000 7f2fb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:51:55.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/2849192342 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0021610 con 0x558fa8341be0 2023-10-26T06:51:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fcc0a2540 con 0x558fa8341be0 2023-10-26T06:51:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/2849192342 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f2fc0000e40 con 0x558fa8341be0 2023-10-26T06:51:55.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/2849192342 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc0018680 con 0x558fa8341be0 2023-10-26T06:51:55.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 10 client.15999 fetch_fsmap learned FSMap version 20 2023-10-26T06:51:55.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558fa80e0250 con 0x558fa8341be0 2023-10-26T06:51:55.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/2849192342 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fcc0a2540 con 0x558fa8341be0 2023-10-26T06:51:55.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/2849192342 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fc0031070 con 0x558fa8341be0 2023-10-26T06:51:55.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:51:55.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:51:55.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 20 client.15999 populate_metadata read hostname 'smithi089' 2023-10-26T06:51:55.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa83aaf10 0x558fa83ad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:51:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 4 client.15999 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:51:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.280+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fa8287e50 con 0x558fa83aaf10 2023-10-26T06:51:55.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2fd579b700 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa83aaf10 0x558fa83ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:51:55.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2fd579b700 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa83aaf10 0x558fa83ad300 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e1be6cafb2e4db server_cookie=b7e39b10a0270e15 in_seq=0 out_seq=0 2023-10-26T06:51:55.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2fb8ff9700 10 client.15999 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:55.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2fb8ff9700 1 -- 172.21.15.89:0/2849192342 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558fa8287e50 con 0x558fa83aaf10 2023-10-26T06:51:55.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2fb8ff9700 10 client.15999 handle_command_reply: tid=0 2023-10-26T06:51:55.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2feb396b80 1 client.15999 shutdown 2023-10-26T06:51:55.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 msgr2=0x558fa834d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.281+0000 7f2feb396b80 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834d0a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0021ba0 tx=0x7f2fc0005170 comp rx=0 tx=0).stop 2023-10-26T06:51:55.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 shutdown_connections 2023-10-26T06:51:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558fa83aaf10 0x558fa83ad300 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558fa8342500 0x558fa833b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558fa833c5d0 0x558fa834cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 --2- 172.21.15.89:0/2849192342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558fa8341be0 0x558fa834d0a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fb8ff9700 0 client.15999 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:51:55.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 >> 172.21.15.89:0/2849192342 conn(0x558fa82b0720 msgr2=0x558fa8340920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 shutdown_connections 2023-10-26T06:51:55.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 1 -- 172.21.15.89:0/2849192342 wait complete. 2023-10-26T06:51:55.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2feb396b80 20 client.15999 trim_cache size 0 max 16384 2023-10-26T06:51:55.315 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:51:55.316 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:51:55.316 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:51:55.316 INFO:teuthology.orchestra.run.smithi089.stdout: "84ca476c-78b5-4165-bcb7-c7efdf53ca0f": { 2023-10-26T06:51:55.316 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:51:55.317 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "84ca476c-78b5-4165-bcb7-c7efdf53ca0f", 2023-10-26T06:51:55.317 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:51:55.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:51:55.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:51:55.318 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:51:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2fbc0a1670 msgr2=0x7f2fbc0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b780 tx=0x7f2fc8006230 comp rx=0 tx=0).stop 2023-10-26T06:51:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd0100710 msgr2=0x7f2fd0194a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:51:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd0100710 0x7f2fd0194a90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e9a0 tx=0x7f2fcc00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 shutdown_connections 2023-10-26T06:51:55.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2fbc0a1670 0x7f2fbc0a3b30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fd0100710 0x7f2fd0194a90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fd0106720 0x7f2fd0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 --2- 172.21.15.89:0/3278104376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fd01010d0 0x7f2fd0194fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:51:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.282+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 >> 172.21.15.89:0/3278104376 conn(0x7f2fd00fc200 msgr2=0x7f2fd00fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:51:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.283+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 shutdown_connections 2023-10-26T06:51:55.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:51:55.283+0000 7f2fd79ff700 1 -- 172.21.15.89:0/3278104376 wait complete. 2023-10-26T06:51:55.323 INFO:tasks.cephfs.filesystem:scrub status for tag:84ca476c-78b5-4165-bcb7-c7efdf53ca0f - {'path': '/', 'tag': '84ca476c-78b5-4165-bcb7-c7efdf53ca0f', 'options': 'recursive,force'} 2023-10-26T06:51:55.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:55 smithi089 ceph-mon[95031]: pgmap v460: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:55 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:51:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:55 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:55.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:55 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:51:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:55 smithi192 ceph-mon[131661]: pgmap v460: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:55 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:51:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:55 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:55.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:55 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:51:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:55 smithi135 ceph-mon[131403]: pgmap v460: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:55 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:51:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:55 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:55.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:55 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:51:56.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:56 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:56.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:56 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:56.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:56 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:51:57.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:57 smithi089 ceph-mon[95031]: pgmap v461: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:57 smithi192 ceph-mon[131661]: pgmap v461: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:57 smithi135 ceph-mon[131403]: pgmap v461: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 2 op/s 2023-10-26T06:51:59.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:59 smithi089 ceph-mon[95031]: pgmap v462: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2023-10-26T06:51:59.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:51:59 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:51:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:59 smithi192 ceph-mon[131661]: pgmap v462: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2023-10-26T06:51:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:51:59 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:51:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:59 smithi135 ceph-mon[131403]: pgmap v462: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2023-10-26T06:51:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:51:59 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:52:01.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:01 smithi089 ceph-mon[95031]: pgmap v463: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.8 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:52:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:01 smithi192 ceph-mon[131661]: pgmap v463: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.8 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:52:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:01 smithi135 ceph-mon[131403]: pgmap v463: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.8 KiB/s rd, 938 B/s wr, 2 op/s 2023-10-26T06:52:03.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:03 smithi089 ceph-mon[95031]: pgmap v464: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2023-10-26T06:52:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:03 smithi192 ceph-mon[131661]: pgmap v464: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2023-10-26T06:52:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:03 smithi135 ceph-mon[131403]: pgmap v464: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2023-10-26T06:52:05.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:05 smithi089 ceph-mon[95031]: pgmap v465: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-10-26T06:52:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:05 smithi192 ceph-mon[131661]: pgmap v465: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-10-26T06:52:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:05 smithi135 ceph-mon[131403]: pgmap v465: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-10-26T06:52:07.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:07 smithi089 ceph-mon[95031]: pgmap v466: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:52:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:07 smithi192 ceph-mon[131661]: pgmap v466: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:52:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:07 smithi135 ceph-mon[131403]: pgmap v466: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-10-26T06:52:09.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:09 smithi089 ceph-mon[95031]: pgmap v467: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-10-26T06:52:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:09 smithi192 ceph-mon[131661]: pgmap v467: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-10-26T06:52:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:09 smithi135 ceph-mon[131403]: pgmap v467: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 17 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-10-26T06:52:11.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:11 smithi089 ceph-mon[95031]: pgmap v468: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 15 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-10-26T06:52:11.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:11 smithi192 ceph-mon[131661]: pgmap v468: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 15 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-10-26T06:52:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:11 smithi135 ceph-mon[131403]: pgmap v468: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 15 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-10-26T06:52:13.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:13 smithi089 ceph-mon[95031]: pgmap v469: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-10-26T06:52:13.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:13 smithi192 ceph-mon[131661]: pgmap v469: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-10-26T06:52:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:13 smithi135 ceph-mon[131403]: pgmap v469: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-10-26T06:52:15.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:15 smithi089 ceph-mon[95031]: pgmap v470: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-10-26T06:52:15.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:15 smithi192 ceph-mon[131661]: pgmap v470: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-10-26T06:52:15.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:15 smithi135 ceph-mon[131403]: pgmap v470: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-10-26T06:52:17.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:17 smithi089 ceph-mon[95031]: pgmap v471: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 4 op/s 2023-10-26T06:52:17.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:17 smithi192 ceph-mon[131661]: pgmap v471: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 4 op/s 2023-10-26T06:52:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:17 smithi135 ceph-mon[131403]: pgmap v471: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 4 op/s 2023-10-26T06:52:19.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:19 smithi192 ceph-mon[131661]: pgmap v472: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 3 op/s 2023-10-26T06:52:19.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:19 smithi135 ceph-mon[131403]: pgmap v472: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 3 op/s 2023-10-26T06:52:19.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:19 smithi089 ceph-mon[95031]: pgmap v472: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 3 op/s 2023-10-26T06:52:21.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:21 smithi192 ceph-mon[131661]: pgmap v473: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2023-10-26T06:52:21.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:21 smithi135 ceph-mon[131403]: pgmap v473: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2023-10-26T06:52:21.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:21 smithi089 ceph-mon[95031]: pgmap v473: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2023-10-26T06:52:23.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:23 smithi192 ceph-mon[131661]: pgmap v474: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2023-10-26T06:52:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:23 smithi135 ceph-mon[131403]: pgmap v474: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2023-10-26T06:52:23.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:23 smithi089 ceph-mon[95031]: pgmap v474: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2023-10-26T06:52:25.324 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:52:25.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 -- 172.21.15.89:0/2455044803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4073540 msgr2=0x7fbbf410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 --2- 172.21.15.89:0/2455044803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4073540 0x7fbbf410ca10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009990 tx=0x7fbbe4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 -- 172.21.15.89:0/2455044803 shutdown_connections 2023-10-26T06:52:25.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 --2- 172.21.15.89:0/2455044803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf410cf50 0x7fbbf410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 --2- 172.21.15.89:0/2455044803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbf4072c20 0x7fbbf4073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 --2- 172.21.15.89:0/2455044803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4073540 0x7fbbf410ca10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 -- 172.21.15.89:0/2455044803 >> 172.21.15.89:0/2455044803 conn(0x7fbbf40fc200 msgr2=0x7fbbf40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:25.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 -- 172.21.15.89:0/2455044803 shutdown_connections 2023-10-26T06:52:25.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.534+0000 7fbbfadff700 1 -- 172.21.15.89:0/2455044803 wait complete. 2023-10-26T06:52:25.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 Processor -- start 2023-10-26T06:52:25.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 -- start start 2023-10-26T06:52:25.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbf4072c20 0x7fbbf4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf4073540 0x7fbbf4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf410cf50 0x7fbbf419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf419dd30 con 0x7fbbf410cf50 2023-10-26T06:52:25.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf419dea0 con 0x7fbbf4072c20 2023-10-26T06:52:25.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.535+0000 7fbbfadff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf419e060 con 0x7fbbf4073540 2023-10-26T06:52:25.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf410cf50 0x7fbbf419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf410cf50 0x7fbbf419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54630/0 (socket says 172.21.15.89:54630) 2023-10-26T06:52:25.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 -- 172.21.15.89:0/610683628 learned_addr learned my addr 172.21.15.89:0/610683628 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:25.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbebfff700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf4073540 0x7fbbf4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbf4072c20 0x7fbbf4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 -- 172.21.15.89:0/610683628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf4073540 msgr2=0x7fbbf4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf4073540 0x7fbbf4199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 -- 172.21.15.89:0/610683628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbf4072c20 msgr2=0x7fbbf4198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbf4072c20 0x7fbbf4198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 -- 172.21.15.89:0/610683628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40095f0 con 0x7fbbf410cf50 2023-10-26T06:52:25.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.536+0000 7fbbf939c700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf410cf50 0x7fbbf419d6f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007ee0 tx=0x7fbbec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.537+0000 7fbbe9ffb700 1 -- 172.21.15.89:0/610683628 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00dd20 con 0x7fbbf410cf50 2023-10-26T06:52:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.537+0000 7fbbe9ffb700 1 -- 172.21.15.89:0/610683628 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbec00f040 con 0x7fbbf410cf50 2023-10-26T06:52:25.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.537+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf419e340 con 0x7fbbf410cf50 2023-10-26T06:52:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.537+0000 7fbbe9ffb700 1 -- 172.21.15.89:0/610683628 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0048a0 con 0x7fbbf410cf50 2023-10-26T06:52:25.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.537+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf41a5b80 con 0x7fbbf410cf50 2023-10-26T06:52:25.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.541+0000 7fbbe9ffb700 1 -- 172.21.15.89:0/610683628 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbbec003a00 con 0x7fbbf410cf50 2023-10-26T06:52:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.542+0000 7fbc0e637b80 1 Processor -- start 2023-10-26T06:52:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.542+0000 7fbc0e637b80 1 -- start start 2023-10-26T06:52:25.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b3500 0x563fbb3ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb3206e0 con 0x563fbb3ad5d0 2023-10-26T06:52:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb306c60 con 0x563fbb3b3500 2023-10-26T06:52:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb304580 con 0x563fbb3b2be0 2023-10-26T06:52:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbe9ffb700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbbdc0a15f0 0x7fbbdc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbe9ffb700 1 -- 172.21.15.89:0/610683628 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbbec0cc920 con 0x7fbbf410cf50 2023-10-26T06:52:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:49404/0 (socket says 172.21.15.89:49404) 2023-10-26T06:52:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbebfff700 1 -- 172.21.15.89:0/2061636231 learned_addr learned my addr 172.21.15.89:0/2061636231 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:25.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbebfff700 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbf939c700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbbdc0a15f0 0x7fbbdc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b3500 0x563fbb3ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbf8b9b700 1 -- 172.21.15.89:0/2061636231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 msgr2=0x563fbb3b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbf8b9b700 1 -- 172.21.15.89:0/2061636231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 msgr2=0x563fbb3ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.543+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.544+0000 7fbbf8b9b700 1 -- 172.21.15.89:0/2061636231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fbb2bd3f0 con 0x563fbb3b3500 2023-10-26T06:52:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.544+0000 7fbbf939c700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbbdc0a15f0 0x7fbbdc0a3ab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0ca0b0 tx=0x7fbbec004ad0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.544+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b3500 0x563fbb3ad090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009cb0 tx=0x7fbbf0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f1820e255e0c49 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.544+0000 7fbbdaffd700 1 -- 172.21.15.89:0/2061636231 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0011d20 con 0x563fbb3b3500 2023-10-26T06:52:25.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.544+0000 7fbbdaffd700 1 -- 172.21.15.89:0/2061636231 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf0011e80 con 0x563fbb3b3500 2023-10-26T06:52:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.544+0000 7fbbdaffd700 1 -- 172.21.15.89:0/2061636231 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0005280 con 0x563fbb3b3500 2023-10-26T06:52:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 -- 172.21.15.89:0/2061636231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b3500 msgr2=0x563fbb3ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b3500 0x563fbb3ad090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009cb0 tx=0x7fbbf0011440 comp rx=0 tx=0).stop 2023-10-26T06:52:25.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 -- 172.21.15.89:0/2061636231 shutdown_connections 2023-10-26T06:52:25.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/2061636231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b3500 0x563fbb3ad090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 -- 172.21.15.89:0/2061636231 >> 172.21.15.89:0/2061636231 conn(0x563fbb321720 msgr2=0x563fbb3a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:25.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 -- 172.21.15.89:0/2061636231 shutdown_connections 2023-10-26T06:52:25.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.545+0000 7fbc0e637b80 1 -- 172.21.15.89:0/2061636231 wait complete. 2023-10-26T06:52:25.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 -- start start 2023-10-26T06:52:25.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 Processor -- start 2023-10-26T06:52:25.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 0x563fbb3b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb2ffe70 con 0x563fbb3ad5d0 2023-10-26T06:52:25.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb29c710 con 0x563fbb3b2be0 2023-10-26T06:52:25.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb29c3b0 con 0x563fbb3b3500 2023-10-26T06:52:25.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.546+0000 7fbc0e637b80 10 client.0 _ll_register_callbacks cb 0x563fbb1e6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:25.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3b94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54638/0 (socket says 172.21.15.89:54638) 2023-10-26T06:52:25.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbebfff700 1 -- 172.21.15.89:0/514599615 learned_addr learned my addr 172.21.15.89:0/514599615 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:25.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbebfff700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 0x563fbb3b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbf8b9b700 1 -- 172.21.15.89:0/514599615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 msgr2=0x563fbb3b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 0x563fbb3b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbf8b9b700 1 -- 172.21.15.89:0/514599615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 msgr2=0x563fbb3b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbf8b9b700 1 -- 172.21.15.89:0/514599615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf00098e0 con 0x563fbb3b2be0 2023-10-26T06:52:25.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbebfff700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:25.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b67c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0000c00 tx=0x7fbbf0004b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c83f98f56dab355c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.547+0000 7fbbd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:52:25.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbbd3fff700 1 -- 172.21.15.89:0/514599615 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0026070 con 0x563fbb3b2be0 2023-10-26T06:52:25.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fbb37c8b0 con 0x563fbb3b2be0 2023-10-26T06:52:25.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbbd3fff700 1 -- 172.21.15.89:0/514599615 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbbf0004050 con 0x563fbb3b2be0 2023-10-26T06:52:25.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbbd3fff700 1 -- 172.21.15.89:0/514599615 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf0004210 con 0x563fbb3b2be0 2023-10-26T06:52:25.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 10 client.26048 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:25.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563fbb12ade0 con 0x563fbb3b2be0 2023-10-26T06:52:25.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbbd3fff700 1 -- 172.21.15.89:0/514599615 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563fbb37c8b0 con 0x563fbb3b2be0 2023-10-26T06:52:25.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbbd3fff700 1 -- 172.21.15.89:0/514599615 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbbf0029030 con 0x563fbb3b2be0 2023-10-26T06:52:25.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 10 client.26048 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:25.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 10 client.26048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:25.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 20 client.26048 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:25.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb3c6cb0 0x563fbb3c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 4 client.26048 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:52:25.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.548+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fbb2f8e50 con 0x563fbb3c6cb0 2023-10-26T06:52:25.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.549+0000 7fbbebfff700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb3c6cb0 0x563fbb3c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.549+0000 7fbbd3fff700 10 client.26048 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.549+0000 7fbbebfff700 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb3c6cb0 0x563fbb3c90d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5aa58ac8eaae07 server_cookie=e2ba4e635ffb0c51 in_seq=0 out_seq=0 2023-10-26T06:52:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbbd3fff700 1 -- 172.21.15.89:0/514599615 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563fbb2f8e50 con 0x563fbb3c6cb0 2023-10-26T06:52:25.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbbd3fff700 10 client.26048 handle_command_reply: tid=0 2023-10-26T06:52:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 client.26048 shutdown 2023-10-26T06:52:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 msgr2=0x563fbb3b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b67c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0000c00 tx=0x7fbbf0004b70 comp rx=0 tx=0).stop 2023-10-26T06:52:25.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 shutdown_connections 2023-10-26T06:52:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb3c6cb0 0x563fbb3c90d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 0x563fbb3b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/514599615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b67c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbbd3fff700 0 client.26048 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 >> 172.21.15.89:0/514599615 conn(0x563fbb321720 msgr2=0x563fbb3ac080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:25.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 shutdown_connections 2023-10-26T06:52:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 1 -- 172.21.15.89:0/514599615 wait complete. 2023-10-26T06:52:25.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.550+0000 7fbc0e637b80 20 client.26048 trim_cache size 0 max 16384 2023-10-26T06:52:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.567+0000 7fbc0e637b80 1 Processor -- start 2023-10-26T06:52:25.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.567+0000 7fbc0e637b80 1 -- start start 2023-10-26T06:52:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3b3500 0x563fbb3b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb180920 con 0x563fbb3b3500 2023-10-26T06:52:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb3ad090 con 0x563fbb3ad5d0 2023-10-26T06:52:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb3a8650 con 0x563fbb3b2be0 2023-10-26T06:52:25.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51302/0 (socket says 172.21.15.89:51302) 2023-10-26T06:52:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 -- 172.21.15.89:0/772843130 learned_addr learned my addr 172.21.15.89:0/772843130 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3b3500 0x563fbb3b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 -- 172.21.15.89:0/772843130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 msgr2=0x563fbb3b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 -- 172.21.15.89:0/772843130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3b3500 msgr2=0x563fbb3b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3b3500 0x563fbb3b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.568+0000 7fbbebfff700 1 -- 172.21.15.89:0/772843130 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fbb113c60 con 0x563fbb3ad5d0 2023-10-26T06:52:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbbebfff700 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009960 tx=0x7fbbe4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17f000b8bbf4d34 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbbdb7fe700 1 -- 172.21.15.89:0/772843130 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401d070 con 0x563fbb3ad5d0 2023-10-26T06:52:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbbdb7fe700 1 -- 172.21.15.89:0/772843130 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe400be60 con 0x563fbb3ad5d0 2023-10-26T06:52:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbbdb7fe700 1 -- 172.21.15.89:0/772843130 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe402a7b0 con 0x563fbb3ad5d0 2023-10-26T06:52:25.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 -- 172.21.15.89:0/772843130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3ad5d0 msgr2=0x563fbb3bdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009960 tx=0x7fbbe4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 -- 172.21.15.89:0/772843130 shutdown_connections 2023-10-26T06:52:25.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3b3500 0x563fbb3b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdee0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/772843130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b2be0 0x563fbb3b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 -- 172.21.15.89:0/772843130 >> 172.21.15.89:0/772843130 conn(0x563fbb321720 msgr2=0x563fbb3b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 -- 172.21.15.89:0/772843130 shutdown_connections 2023-10-26T06:52:25.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.569+0000 7fbc0e637b80 1 -- 172.21.15.89:0/772843130 wait complete. 2023-10-26T06:52:25.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 -- start start 2023-10-26T06:52:25.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 Processor -- start 2023-10-26T06:52:25.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 0x563fbb3b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb3b2450 con 0x563fbb3ad5d0 2023-10-26T06:52:25.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb3b2590 con 0x563fbb3b2be0 2023-10-26T06:52:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fbb3b2750 con 0x563fbb3b3500 2023-10-26T06:52:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.570+0000 7fbc0e637b80 10 client.0 _ll_register_callbacks cb 0x563fbb1e6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54664/0 (socket says 172.21.15.89:54664) 2023-10-26T06:52:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 -- 172.21.15.89:0/1441806440 learned_addr learned my addr 172.21.15.89:0/1441806440 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 -- 172.21.15.89:0/1441806440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 msgr2=0x563fbb3b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 0x563fbb3b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 -- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 msgr2=0x563fbb3b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 -- 172.21.15.89:0/1441806440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe4032040 con 0x563fbb3ad5d0 2023-10-26T06:52:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbebfff700 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdd90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c010 tx=0x7fbbe4005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8819ef0e9d1b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.571+0000 7fbbd3fff700 1 -- 172.21.15.89:0/1441806440 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401d070 con 0x563fbb3ad5d0 2023-10-26T06:52:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fbb304ef0 con 0x563fbb3ad5d0 2023-10-26T06:52:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbbd3fff700 1 -- 172.21.15.89:0/1441806440 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbbe4010890 con 0x563fbb3ad5d0 2023-10-26T06:52:25.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbbd3fff700 1 -- 172.21.15.89:0/1441806440 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe4010a50 con 0x563fbb3ad5d0 2023-10-26T06:52:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 10 client.16017 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563fbb151250 con 0x563fbb3ad5d0 2023-10-26T06:52:25.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbbd3fff700 1 -- 172.21.15.89:0/1441806440 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563fbb304ef0 con 0x563fbb3ad5d0 2023-10-26T06:52:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbbd3fff700 1 -- 172.21.15.89:0/1441806440 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbbe4029470 con 0x563fbb3ad5d0 2023-10-26T06:52:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:25.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 20 client.16017 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb41bce0 0x563fbb41e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 4 client.16017 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:52:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.572+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563fbb2f8b70 con 0x563fbb41bce0 2023-10-26T06:52:25.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.573+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb41bce0 0x563fbb41e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.573+0000 7fbbf8b9b700 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb41bce0 0x563fbb41e100 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ac77673cf68954 server_cookie=449f3fe8d3bdbfb1 in_seq=0 out_seq=0 2023-10-26T06:52:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.573+0000 7fbbd3fff700 10 client.16017 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.573+0000 7fbbd3fff700 1 -- 172.21.15.89:0/1441806440 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563fbb2f8b70 con 0x563fbb41bce0 2023-10-26T06:52:25.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.573+0000 7fbbd3fff700 10 client.16017 handle_command_reply: tid=0 2023-10-26T06:52:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.573+0000 7fbc0e637b80 1 client.16017 shutdown 2023-10-26T06:52:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 msgr2=0x563fbb3bdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdd90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c010 tx=0x7fbbe4005440 comp rx=0 tx=0).stop 2023-10-26T06:52:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 shutdown_connections 2023-10-26T06:52:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563fbb3ad5d0 0x563fbb3bdd90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563fbb41bce0 0x563fbb41e100 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563fbb3b3500 0x563fbb3b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 --2- 172.21.15.89:0/1441806440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fbb3b2be0 0x563fbb3b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbbd3fff700 0 client.16017 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:25.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 >> 172.21.15.89:0/1441806440 conn(0x563fbb321720 msgr2=0x563fbb3b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 shutdown_connections 2023-10-26T06:52:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 1 -- 172.21.15.89:0/1441806440 wait complete. 2023-10-26T06:52:25.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbc0e637b80 20 client.16017 trim_cache size 0 max 16384 2023-10-26T06:52:25.609 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:52:25.609 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:52:25.609 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:52:25.610 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:52:25.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbbdc0a15f0 msgr2=0x7fbbdc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbbfadff700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbbdc0a15f0 0x7fbbdc0a3ab0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0ca0b0 tx=0x7fbbec004ad0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf410cf50 msgr2=0x7fbbf419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.574+0000 7fbbfadff700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf410cf50 0x7fbbf419d6f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007ee0 tx=0x7fbbec00d3b0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 shutdown_connections 2023-10-26T06:52:25.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbbdc0a15f0 0x7fbbdc0a3ab0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf410cf50 0x7fbbf419d6f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbf4072c20 0x7fbbf4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 --2- 172.21.15.89:0/610683628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf4073540 0x7fbbf4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 >> 172.21.15.89:0/610683628 conn(0x7fbbf40fc200 msgr2=0x7fbbf40fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 shutdown_connections 2023-10-26T06:52:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.575+0000 7fbbfadff700 1 -- 172.21.15.89:0/610683628 wait complete. 2023-10-26T06:52:25.614 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:25 smithi089 ceph-mon[95031]: pgmap v475: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2023-10-26T06:52:25.615 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:52:25.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:52:25.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:25 smithi192 ceph-mon[131661]: pgmap v475: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2023-10-26T06:52:25.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:25 smithi135 ceph-mon[131403]: pgmap v475: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2023-10-26T06:52:25.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.812+0000 7efd291ff700 1 -- 172.21.15.89:0/2198471474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd241066b0 msgr2=0x7efd24106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.812+0000 7efd291ff700 1 --2- 172.21.15.89:0/2198471474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd241066b0 0x7efd24106a90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efd14009990 tx=0x7efd14009ca0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.812+0000 7efd291ff700 1 -- 172.21.15.89:0/2198471474 shutdown_connections 2023-10-26T06:52:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.812+0000 7efd291ff700 1 --2- 172.21.15.89:0/2198471474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd241066b0 0x7efd24106a90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.812+0000 7efd291ff700 1 --2- 172.21.15.89:0/2198471474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd24101060 0x7efd2410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.812+0000 7efd291ff700 1 --2- 172.21.15.89:0/2198471474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd241006a0 0x7efd24100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.812+0000 7efd291ff700 1 -- 172.21.15.89:0/2198471474 >> 172.21.15.89:0/2198471474 conn(0x7efd240fc170 msgr2=0x7efd240fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.813+0000 7efd291ff700 1 -- 172.21.15.89:0/2198471474 shutdown_connections 2023-10-26T06:52:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.813+0000 7efd291ff700 1 -- 172.21.15.89:0/2198471474 wait complete. 2023-10-26T06:52:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.813+0000 7efd291ff700 1 Processor -- start 2023-10-26T06:52:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd291ff700 1 -- start start 2023-10-26T06:52:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd291ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd241006a0 0x7efd24072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd291ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd24101060 0x7efd24073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd291ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd241066b0 0x7efd24073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd291ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241a4eb0 con 0x7efd24101060 2023-10-26T06:52:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd291ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241a5020 con 0x7efd241006a0 2023-10-26T06:52:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd291ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241a51e0 con 0x7efd241066b0 2023-10-26T06:52:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd241006a0 0x7efd24072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd2259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd24101060 0x7efd24073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd241006a0 0x7efd24072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51318/0 (socket says 172.21.15.89:51318) 2023-10-26T06:52:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd22d9d700 1 -- 172.21.15.89:0/3579651527 learned_addr learned my addr 172.21.15.89:0/3579651527 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.814+0000 7efd2359e700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd241066b0 0x7efd24073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd22d9d700 1 -- 172.21.15.89:0/3579651527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd241066b0 msgr2=0x7efd24073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd22d9d700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd241066b0 0x7efd24073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd22d9d700 1 -- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd24101060 msgr2=0x7efd24073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd22d9d700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd24101060 0x7efd24073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd22d9d700 1 -- 172.21.15.89:0/3579651527 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd140095f0 con 0x7efd241006a0 2023-10-26T06:52:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd2359e700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd241066b0 0x7efd24073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd22d9d700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd241006a0 0x7efd24072af0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd14005910 tx=0x7efd14004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd0bfff700 1 -- 172.21.15.89:0/3579651527 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1401d070 con 0x7efd241006a0 2023-10-26T06:52:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd241a5460 con 0x7efd241006a0 2023-10-26T06:52:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd0bfff700 1 -- 172.21.15.89:0/3579651527 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd14022470 con 0x7efd241006a0 2023-10-26T06:52:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd0bfff700 1 -- 172.21.15.89:0/3579651527 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14017650 con 0x7efd241006a0 2023-10-26T06:52:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.815+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd241a5990 con 0x7efd241006a0 2023-10-26T06:52:25.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.816+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd24108ca0 con 0x7efd241006a0 2023-10-26T06:52:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.818+0000 7efd0bfff700 1 -- 172.21.15.89:0/3579651527 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efd140225e0 con 0x7efd241006a0 2023-10-26T06:52:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.820+0000 7efd0bfff700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd0c0a15f0 0x7efd0c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.820+0000 7efd0bfff700 1 -- 172.21.15.89:0/3579651527 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7efd140ce7c0 con 0x7efd241006a0 2023-10-26T06:52:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.820+0000 7efd2259c700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd0c0a15f0 0x7efd0c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.820+0000 7efd2259c700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd0c0a15f0 0x7efd0c0a3ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efd240741d0 tx=0x7efd18009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:25.822+0000 7efd0bfff700 1 -- 172.21.15.89:0/3579651527 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd14097b50 con 0x7efd241006a0 2023-10-26T06:52:26.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.068+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd24073f80 con 0x7efd241006a0 2023-10-26T06:52:26.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.070+0000 7efd0bfff700 1 -- 172.21.15.89:0/3579651527 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7efd1409b170 con 0x7efd241006a0 2023-10-26T06:52:26.071 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:52:26.071 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:52:26.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd0c0a15f0 msgr2=0x7efd0c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd0c0a15f0 0x7efd0c0a3ab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efd240741d0 tx=0x7efd18009260 comp rx=0 tx=0).stop 2023-10-26T06:52:26.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd241006a0 msgr2=0x7efd24072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd241006a0 0x7efd24072af0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd14005910 tx=0x7efd14004730 comp rx=0 tx=0).stop 2023-10-26T06:52:26.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 shutdown_connections 2023-10-26T06:52:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efd0c0a15f0 0x7efd0c0a3ab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd241066b0 0x7efd24073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd241006a0 0x7efd24072af0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 --2- 172.21.15.89:0/3579651527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd24101060 0x7efd24073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 >> 172.21.15.89:0/3579651527 conn(0x7efd240fc170 msgr2=0x7efd240fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 shutdown_connections 2023-10-26T06:52:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.072+0000 7efd291ff700 1 -- 172.21.15.89:0/3579651527 wait complete. 2023-10-26T06:52:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:52:26.094 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:52:26.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.300+0000 7f314e1ff700 1 -- 172.21.15.89:0/1897375810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148073540 msgr2=0x7f314810ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.300+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1897375810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148073540 0x7f314810ca10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f313c009990 tx=0x7f313c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.300+0000 7f314e1ff700 1 -- 172.21.15.89:0/1897375810 shutdown_connections 2023-10-26T06:52:26.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.300+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1897375810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f314810cf50 0x7f314810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.300+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1897375810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148072c20 0x7f3148073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.300+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1897375810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148073540 0x7f314810ca10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.300+0000 7f314e1ff700 1 -- 172.21.15.89:0/1897375810 >> 172.21.15.89:0/1897375810 conn(0x7f31480fc200 msgr2=0x7f31480fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:26.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.301+0000 7f314e1ff700 1 -- 172.21.15.89:0/1897375810 shutdown_connections 2023-10-26T06:52:26.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.301+0000 7f314e1ff700 1 -- 172.21.15.89:0/1897375810 wait complete. 2023-10-26T06:52:26.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 Processor -- start 2023-10-26T06:52:26.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 -- start start 2023-10-26T06:52:26.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148072c20 0x7f3148198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148073540 0x7f3148199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f314810cf50 0x7f314819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314819dd50 con 0x7f3148072c20 2023-10-26T06:52:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314819dec0 con 0x7f3148073540 2023-10-26T06:52:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f314e1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314819e080 con 0x7f314810cf50 2023-10-26T06:52:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f3146ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148073540 0x7f3148199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f3147fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f314810cf50 0x7f314819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.302+0000 7f3146ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148073540 0x7f3148199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51338/0 (socket says 172.21.15.89:51338) 2023-10-26T06:52:26.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f3146ffd700 1 -- 172.21.15.89:0/1565304065 learned_addr learned my addr 172.21.15.89:0/1565304065 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:26.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f31477fe700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148072c20 0x7f3148198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f3146ffd700 1 -- 172.21.15.89:0/1565304065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f314810cf50 msgr2=0x7f314819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f314810cf50 0x7f314819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f3146ffd700 1 -- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148072c20 msgr2=0x7f3148198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148072c20 0x7f3148198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f3146ffd700 1 -- 172.21.15.89:0/1565304065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c0095f0 con 0x7f3148073540 2023-10-26T06:52:26.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.303+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148073540 0x7f3148199370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f313c000c00 tx=0x7f313c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.306+0000 7f3144ff9700 1 -- 172.21.15.89:0/1565304065 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c01d070 con 0x7f3148073540 2023-10-26T06:52:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.307+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314819e300 con 0x7f3148073540 2023-10-26T06:52:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.307+0000 7f3144ff9700 1 -- 172.21.15.89:0/1565304065 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313c022470 con 0x7f3148073540 2023-10-26T06:52:26.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.307+0000 7f3144ff9700 1 -- 172.21.15.89:0/1565304065 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c017650 con 0x7f3148073540 2023-10-26T06:52:26.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.307+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31481a5ba0 con 0x7f3148073540 2023-10-26T06:52:26.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.311+0000 7f3161ad4b80 1 Processor -- start 2023-10-26T06:52:26.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.311+0000 7f3161ad4b80 1 -- start start 2023-10-26T06:52:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.311+0000 7f3144ff9700 1 -- 172.21.15.89:0/1565304065 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f313c0225e0 con 0x7f3148073540 2023-10-26T06:52:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f13be0 0x55d757f13fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f0e5d0 0x55d757f0ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757e816e0 con 0x55d757f14500 2023-10-26T06:52:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757e67c60 con 0x55d757f0e5d0 2023-10-26T06:52:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757e65580 con 0x55d757f13be0 2023-10-26T06:52:26.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3146ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f0e5d0 0x55d757f0ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3146ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f0e5d0 0x55d757f0ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51352/0 (socket says 172.21.15.89:51352) 2023-10-26T06:52:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3147fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f13be0 0x55d757f13fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f3146ffd700 1 -- 172.21.15.89:0/1930984647 learned_addr learned my addr 172.21.15.89:0/1930984647 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.312+0000 7f31477fe700 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3146ffd700 1 -- 172.21.15.89:0/1930984647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f13be0 msgr2=0x55d757f13fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f13be0 0x55d757f13fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3146ffd700 1 -- 172.21.15.89:0/1930984647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 msgr2=0x55d757f0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3146ffd700 1 -- 172.21.15.89:0/1930984647 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d757e1e3f0 con 0x55d757f0e5d0 2023-10-26T06:52:26.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3147fff700 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f13be0 0x55d757f13fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:26.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f0e5d0 0x55d757f0ea50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f313c0cbc40 tx=0x7f313c0d4820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6fea001b5a51 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f312dffb700 1 -- 172.21.15.89:0/1930984647 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0ced90 con 0x55d757f0e5d0 2023-10-26T06:52:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f312dffb700 1 -- 172.21.15.89:0/1930984647 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313c0d0590 con 0x55d757f0e5d0 2023-10-26T06:52:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f312dffb700 1 -- 172.21.15.89:0/1930984647 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0da940 con 0x55d757f0e5d0 2023-10-26T06:52:26.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.313+0000 7f3144ff9700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f31300a15f0 0x7f31300a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3144ff9700 1 -- 172.21.15.89:0/1565304065 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f313c0ce5d0 con 0x7f3148073540 2023-10-26T06:52:26.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3147fff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f31300a15f0 0x7f31300a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1930984647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f0e5d0 msgr2=0x55d757f0ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f0e5d0 0x55d757f0ea50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f313c0cbc40 tx=0x7f313c0d4820 comp rx=0 tx=0).stop 2023-10-26T06:52:26.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1930984647 shutdown_connections 2023-10-26T06:52:26.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f0e5d0 0x55d757f0ea50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f13be0 0x55d757f13fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1930984647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.314+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1930984647 >> 172.21.15.89:0/1930984647 conn(0x55d757e82720 msgr2=0x55d757f08b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:26.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.315+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1930984647 shutdown_connections 2023-10-26T06:52:26.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.315+0000 7f3147fff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f31300a15f0 0x7f31300a3ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f31400075c0 tx=0x7f314000b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:26.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.315+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1930984647 wait complete. 2023-10-26T06:52:26.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.315+0000 7f3161ad4b80 1 -- start start 2023-10-26T06:52:26.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.315+0000 7f3161ad4b80 1 Processor -- start 2023-10-26T06:52:26.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f13be0 0x55d757f17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f14500 0x55d757f17d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757e60e70 con 0x55d757f13be0 2023-10-26T06:52:26.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757dfd710 con 0x55d757f14500 2023-10-26T06:52:26.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757dfd3b0 con 0x55d757f0e5d0 2023-10-26T06:52:26.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3161ad4b80 10 client.0 _ll_register_callbacks cb 0x55d757d47a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:26.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f31477fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f31477fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:49492/0 (socket says 172.21.15.89:49492) 2023-10-26T06:52:26.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f31477fe700 1 -- 172.21.15.89:0/1801495822 learned_addr learned my addr 172.21.15.89:0/1801495822 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:26.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f13be0 0x55d757f17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.316+0000 7f3147fff700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f14500 0x55d757f17d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3147fff700 1 -- 172.21.15.89:0/1801495822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 msgr2=0x55d757f1a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3147fff700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3147fff700 1 -- 172.21.15.89:0/1801495822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f13be0 msgr2=0x55d757f17760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3147fff700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f13be0 0x55d757f17760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3147fff700 1 -- 172.21.15.89:0/1801495822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3140011230 con 0x55d757f14500 2023-10-26T06:52:26.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f31477fe700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:26.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3147fff700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f14500 0x55d757f17d70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f31400045b0 tx=0x7f3140004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3adf3aad3fb63de server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:26.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3126ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:52:26.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3126ffd700 1 -- 172.21.15.89:0/1801495822 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314000e040 con 0x55d757f14500 2023-10-26T06:52:26.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d757e65ef0 con 0x55d757f14500 2023-10-26T06:52:26.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3126ffd700 1 -- 172.21.15.89:0/1801495822 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f314001c030 con 0x55d757f14500 2023-10-26T06:52:26.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3126ffd700 1 -- 172.21.15.89:0/1801495822 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3140009250 con 0x55d757f14500 2023-10-26T06:52:26.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3161ad4b80 10 client.26072 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:26.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.317+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f31481a5ba0 con 0x55d757f14500 2023-10-26T06:52:26.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3126ffd700 1 -- 172.21.15.89:0/1801495822 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d757e65ef0 con 0x55d757f14500 2023-10-26T06:52:26.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3126ffd700 1 -- 172.21.15.89:0/1801495822 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3140023c70 con 0x55d757f14500 2023-10-26T06:52:26.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3161ad4b80 10 client.26072 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:26.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3161ad4b80 10 client.26072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:26.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3161ad4b80 20 client.26072 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:26.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f27b10 0x55d757f29f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3161ad4b80 4 client.26072 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:52:26.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d757edd8b0 con 0x55d757f27b10 2023-10-26T06:52:26.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.318+0000 7f31477fe700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f27b10 0x55d757f29f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.319+0000 7f3126ffd700 10 client.26072 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:26.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.319+0000 7f31477fe700 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f27b10 0x55d757f29f30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35e8185b59bc7ea server_cookie=7e6a240adb21d618 in_seq=0 out_seq=0 2023-10-26T06:52:26.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.319+0000 7f3126ffd700 1 -- 172.21.15.89:0/1801495822 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d757edd8b0 con 0x55d757f27b10 2023-10-26T06:52:26.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.319+0000 7f3126ffd700 10 client.26072 handle_command_reply: tid=0 2023-10-26T06:52:26.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.319+0000 7f3161ad4b80 1 client.26072 shutdown 2023-10-26T06:52:26.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f14500 msgr2=0x55d757f17d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f14500 0x55d757f17d70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f31400045b0 tx=0x7f3140004b40 comp rx=0 tx=0).stop 2023-10-26T06:52:26.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 shutdown_connections 2023-10-26T06:52:26.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f27b10 0x55d757f29f30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f14500 0x55d757f17d70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1801495822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f13be0 0x55d757f17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3126ffd700 0 client.26072 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:26.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 >> 172.21.15.89:0/1801495822 conn(0x55d757e82720 msgr2=0x55d757f0d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:26.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 shutdown_connections 2023-10-26T06:52:26.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1801495822 wait complete. 2023-10-26T06:52:26.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.320+0000 7f3161ad4b80 20 client.26072 trim_cache size 0 max 16384 2023-10-26T06:52:26.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.333+0000 7f3161ad4b80 1 Processor -- start 2023-10-26T06:52:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3161ad4b80 1 -- start start 2023-10-26T06:52:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f0e5d0 0x55d757f1ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f19fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f14500 0x55d757f11640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757f03af0 con 0x55d757f0e5d0 2023-10-26T06:52:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757ce1920 con 0x55d757f13be0 2023-10-26T06:52:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757f18d60 con 0x55d757f14500 2023-10-26T06:52:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f31477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f0e5d0 0x55d757f1ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3146ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f19fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f31477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f0e5d0 0x55d757f1ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54716/0 (socket says 172.21.15.89:54716) 2023-10-26T06:52:26.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f31477fe700 1 -- 172.21.15.89:0/1462944032 learned_addr learned my addr 172.21.15.89:0/1462944032 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.334+0000 7f3147fff700 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f14500 0x55d757f11640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f3146ffd700 1 -- 172.21.15.89:0/1462944032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f14500 msgr2=0x55d757f11640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f14500 0x55d757f11640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f3146ffd700 1 -- 172.21.15.89:0/1462944032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f0e5d0 msgr2=0x55d757f1ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f0e5d0 0x55d757f1ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f3146ffd700 1 -- 172.21.15.89:0/1462944032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d757c8bde0 con 0x55d757f13be0 2023-10-26T06:52:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f31477fe700 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f0e5d0 0x55d757f1ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f3146ffd700 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f19fa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f313c097c40 tx=0x7f313c022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abab5c58cfccb85c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f312e7fc700 1 -- 172.21.15.89:0/1462944032 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0d1b70 con 0x55d757f13be0 2023-10-26T06:52:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f312e7fc700 1 -- 172.21.15.89:0/1462944032 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313c097a80 con 0x55d757f13be0 2023-10-26T06:52:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.335+0000 7f312e7fc700 1 -- 172.21.15.89:0/1462944032 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c09b510 con 0x55d757f13be0 2023-10-26T06:52:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1462944032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 msgr2=0x55d757f19fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f19fa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f313c097c40 tx=0x7f313c022c90 comp rx=0 tx=0).stop 2023-10-26T06:52:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1462944032 shutdown_connections 2023-10-26T06:52:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f0e5d0 0x55d757f1ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f14500 0x55d757f11640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/1462944032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f19fa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1462944032 >> 172.21.15.89:0/1462944032 conn(0x55d757e82720 msgr2=0x55d757f0d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1462944032 shutdown_connections 2023-10-26T06:52:26.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 -- 172.21.15.89:0/1462944032 wait complete. 2023-10-26T06:52:26.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 -- start start 2023-10-26T06:52:26.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.336+0000 7f3161ad4b80 1 Processor -- start 2023-10-26T06:52:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f1d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3161ad4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f1d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757f7a960 con 0x55d757f14500 2023-10-26T06:52:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757f7aaa0 con 0x55d757f13be0 2023-10-26T06:52:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3161ad4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d757f7ac10 con 0x55d757f0e5d0 2023-10-26T06:52:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3161ad4b80 10 client.0 _ll_register_callbacks cb 0x55d757d47a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:26.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3147fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f1d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3146ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f1d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3147fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f1d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54720/0 (socket says 172.21.15.89:54720) 2023-10-26T06:52:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f3147fff700 1 -- 172.21.15.89:0/159314277 learned_addr learned my addr 172.21.15.89:0/159314277 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.337+0000 7f31477fe700 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3147fff700 1 -- 172.21.15.89:0/159314277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 msgr2=0x55d757f1cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3147fff700 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3147fff700 1 -- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 msgr2=0x55d757f1d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3147fff700 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f1d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3147fff700 1 -- 172.21.15.89:0/159314277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3138004040 con 0x55d757f14500 2023-10-26T06:52:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3146ffd700 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3147fff700 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f1d690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3140022400 tx=0x7f3140009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6317288e7e745294 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3126ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3126ffd700 1 -- 172.21.15.89:0/159314277 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314000e040 con 0x55d757f14500 2023-10-26T06:52:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3126ffd700 1 -- 172.21.15.89:0/159314277 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f314003d070 con 0x55d757f14500 2023-10-26T06:52:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f313c0d0590 con 0x55d757f14500 2023-10-26T06:52:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3126ffd700 1 -- 172.21.15.89:0/159314277 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3140022780 con 0x55d757f14500 2023-10-26T06:52:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3161ad4b80 10 client.16041 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.338+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d757e1e3f0 con 0x55d757f14500 2023-10-26T06:52:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3126ffd700 1 -- 172.21.15.89:0/159314277 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313c0d0590 con 0x55d757f14500 2023-10-26T06:52:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3126ffd700 1 -- 172.21.15.89:0/159314277 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3140023450 con 0x55d757f14500 2023-10-26T06:52:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3161ad4b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3161ad4b80 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3161ad4b80 20 client.16041 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f7c2f0 0x55d757f7e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3161ad4b80 4 client.16041 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:52:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d757e59e50 con 0x55d757f7c2f0 2023-10-26T06:52:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.339+0000 7f31477fe700 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f7c2f0 0x55d757f7e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3126ffd700 10 client.16041 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f31477fe700 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f7c2f0 0x55d757f7e710 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f93b627970e5ea server_cookie=ad5d22ebc5cd5997 in_seq=0 out_seq=0 2023-10-26T06:52:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3126ffd700 1 -- 172.21.15.89:0/159314277 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d757e59e50 con 0x55d757f7c2f0 2023-10-26T06:52:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3126ffd700 10 client.16041 handle_command_reply: tid=0 2023-10-26T06:52:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 client.16041 shutdown 2023-10-26T06:52:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 msgr2=0x55d757f1d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f1d690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3140022400 tx=0x7f3140009920 comp rx=0 tx=0).stop 2023-10-26T06:52:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 shutdown_connections 2023-10-26T06:52:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d757f14500 0x55d757f1d690 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d757f7c2f0 0x55d757f7e710 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d757f0e5d0 0x55d757f1cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3126ffd700 0 client.16041 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 --2- 172.21.15.89:0/159314277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d757f13be0 0x55d757f1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 >> 172.21.15.89:0/159314277 conn(0x55d757e82720 msgr2=0x55d757f11640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:26.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 shutdown_connections 2023-10-26T06:52:26.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.340+0000 7f3161ad4b80 1 -- 172.21.15.89:0/159314277 wait complete. 2023-10-26T06:52:26.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f3161ad4b80 20 client.16041 trim_cache size 0 max 16384 2023-10-26T06:52:26.371 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:52:26.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f31300a15f0 msgr2=0x7f31300a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f31300a15f0 0x7f31300a3ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f31400075c0 tx=0x7f314000b040 comp rx=0 tx=0).stop 2023-10-26T06:52:26.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148073540 msgr2=0x7f3148199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:26.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148073540 0x7f3148199370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f313c000c00 tx=0x7f313c005740 comp rx=0 tx=0).stop 2023-10-26T06:52:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 shutdown_connections 2023-10-26T06:52:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f31300a15f0 0x7f31300a3ab0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3148072c20 0x7f3148198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f314810cf50 0x7f314819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 --2- 172.21.15.89:0/1565304065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148073540 0x7f3148199370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 >> 172.21.15.89:0/1565304065 conn(0x7f31480fc200 msgr2=0x7f31480fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 shutdown_connections 2023-10-26T06:52:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:26.341+0000 7f314e1ff700 1 -- 172.21.15.89:0/1565304065 wait complete. 2023-10-26T06:52:26.376 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:26 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3579651527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:52:26.376 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:26 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:52:26.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:26 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3579651527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:52:26.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:26 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:52:26.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:26 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3579651527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:52:26.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:26 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:52:27.377 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:52:27.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:52:27.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 -- 172.21.15.89:0/2470719688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec106730 msgr2=0x7fd1ec106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/2470719688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec106730 0x7fd1ec106b10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009990 tx=0x7fd1e8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 -- 172.21.15.89:0/2470719688 shutdown_connections 2023-10-26T06:52:27.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/2470719688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec106730 0x7fd1ec106b10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/2470719688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1ec1010e0 0x7fd1ec10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/2470719688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1ec100720 0x7fd1ec100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 -- 172.21.15.89:0/2470719688 >> 172.21.15.89:0/2470719688 conn(0x7fd1ec0fc1b0 msgr2=0x7fd1ec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 -- 172.21.15.89:0/2470719688 shutdown_connections 2023-10-26T06:52:27.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.582+0000 7fd1f3fff700 1 -- 172.21.15.89:0/2470719688 wait complete. 2023-10-26T06:52:27.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 Processor -- start 2023-10-26T06:52:27.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 -- start start 2023-10-26T06:52:27.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1ec100720 0x7fd1ec19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec1010e0 0x7fd1ec19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1ec106730 0x7fd1ec1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec197030 con 0x7fd1ec1010e0 2023-10-26T06:52:27.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1971a0 con 0x7fd1ec106730 2023-10-26T06:52:27.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.583+0000 7fd1f3fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec197360 con 0x7fd1ec100720 2023-10-26T06:52:27.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec1010e0 0x7fd1ec19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f1d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1ec100720 0x7fd1ec19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec1010e0 0x7fd1ec19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54730/0 (socket says 172.21.15.89:54730) 2023-10-26T06:52:27.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 -- 172.21.15.89:0/1696223593 learned_addr learned my addr 172.21.15.89:0/1696223593 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f259c700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1ec106730 0x7fd1ec1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 -- 172.21.15.89:0/1696223593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1ec100720 msgr2=0x7fd1ec19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1ec100720 0x7fd1ec19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 -- 172.21.15.89:0/1696223593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1ec106730 msgr2=0x7fd1ec1969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1ec106730 0x7fd1ec1969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 -- 172.21.15.89:0/1696223593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80095f0 con 0x7fd1ec1010e0 2023-10-26T06:52:27.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.584+0000 7fd1f159a700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec1010e0 0x7fd1ec19cde0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00d980 tx=0x7fd1dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.585+0000 7fd1e2ffd700 1 -- 172.21.15.89:0/1696223593 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc004220 con 0x7fd1ec1010e0 2023-10-26T06:52:27.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.585+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec197640 con 0x7fd1ec1010e0 2023-10-26T06:52:27.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.585+0000 7fd1e2ffd700 1 -- 172.21.15.89:0/1696223593 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1dc007640 con 0x7fd1ec1010e0 2023-10-26T06:52:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.585+0000 7fd1e2ffd700 1 -- 172.21.15.89:0/1696223593 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc005690 con 0x7fd1ec1010e0 2023-10-26T06:52:27.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.585+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec1a3890 con 0x7fd1ec1010e0 2023-10-26T06:52:27.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.588+0000 7fd1e2ffd700 1 -- 172.21.15.89:0/1696223593 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd1dc004750 con 0x7fd1ec1010e0 2023-10-26T06:52:27.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 Processor -- start 2023-10-26T06:52:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd1e2ffd700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd1d80a15f0 0x7fd1d80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 -- start start 2023-10-26T06:52:27.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd1e2ffd700 1 -- 172.21.15.89:0/1696223593 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd1dc0cce90 con 0x7fd1ec1010e0 2023-10-26T06:52:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd1f259c700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd1d80a15f0 0x7fd1d80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580b880 0x560f45806520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f45806a60 0x560f45806ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f45775760 con 0x560f45806a60 2023-10-26T06:52:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4575bce0 con 0x560f4580af60 2023-10-26T06:52:27.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.589+0000 7fd207963b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f45759600 con 0x560f4580b880 2023-10-26T06:52:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f45806a60 0x560f45806ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51388/0 (socket says 172.21.15.89:51388) 2023-10-26T06:52:27.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 -- 172.21.15.89:0/958949467 learned_addr learned my addr 172.21.15.89:0/958949467 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f159a700 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580b880 0x560f45806520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 -- 172.21.15.89:0/958949467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580b880 msgr2=0x560f45806520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580b880 0x560f45806520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 -- 172.21.15.89:0/958949467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f45806a60 msgr2=0x560f45806ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f45806a60 0x560f45806ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 -- 172.21.15.89:0/958949467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f45712470 con 0x560f4580af60 2023-10-26T06:52:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f159a700 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580b880 0x560f45806520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:27.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580b340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8005f30 tx=0x7fd1e8009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a2a233fc05d093 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.590+0000 7fd1f259c700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd1d80a15f0 0x7fd1d80a3ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4004150 tx=0x7fd1e4003e10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd1d7fff700 1 -- 172.21.15.89:0/958949467 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8005a00 con 0x560f4580af60 2023-10-26T06:52:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd1d7fff700 1 -- 172.21.15.89:0/958949467 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e801e040 con 0x560f4580af60 2023-10-26T06:52:27.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd1d7fff700 1 -- 172.21.15.89:0/958949467 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80039f0 con 0x560f4580af60 2023-10-26T06:52:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 -- 172.21.15.89:0/958949467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 msgr2=0x560f4580b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580b340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8005f30 tx=0x7fd1e8009550 comp rx=0 tx=0).stop 2023-10-26T06:52:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 -- 172.21.15.89:0/958949467 shutdown_connections 2023-10-26T06:52:27.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f45806a60 0x560f45806ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580b340 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 --2- 172.21.15.89:0/958949467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580b880 0x560f45806520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 -- 172.21.15.89:0/958949467 >> 172.21.15.89:0/958949467 conn(0x560f457767a0 msgr2=0x560f457fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 -- 172.21.15.89:0/958949467 shutdown_connections 2023-10-26T06:52:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.591+0000 7fd207963b80 1 -- 172.21.15.89:0/958949467 wait complete. 2023-10-26T06:52:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.592+0000 7fd207963b80 1 -- start start 2023-10-26T06:52:27.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.592+0000 7fd207963b80 1 Processor -- start 2023-10-26T06:52:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd207963b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45812910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd207963b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd207963b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45810340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd207963b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f45754ef0 con 0x560f4580b880 2023-10-26T06:52:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd207963b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f456f1790 con 0x560f4580af60 2023-10-26T06:52:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd207963b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f456f1430 con 0x560f45806a60 2023-10-26T06:52:27.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd207963b80 10 client.0 _ll_register_callbacks cb 0x560f4563bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45810340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f1d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45812910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45810340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54740/0 (socket says 172.21.15.89:54740) 2023-10-26T06:52:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f259c700 1 -- 172.21.15.89:0/3156972203 learned_addr learned my addr 172.21.15.89:0/3156972203 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f159a700 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f1d9b700 1 -- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 msgr2=0x560f4580fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f1d9b700 1 -- 172.21.15.89:0/3156972203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 msgr2=0x560f45810340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45810340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f1d9b700 1 -- 172.21.15.89:0/3156972203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec10def0 con 0x560f45806a60 2023-10-26T06:52:27.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.593+0000 7fd1f259c700 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45810340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:27.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45812910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8025040 tx=0x7fd1e800bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee656cb04ff38331 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd1d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:52:27.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd1d57fa700 1 -- 172.21.15.89:0/3156972203 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80059a0 con 0x560f45806a60 2023-10-26T06:52:27.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f45759f70 con 0x560f45806a60 2023-10-26T06:52:27.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd1d57fa700 1 -- 172.21.15.89:0/3156972203 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd1e8003790 con 0x560f45806a60 2023-10-26T06:52:27.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd1d57fa700 1 -- 172.21.15.89:0/3156972203 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1e8003950 con 0x560f45806a60 2023-10-26T06:52:27.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd207963b80 10 client.26104 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:27.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560f4557fe60 con 0x560f45806a60 2023-10-26T06:52:27.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.594+0000 7fd1d57fa700 1 -- 172.21.15.89:0/3156972203 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f45759f70 con 0x560f45806a60 2023-10-26T06:52:27.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd1d57fa700 1 -- 172.21.15.89:0/3156972203 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd1e802b990 con 0x560f45806a60 2023-10-26T06:52:27.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd207963b80 10 client.26104 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:27.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd207963b80 10 client.26104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:27.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd207963b80 20 client.26104 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:27.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd207963b80 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f4581fdd0 0x560f458221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd207963b80 4 client.26104 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:52:27.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f457d1930 con 0x560f4581fdd0 2023-10-26T06:52:27.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd1f159a700 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f4581fdd0 0x560f458221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd1d57fa700 10 client.26104 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.595+0000 7fd1f159a700 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f4581fdd0 0x560f458221c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b37e79cf9afae8c server_cookie=539ea6785b0d24b4 in_seq=0 out_seq=0 2023-10-26T06:52:27.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.596+0000 7fd1d57fa700 1 -- 172.21.15.89:0/3156972203 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f457d1930 con 0x560f4581fdd0 2023-10-26T06:52:27.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.596+0000 7fd1d57fa700 10 client.26104 handle_command_reply: tid=0 2023-10-26T06:52:27.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.596+0000 7fd207963b80 1 client.26104 shutdown 2023-10-26T06:52:27.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 msgr2=0x560f45812910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45812910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8025040 tx=0x7fd1e800bc20 comp rx=0 tx=0).stop 2023-10-26T06:52:27.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 shutdown_connections 2023-10-26T06:52:27.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45810340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f4581fdd0 0x560f458221c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45812910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 --2- 172.21.15.89:0/3156972203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f4580fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd1d57fa700 0 client.26104 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 >> 172.21.15.89:0/3156972203 conn(0x560f457767a0 msgr2=0x560f457fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 shutdown_connections 2023-10-26T06:52:27.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 1 -- 172.21.15.89:0/3156972203 wait complete. 2023-10-26T06:52:27.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.597+0000 7fd207963b80 20 client.26104 trim_cache size 0 max 16384 2023-10-26T06:52:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.614+0000 7fd207963b80 1 Processor -- start 2023-10-26T06:52:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.614+0000 7fd207963b80 1 -- start start 2023-10-26T06:52:27.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd207963b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f45806a60 0x560f458172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd207963b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580af60 0x560f4580e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd207963b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f4580ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd207963b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f455d59a0 con 0x560f4580b880 2023-10-26T06:52:27.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd207963b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f457febf0 con 0x560f45806a60 2023-10-26T06:52:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd207963b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4580ab80 con 0x560f4580af60 2023-10-26T06:52:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f4580ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f4580ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54750/0 (socket says 172.21.15.89:54750) 2023-10-26T06:52:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 -- 172.21.15.89:0/3406247412 learned_addr learned my addr 172.21.15.89:0/3406247412 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f159a700 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580af60 0x560f4580e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f45806a60 0x560f458172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 -- 172.21.15.89:0/3406247412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580af60 msgr2=0x560f4580e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580af60 0x560f4580e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 -- 172.21.15.89:0/3406247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f45806a60 msgr2=0x560f458172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f45806a60 0x560f458172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.615+0000 7fd1f259c700 1 -- 172.21.15.89:0/3406247412 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f457fcdf0 con 0x560f4580b880 2023-10-26T06:52:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.616+0000 7fd1f259c700 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f4580ea30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007b00 tx=0x7fd1e401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ebc69206c5de6e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.616+0000 7fd1e0ff9700 1 -- 172.21.15.89:0/3406247412 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e402a070 con 0x560f4580b880 2023-10-26T06:52:27.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.616+0000 7fd1e0ff9700 1 -- 172.21.15.89:0/3406247412 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e400f040 con 0x560f4580b880 2023-10-26T06:52:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.616+0000 7fd1e0ff9700 1 -- 172.21.15.89:0/3406247412 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40266c0 con 0x560f4580b880 2023-10-26T06:52:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.616+0000 7fd207963b80 1 -- 172.21.15.89:0/3406247412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 msgr2=0x560f4580ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.616+0000 7fd207963b80 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f4580ea30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007b00 tx=0x7fd1e401c000 comp rx=0 tx=0).stop 2023-10-26T06:52:27.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 -- 172.21.15.89:0/3406247412 shutdown_connections 2023-10-26T06:52:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f4580ea30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f45806a60 0x560f458172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 --2- 172.21.15.89:0/3406247412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f4580af60 0x560f4580e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 -- 172.21.15.89:0/3406247412 >> 172.21.15.89:0/3406247412 conn(0x560f457767a0 msgr2=0x560f4580bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 -- 172.21.15.89:0/3406247412 shutdown_connections 2023-10-26T06:52:27.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 -- 172.21.15.89:0/3406247412 wait complete. 2023-10-26T06:52:27.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 -- start start 2023-10-26T06:52:27.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.617+0000 7fd207963b80 1 Processor -- start 2023-10-26T06:52:27.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd207963b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45816de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd207963b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f458087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd207963b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45808ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd207963b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f458176c0 con 0x560f4580b880 2023-10-26T06:52:27.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd207963b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f45809220 con 0x560f4580af60 2023-10-26T06:52:27.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd207963b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f458093e0 con 0x560f45806a60 2023-10-26T06:52:27.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd207963b80 10 client.0 _ll_register_callbacks cb 0x560f4563bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:27.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45808ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45808ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54762/0 (socket says 172.21.15.89:54762) 2023-10-26T06:52:27.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f1d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45816de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 -- 172.21.15.89:0/2047501381 learned_addr learned my addr 172.21.15.89:0/2047501381 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f159a700 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f458087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 -- 172.21.15.89:0/2047501381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 msgr2=0x560f45816de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45816de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 -- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 msgr2=0x560f458087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f458087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f259c700 1 -- 172.21.15.89:0/2047501381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e4007810 con 0x560f4580b880 2023-10-26T06:52:27.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.618+0000 7fd1f159a700 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f458087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:27.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd1f259c700 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45808ce0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd1e401b730 tx=0x7fd1e401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7bf45c3de05929 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd1d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:27.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd1d57fa700 1 -- 172.21.15.89:0/2047501381 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e402a070 con 0x560f4580b880 2023-10-26T06:52:27.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1e801e040 con 0x560f4580b880 2023-10-26T06:52:27.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd1d57fa700 1 -- 172.21.15.89:0/2047501381 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fd1e4021050 con 0x560f4580b880 2023-10-26T06:52:27.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd1d57fa700 1 -- 172.21.15.89:0/2047501381 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1e4007e10 con 0x560f4580b880 2023-10-26T06:52:27.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd207963b80 10 client.16059 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:27.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560f45568ce0 con 0x560f4580b880 2023-10-26T06:52:27.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd1d57fa700 1 -- 172.21.15.89:0/2047501381 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e801e040 con 0x560f4580b880 2023-10-26T06:52:27.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.619+0000 7fd1d57fa700 1 -- 172.21.15.89:0/2047501381 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd1e4030990 con 0x560f4580b880 2023-10-26T06:52:27.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd207963b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:27.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd207963b80 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:27.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd207963b80 20 client.16059 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:27.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd207963b80 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f45875fa0 0x560f458783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd207963b80 4 client.16059 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:52:27.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560f4574ded0 con 0x560f45875fa0 2023-10-26T06:52:27.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f45875fa0 0x560f458783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd1d57fa700 10 client.16059 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.620+0000 7fd1f1d9b700 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f45875fa0 0x560f458783c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd5854a42f0c903 server_cookie=2a15e66e5f3fd5e9 in_seq=0 out_seq=0 2023-10-26T06:52:27.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.621+0000 7fd1d57fa700 1 -- 172.21.15.89:0/2047501381 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560f4574ded0 con 0x560f45875fa0 2023-10-26T06:52:27.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd1d57fa700 10 client.16059 handle_command_reply: tid=0 2023-10-26T06:52:27.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 client.16059 shutdown 2023-10-26T06:52:27.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 msgr2=0x560f45808ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45808ce0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd1e401b730 tx=0x7fd1e401bf90 comp rx=0 tx=0).stop 2023-10-26T06:52:27.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 shutdown_connections 2023-10-26T06:52:27.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f4580b880 0x560f45808ce0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f45875fa0 0x560f458783c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f45806a60 0x560f45816de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 --2- 172.21.15.89:0/2047501381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f4580af60 0x560f458087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 >> 172.21.15.89:0/2047501381 conn(0x560f457767a0 msgr2=0x560f457f6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd1d57fa700 0 client.16059 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 shutdown_connections 2023-10-26T06:52:27.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 1 -- 172.21.15.89:0/2047501381 wait complete. 2023-10-26T06:52:27.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.622+0000 7fd207963b80 20 client.16059 trim_cache size 0 max 16384 2023-10-26T06:52:27.653 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:52:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:52:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "36c12265-c381-4edc-80db-499fed9f061f", 2023-10-26T06:52:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:52:27.654 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:52:27.655 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:27 smithi192 ceph-mon[131661]: pgmap v476: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:27.655 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:52:27.656 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:52:27.656 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:52:27.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd1d80a15f0 msgr2=0x7fd1d80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd1d80a15f0 0x7fd1d80a3ab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4004150 tx=0x7fd1e4003e10 comp rx=0 tx=0).stop 2023-10-26T06:52:27.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec1010e0 msgr2=0x7fd1ec19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec1010e0 0x7fd1ec19cde0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00d980 tx=0x7fd1dc00dc90 comp rx=0 tx=0).stop 2023-10-26T06:52:27.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 shutdown_connections 2023-10-26T06:52:27.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd1d80a15f0 0x7fd1d80a3ab0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1ec106730 0x7fd1ec1969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1ec100720 0x7fd1ec19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 --2- 172.21.15.89:0/1696223593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec1010e0 0x7fd1ec19cde0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 >> 172.21.15.89:0/1696223593 conn(0x7fd1ec0fc1b0 msgr2=0x7fd1ec0fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 shutdown_connections 2023-10-26T06:52:27.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.623+0000 7fd1f3fff700 1 -- 172.21.15.89:0/1696223593 wait complete. 2023-10-26T06:52:27.660 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:27 smithi089 ceph-mon[95031]: pgmap v476: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:27.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:52:27.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:52:27.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:52:27.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:52:27.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:27 smithi135 ceph-mon[131403]: pgmap v476: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:27.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:52:27.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:52:27.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:52:27.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3821493850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac1010e0 msgr2=0x7f5cac10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3821493850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac1010e0 0x7f5cac10d5f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b0a0 tx=0x7f5ca400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3821493850 shutdown_connections 2023-10-26T06:52:27.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3821493850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac1010e0 0x7f5cac10d5f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3821493850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cac106730 0x7f5cac106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3821493850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac100720 0x7f5cac100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3821493850 >> 172.21.15.89:0/3821493850 conn(0x7f5cac0fc1b0 msgr2=0x7f5cac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3821493850 shutdown_connections 2023-10-26T06:52:27.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.863+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3821493850 wait complete. 2023-10-26T06:52:27.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.864+0000 7f5cb33ff700 1 Processor -- start 2023-10-26T06:52:27.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.864+0000 7f5cb33ff700 1 -- start start 2023-10-26T06:52:27.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.864+0000 7f5cb33ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cac100720 0x7f5cac19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.864+0000 7f5cb33ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac1010e0 0x7f5cac19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb33ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac106730 0x7f5cac198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb33ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac199190 con 0x7f5cac106730 2023-10-26T06:52:27.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb33ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac199300 con 0x7f5cac100720 2023-10-26T06:52:27.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb33ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac1994c0 con 0x7f5cac1010e0 2023-10-26T06:52:27.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac106730 0x7f5cac198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac106730 0x7f5cac198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54786/0 (socket says 172.21.15.89:54786) 2023-10-26T06:52:27.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 -- 172.21.15.89:0/3186714201 learned_addr learned my addr 172.21.15.89:0/3186714201 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb119b700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cac100720 0x7f5cac19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb099a700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac1010e0 0x7f5cac19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 -- 172.21.15.89:0/3186714201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac1010e0 msgr2=0x7f5cac19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac1010e0 0x7f5cac19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 -- 172.21.15.89:0/3186714201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cac100720 msgr2=0x7f5cac19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cac100720 0x7f5cac19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 -- 172.21.15.89:0/3186714201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca4009d00 con 0x7f5cac106730 2023-10-26T06:52:27.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb119b700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cac100720 0x7f5cac19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.865+0000 7f5cb199c700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac106730 0x7f5cac198b30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b070 tx=0x7f5ca40077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.866+0000 7f5ca27fc700 1 -- 172.21.15.89:0/3186714201 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400e040 con 0x7f5cac106730 2023-10-26T06:52:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.866+0000 7f5ca27fc700 1 -- 172.21.15.89:0/3186714201 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ca40094c0 con 0x7f5cac106730 2023-10-26T06:52:27.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.866+0000 7f5ca27fc700 1 -- 172.21.15.89:0/3186714201 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40043a0 con 0x7f5cac106730 2023-10-26T06:52:27.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.866+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac199740 con 0x7f5cac106730 2023-10-26T06:52:27.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.866+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac1a5a40 con 0x7f5cac106730 2023-10-26T06:52:27.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.869+0000 7f5ca27fc700 1 -- 172.21.15.89:0/3186714201 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5ca4004500 con 0x7f5cac106730 2023-10-26T06:52:27.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.870+0000 7f5ca27fc700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5c980a1670 0x7f5c980a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.870+0000 7f5cc6dc4b80 1 Processor -- start 2023-10-26T06:52:27.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.870+0000 7f5ca27fc700 1 -- 172.21.15.89:0/3186714201 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f5ca40cdf00 con 0x7f5cac106730 2023-10-26T06:52:27.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.870+0000 7f5cb119b700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5c980a1670 0x7f5c980a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.870+0000 7f5cc6dc4b80 1 -- start start 2023-10-26T06:52:27.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd68fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bd63090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd63a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bcd66e0 con 0x55d06bd69500 2023-10-26T06:52:27.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bcbcc60 con 0x55d06bd635d0 2023-10-26T06:52:27.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bcba580 con 0x55d06bd68be0 2023-10-26T06:52:27.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cb119b700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5c980a1670 0x7f5c980a3b30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009fd0 tx=0x7f5ca8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cb099a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bd63090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cb119b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd68fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cb099a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bd63090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54792/0 (socket says 172.21.15.89:54792) 2023-10-26T06:52:27.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cb099a700 1 -- 172.21.15.89:0/427551969 learned_addr learned my addr 172.21.15.89:0/427551969 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.871+0000 7f5cb199c700 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd63a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5cb099a700 1 -- 172.21.15.89:0/427551969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 msgr2=0x55d06bd68fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5cb099a700 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd68fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5cb099a700 1 -- 172.21.15.89:0/427551969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 msgr2=0x55d06bd63a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5cb099a700 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd63a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5cb099a700 1 -- 172.21.15.89:0/427551969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d06bc733f0 con 0x55d06bd69500 2023-10-26T06:52:27.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5cb099a700 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bd63090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d20 tx=0x7f5c9c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e676283014bbec68 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5c977fe700 1 -- 172.21.15.89:0/427551969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c019630 con 0x55d06bd69500 2023-10-26T06:52:27.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5c977fe700 1 -- 172.21.15.89:0/427551969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9c019c50 con 0x55d06bd69500 2023-10-26T06:52:27.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.872+0000 7f5c977fe700 1 -- 172.21.15.89:0/427551969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00ebe0 con 0x55d06bd69500 2023-10-26T06:52:27.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/427551969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 msgr2=0x55d06bd63090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bd63090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d20 tx=0x7f5c9c011720 comp rx=0 tx=0).stop 2023-10-26T06:52:27.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/427551969 shutdown_connections 2023-10-26T06:52:27.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd63a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd68fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/427551969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bd63090 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/427551969 >> 172.21.15.89:0/427551969 conn(0x55d06bcd7720 msgr2=0x55d06bd5db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/427551969 shutdown_connections 2023-10-26T06:52:27.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.873+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/427551969 wait complete. 2023-10-26T06:52:27.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 -- start start 2023-10-26T06:52:27.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 Processor -- start 2023-10-26T06:52:27.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd635d0 0x55d06bd6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd6c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd69500 0x55d06bd6cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bcb5e70 con 0x55d06bd68be0 2023-10-26T06:52:27.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bc52710 con 0x55d06bd69500 2023-10-26T06:52:27.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bc523b0 con 0x55d06bd635d0 2023-10-26T06:52:27.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cc6dc4b80 10 client.0 _ll_register_callbacks cb 0x55d06bb9ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:27.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cb099a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd6c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.874+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd69500 0x55d06bd6cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd69500 0x55d06bd6cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:51450/0 (socket says 172.21.15.89:51450) 2023-10-26T06:52:27.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb199c700 1 -- 172.21.15.89:0/3726555511 learned_addr learned my addr 172.21.15.89:0/3726555511 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb119b700 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd635d0 0x55d06bd6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb099a700 1 -- 172.21.15.89:0/3726555511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd635d0 msgr2=0x55d06bd6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb099a700 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd635d0 0x55d06bd6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb099a700 1 -- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd69500 msgr2=0x55d06bd6cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb099a700 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd69500 0x55d06bd6cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb099a700 1 -- 172.21.15.89:0/3726555511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c009980 con 0x55d06bd68be0 2023-10-26T06:52:27.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb199c700 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd69500 0x55d06bd6cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:27.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cb099a700 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd6c720 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55d06bd5ea70 tx=0x7f5c9c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a844290604b4a786 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5c94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:27.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5c94ff9700 1 -- 172.21.15.89:0/3726555511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c004220 con 0x55d06bd68be0 2023-10-26T06:52:27.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5c94ff9700 1 -- 172.21.15.89:0/3726555511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f5c9c019dc0 con 0x55d06bd68be0 2023-10-26T06:52:27.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.875+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d06bcbaef0 con 0x55d06bd68be0 2023-10-26T06:52:27.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5c94ff9700 1 -- 172.21.15.89:0/3726555511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9c005ce0 con 0x55d06bd68be0 2023-10-26T06:52:27.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 10 client.16071 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:27.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d06bae0de0 con 0x55d06bd68be0 2023-10-26T06:52:27.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5c94ff9700 1 -- 172.21.15.89:0/3726555511 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d06bcbaef0 con 0x55d06bd68be0 2023-10-26T06:52:27.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5c94ff9700 1 -- 172.21.15.89:0/3726555511 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5c9c029070 con 0x55d06bd68be0 2023-10-26T06:52:27.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:27.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:27.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 20 client.16071 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:27.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bd7cce0 0x55d06bd7f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 4 client.16071 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:52:27.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.876+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d06bd328b0 con 0x55d06bd7cce0 2023-10-26T06:52:27.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.877+0000 7f5cb119b700 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bd7cce0 0x55d06bd7f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.877+0000 7f5cb119b700 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bd7cce0 0x55d06bd7f100 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31829177a02ee920 server_cookie=d69c343bf7e539f8 in_seq=0 out_seq=0 2023-10-26T06:52:27.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.877+0000 7f5c94ff9700 10 client.16071 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5c94ff9700 1 -- 172.21.15.89:0/3726555511 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d06bd328b0 con 0x55d06bd7cce0 2023-10-26T06:52:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5c94ff9700 10 client.16071 handle_command_reply: tid=0 2023-10-26T06:52:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 client.16071 shutdown 2023-10-26T06:52:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 msgr2=0x55d06bd6c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd6c720 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55d06bd5ea70 tx=0x7f5c9c0098e0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 shutdown_connections 2023-10-26T06:52:27.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bd7cce0 0x55d06bd7f100 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd69500 0x55d06bd6cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd635d0 0x55d06bd6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/3726555511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd6c720 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5c94ff9700 0 client.16071 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 >> 172.21.15.89:0/3726555511 conn(0x55d06bcd7720 msgr2=0x55d06bd62080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 shutdown_connections 2023-10-26T06:52:27.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/3726555511 wait complete. 2023-10-26T06:52:27.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.878+0000 7f5cc6dc4b80 20 client.16071 trim_cache size 0 max 16384 2023-10-26T06:52:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.895+0000 7f5cc6dc4b80 1 Processor -- start 2023-10-26T06:52:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.895+0000 7f5cc6dc4b80 1 -- start start 2023-10-26T06:52:27.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd73eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd66720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd69500 0x55d06bd66c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bd58af0 con 0x55d06bd68be0 2023-10-26T06:52:27.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bb36920 con 0x55d06bd635d0 2023-10-26T06:52:27.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bd79990 con 0x55d06bd69500 2023-10-26T06:52:27.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd66720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd66720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54816/0 (socket says 172.21.15.89:54816) 2023-10-26T06:52:27.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 -- 172.21.15.89:0/119445291 learned_addr learned my addr 172.21.15.89:0/119445291 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb119b700 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd73eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb199c700 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd69500 0x55d06bd66c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 -- 172.21.15.89:0/119445291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd69500 msgr2=0x55d06bd66c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd69500 0x55d06bd66c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 -- 172.21.15.89:0/119445291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 msgr2=0x55d06bd73eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd73eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb099a700 1 -- 172.21.15.89:0/119445291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d06bac9c60 con 0x55d06bd68be0 2023-10-26T06:52:27.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.896+0000 7f5cb119b700 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd73eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:27.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cb099a700 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd66720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d20 tx=0x7f5c9c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33467f95e5ab5b41 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5c97fff700 1 -- 172.21.15.89:0/119445291 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c005ce0 con 0x55d06bd68be0 2023-10-26T06:52:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5c97fff700 1 -- 172.21.15.89:0/119445291 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9c00ede0 con 0x55d06bd68be0 2023-10-26T06:52:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5c97fff700 1 -- 172.21.15.89:0/119445291 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c003b80 con 0x55d06bd68be0 2023-10-26T06:52:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/119445291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 msgr2=0x55d06bd66720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd66720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d20 tx=0x7f5c9c011720 comp rx=0 tx=0).stop 2023-10-26T06:52:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/119445291 shutdown_connections 2023-10-26T06:52:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd69500 0x55d06bd66c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd73eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/119445291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd68be0 0x55d06bd66720 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/119445291 >> 172.21.15.89:0/119445291 conn(0x55d06bcd7720 msgr2=0x55d06bd6aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.897+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/119445291 shutdown_connections 2023-10-26T06:52:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.898+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/119445291 wait complete. 2023-10-26T06:52:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.898+0000 7f5cc6dc4b80 1 -- start start 2023-10-26T06:52:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.898+0000 7f5cc6dc4b80 1 Processor -- start 2023-10-26T06:52:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd67820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd67d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cc6dc4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bdcfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bd68650 con 0x55d06bd69500 2023-10-26T06:52:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bdd0050 con 0x55d06bd635d0 2023-10-26T06:52:27.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cc6dc4b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d06bdd0190 con 0x55d06bd68be0 2023-10-26T06:52:27.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cc6dc4b80 10 client.0 _ll_register_callbacks cb 0x55d06bb9ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:27.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bdcfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb199c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bdcfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54822/0 (socket says 172.21.15.89:54822) 2023-10-26T06:52:27.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb199c700 1 -- 172.21.15.89:0/122626847 learned_addr learned my addr 172.21.15.89:0/122626847 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:27.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb099a700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd67d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb119b700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd67820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb199c700 1 -- 172.21.15.89:0/122626847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 msgr2=0x55d06bd67d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb199c700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd67d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb199c700 1 -- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 msgr2=0x55d06bd67820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:27.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.899+0000 7f5cb199c700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd67820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5cb199c700 1 -- 172.21.15.89:0/122626847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d06bae0de0 con 0x55d06bd69500 2023-10-26T06:52:27.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5cb119b700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd67820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:27.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5cb199c700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bdcfb10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40979c0 tx=0x7f5ca409a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5394b70c42a9ab72 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5c94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5c94ff9700 1 -- 172.21.15.89:0/122626847 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca409b650 con 0x55d06bd69500 2023-10-26T06:52:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d06bd64040 con 0x55d06bd69500 2023-10-26T06:52:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5c94ff9700 1 -- 172.21.15.89:0/122626847 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f5ca409af50 con 0x55d06bd69500 2023-10-26T06:52:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5c94ff9700 1 -- 172.21.15.89:0/122626847 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ca409f0a0 con 0x55d06bd69500 2023-10-26T06:52:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5cc6dc4b80 10 client.16077 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d06bb07250 con 0x55d06bd69500 2023-10-26T06:52:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5c94ff9700 1 -- 172.21.15.89:0/122626847 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ca407d080 con 0x55d06bd69500 2023-10-26T06:52:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5c94ff9700 1 -- 172.21.15.89:0/122626847 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ca409b240 con 0x55d06bd69500 2023-10-26T06:52:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.900+0000 7f5cc6dc4b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5cc6dc4b80 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5cc6dc4b80 20 client.16077 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bdd17f0 0x55d06bdd3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5cc6dc4b80 4 client.16077 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:52:27.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f5c9c019c50 con 0x55d06bdd17f0 2023-10-26T06:52:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5cb119b700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bdd17f0 0x55d06bdd3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5cb119b700 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bdd17f0 0x55d06bdd3c10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f6bdc2686f72fb server_cookie=3a241369c45d9994 in_seq=0 out_seq=0 2023-10-26T06:52:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.901+0000 7f5c94ff9700 10 client.16077 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5c94ff9700 1 -- 172.21.15.89:0/122626847 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f5c9c019c50 con 0x55d06bdd17f0 2023-10-26T06:52:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5c94ff9700 10 client.16077 handle_command_reply: tid=0 2023-10-26T06:52:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 client.16077 shutdown 2023-10-26T06:52:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 msgr2=0x55d06bdcfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bdcfb10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40979c0 tx=0x7f5ca409a960 comp rx=0 tx=0).stop 2023-10-26T06:52:27.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 shutdown_connections 2023-10-26T06:52:27.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d06bd69500 0x55d06bdcfb10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d06bdd17f0 0x55d06bdd3c10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06bd635d0 0x55d06bd67820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 --2- 172.21.15.89:0/122626847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d06bd68be0 0x55d06bd67d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5c94ff9700 0 client.16077 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:27.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 >> 172.21.15.89:0/122626847 conn(0x55d06bcd7720 msgr2=0x55d06bd6af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 shutdown_connections 2023-10-26T06:52:27.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 1 -- 172.21.15.89:0/122626847 wait complete. 2023-10-26T06:52:27.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.902+0000 7f5cc6dc4b80 20 client.16077 trim_cache size 0 max 16384 2023-10-26T06:52:27.934 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:52:27.934 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:52:27.934 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:52:27.935 INFO:teuthology.orchestra.run.smithi089.stdout: "36c12265-c381-4edc-80db-499fed9f061f": { 2023-10-26T06:52:27.935 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:52:27.935 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "36c12265-c381-4edc-80db-499fed9f061f", 2023-10-26T06:52:27.936 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:52:27.936 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:52:27.936 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:52:27.936 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:52:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5c980a1670 msgr2=0x7f5c980a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5c980a1670 0x7f5c980a3b30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009fd0 tx=0x7f5ca8009260 comp rx=0 tx=0).stop 2023-10-26T06:52:27.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac106730 msgr2=0x7f5cac198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac106730 0x7f5cac198b30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b070 tx=0x7f5ca40077f0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 shutdown_connections 2023-10-26T06:52:27.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5c980a1670 0x7f5c980a3b30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5cac106730 0x7f5cac198b30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cac100720 0x7f5cac19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 --2- 172.21.15.89:0/3186714201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac1010e0 0x7f5cac19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 >> 172.21.15.89:0/3186714201 conn(0x7f5cac0fc1b0 msgr2=0x7f5cac0fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 shutdown_connections 2023-10-26T06:52:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:27.903+0000 7f5cb33ff700 1 -- 172.21.15.89:0/3186714201 wait complete. 2023-10-26T06:52:27.941 INFO:tasks.cephfs.filesystem:scrub status for tag:36c12265-c381-4edc-80db-499fed9f061f - {'path': '/', 'tag': '36c12265-c381-4edc-80db-499fed9f061f', 'options': 'recursive,force'} 2023-10-26T06:52:28.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:28 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:52:28.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:28 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:52:28.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:28 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:52:28.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:28 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:52:28.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:28 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:52:28.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:28 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:52:28.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:28 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:52:28.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:28 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:52:28.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:28 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:52:28.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:28 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:52:28.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:28 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:52:28.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:28 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:52:29.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:29 smithi192 ceph-mon[131661]: pgmap v477: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:29.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:29 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:52:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:29 smithi135 ceph-mon[131403]: pgmap v477: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:29 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:52:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:29 smithi089 ceph-mon[95031]: pgmap v477: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:29 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:52:31.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:31 smithi192 ceph-mon[131661]: pgmap v478: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:52:31.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:31 smithi135 ceph-mon[131403]: pgmap v478: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:52:31.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:31 smithi089 ceph-mon[95031]: pgmap v478: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:52:33.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:33 smithi192 ceph-mon[131661]: pgmap v479: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:52:33.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:33 smithi135 ceph-mon[131403]: pgmap v479: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:52:33.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:33 smithi089 ceph-mon[95031]: pgmap v479: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:52:35.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:35 smithi192 ceph-mon[131661]: pgmap v480: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:52:35.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:35 smithi135 ceph-mon[131403]: pgmap v480: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:52:35.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:35 smithi089 ceph-mon[95031]: pgmap v480: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:52:37.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:37 smithi192 ceph-mon[131661]: pgmap v481: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:52:37.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:37 smithi135 ceph-mon[131403]: pgmap v481: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:52:37.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:37 smithi089 ceph-mon[95031]: pgmap v481: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:52:39.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:39 smithi192 ceph-mon[131661]: pgmap v482: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:52:39.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:39 smithi135 ceph-mon[131403]: pgmap v482: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:52:39.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:39 smithi089 ceph-mon[95031]: pgmap v482: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:52:41.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:41 smithi192 ceph-mon[131661]: pgmap v483: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:52:41.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:41 smithi135 ceph-mon[131403]: pgmap v483: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:52:41.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:41 smithi089 ceph-mon[95031]: pgmap v483: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:52:43.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:43 smithi192 ceph-mon[131661]: pgmap v484: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:52:43.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:43 smithi135 ceph-mon[131403]: pgmap v484: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:52:43.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:43 smithi089 ceph-mon[95031]: pgmap v484: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:52:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:45 smithi192 ceph-mon[131661]: pgmap v485: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:52:45.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:45 smithi135 ceph-mon[131403]: pgmap v485: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:52:45.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:45 smithi089 ceph-mon[95031]: pgmap v485: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:52:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:47 smithi192 ceph-mon[131661]: pgmap v486: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:47.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:47 smithi135 ceph-mon[131403]: pgmap v486: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:47.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:47 smithi089 ceph-mon[95031]: pgmap v486: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:49 smithi192 ceph-mon[131661]: pgmap v487: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:49.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:49 smithi135 ceph-mon[131403]: pgmap v487: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:49.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:49 smithi089 ceph-mon[95031]: pgmap v487: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:51 smithi192 ceph-mon[131661]: pgmap v488: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:51 smithi135 ceph-mon[131403]: pgmap v488: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:51.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:51 smithi089 ceph-mon[95031]: pgmap v488: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:53 smithi192 ceph-mon[131661]: pgmap v489: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:53.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:53 smithi135 ceph-mon[131403]: pgmap v489: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:53 smithi089 ceph-mon[95031]: pgmap v489: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:55 smithi192 ceph-mon[131661]: pgmap v490: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:52:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:55 smithi135 ceph-mon[131403]: pgmap v490: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:52:55.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:55 smithi089 ceph-mon[95031]: pgmap v490: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:52:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:57 smithi192 ceph-mon[131661]: pgmap v491: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:57 smithi135 ceph-mon[131403]: pgmap v491: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:57.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:57 smithi089 ceph-mon[95031]: pgmap v491: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:52:57.943 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:52:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 -- 172.21.15.89:0/4285488880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76141010d0 msgr2=0x7f761410d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 --2- 172.21.15.89:0/4285488880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76141010d0 0x7f761410d5e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b0a0 tx=0x7f760c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 -- 172.21.15.89:0/4285488880 shutdown_connections 2023-10-26T06:52:58.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 --2- 172.21.15.89:0/4285488880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76141010d0 0x7f761410d5e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 --2- 172.21.15.89:0/4285488880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7614106720 0x7f7614106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 --2- 172.21.15.89:0/4285488880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7614100710 0x7f7614100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 -- 172.21.15.89:0/4285488880 >> 172.21.15.89:0/4285488880 conn(0x7f76140fc200 msgr2=0x7f76140fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 -- 172.21.15.89:0/4285488880 shutdown_connections 2023-10-26T06:52:58.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.152+0000 7f761b3ff700 1 -- 172.21.15.89:0/4285488880 wait complete. 2023-10-26T06:52:58.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.153+0000 7f761b3ff700 1 Processor -- start 2023-10-26T06:52:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.153+0000 7f761b3ff700 1 -- start start 2023-10-26T06:52:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761b3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7614100710 0x7f761419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761b3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76141010d0 0x7f761419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761b3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7614106720 0x7f7614196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761b3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7614196fc0 con 0x7f7614106720 2023-10-26T06:52:58.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761b3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7614197130 con 0x7f7614100710 2023-10-26T06:52:58.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761b3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76141972f0 con 0x7f76141010d0 2023-10-26T06:52:58.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761999c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7614106720 0x7f7614196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761999c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7614106720 0x7f7614196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37462/0 (socket says 172.21.15.89:37462) 2023-10-26T06:52:58.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761999c700 1 -- 172.21.15.89:0/1388122331 learned_addr learned my addr 172.21.15.89:0/1388122331 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761919b700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7614100710 0x7f761419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.154+0000 7f761899a700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76141010d0 0x7f761419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761999c700 1 -- 172.21.15.89:0/1388122331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76141010d0 msgr2=0x7f761419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761999c700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76141010d0 0x7f761419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761999c700 1 -- 172.21.15.89:0/1388122331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7614100710 msgr2=0x7f761419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761999c700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7614100710 0x7f761419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761999c700 1 -- 172.21.15.89:0/1388122331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c009d00 con 0x7f7614106720 2023-10-26T06:52:58.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761999c700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7614106720 0x7f7614196960 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b070 tx=0x7f760c008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f760a7fc700 1 -- 172.21.15.89:0/1388122331 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c00e040 con 0x7f7614106720 2023-10-26T06:52:58.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f760a7fc700 1 -- 172.21.15.89:0/1388122331 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f760c01d470 con 0x7f7614106720 2023-10-26T06:52:58.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f760a7fc700 1 -- 172.21.15.89:0/1388122331 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c0052a0 con 0x7f7614106720 2023-10-26T06:52:58.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7614197570 con 0x7f7614106720 2023-10-26T06:52:58.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.155+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76141a37d0 con 0x7f7614106720 2023-10-26T06:52:58.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.158+0000 7f760a7fc700 1 -- 172.21.15.89:0/1388122331 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f760c01d5e0 con 0x7f7614106720 2023-10-26T06:52:58.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.159+0000 7f762ec34b80 1 Processor -- start 2023-10-26T06:52:58.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.159+0000 7f760a7fc700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f76000a1670 0x7f76000a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f762ec34b80 1 -- start start 2023-10-26T06:52:58.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f760a7fc700 1 -- 172.21.15.89:0/1388122331 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f760c0ce340 con 0x7f7614106720 2023-10-26T06:52:58.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f761899a700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f76000a1670 0x7f76000a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f762ec34b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7402b6e0 con 0x55da740be500 2023-10-26T06:52:58.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f762ec34b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da74011c60 con 0x55da740b85d0 2023-10-26T06:52:58.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f762ec34b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7400f580 con 0x55da740bdbe0 2023-10-26T06:52:58.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f761899a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f761919b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f761899a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740b8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37478/0 (socket says 172.21.15.89:37478) 2023-10-26T06:52:58.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f761899a700 1 -- 172.21.15.89:0/1167898802 learned_addr learned my addr 172.21.15.89:0/1167898802 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f761919b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740bdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:52698/0 (socket says 172.21.15.89:52698) 2023-10-26T06:52:58.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.160+0000 7f761999c700 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f761919b700 1 -- 172.21.15.89:0/1167898802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 msgr2=0x55da740b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f761919b700 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f761919b700 1 -- 172.21.15.89:0/1167898802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 msgr2=0x55da740b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f761919b700 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f761899a700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f76000a1670 0x7f76000a3b30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7604012500 tx=0x7f7604007f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f761919b700 1 -- 172.21.15.89:0/1167898802 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da73fc83f0 con 0x55da740bdbe0 2023-10-26T06:52:58.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f761919b700 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740bdfc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7610009cb0 tx=0x7f76100058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91703774f6d3478c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f75ff7fe700 1 -- 172.21.15.89:0/1167898802 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011c60 con 0x55da740bdbe0 2023-10-26T06:52:58.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f75ff7fe700 1 -- 172.21.15.89:0/1167898802 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7610004510 con 0x55da740bdbe0 2023-10-26T06:52:58.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.161+0000 7f75ff7fe700 1 -- 172.21.15.89:0/1167898802 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610005280 con 0x55da740bdbe0 2023-10-26T06:52:58.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 -- 172.21.15.89:0/1167898802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 msgr2=0x55da740bdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740bdfc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7610009cb0 tx=0x7f76100058e0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 -- 172.21.15.89:0/1167898802 shutdown_connections 2023-10-26T06:52:58.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740bdfc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1167898802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 -- 172.21.15.89:0/1167898802 >> 172.21.15.89:0/1167898802 conn(0x55da7402c720 msgr2=0x55da740b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 -- 172.21.15.89:0/1167898802 shutdown_connections 2023-10-26T06:52:58.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.162+0000 7f762ec34b80 1 -- 172.21.15.89:0/1167898802 wait complete. 2023-10-26T06:52:58.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 -- start start 2023-10-26T06:52:58.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 Processor -- start 2023-10-26T06:52:58.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7400ae70 con 0x55da740be500 2023-10-26T06:52:58.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73fa7710 con 0x55da740b85d0 2023-10-26T06:52:58.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73fa73b0 con 0x55da740bdbe0 2023-10-26T06:52:58.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.163+0000 7f762ec34b80 10 client.0 _ll_register_callbacks cb 0x55da73ef1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:58.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740c1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37488/0 (socket says 172.21.15.89:37488) 2023-10-26T06:52:58.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 -- 172.21.15.89:0/1666351569 learned_addr learned my addr 172.21.15.89:0/1666351569 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761899a700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761919b700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 -- 172.21.15.89:0/1666351569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 msgr2=0x55da740c1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740c1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 -- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 msgr2=0x55da740c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 -- 172.21.15.89:0/1666351569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76100098e0 con 0x55da740be500 2023-10-26T06:52:58.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761919b700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:58.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f761999c700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740c1c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f760c097110 tx=0x7f760c09bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddeeb1134bd5ba2 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f75fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:58.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f75fcff9700 1 -- 172.21.15.89:0/1666351569 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c0b1030 con 0x55da740be500 2023-10-26T06:52:58.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.164+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da7400fef0 con 0x55da740be500 2023-10-26T06:52:58.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f75fcff9700 1 -- 172.21.15.89:0/1666351569 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f760c0b2070 con 0x55da740be500 2023-10-26T06:52:58.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f75fcff9700 1 -- 172.21.15.89:0/1666351569 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f760c0a3690 con 0x55da740be500 2023-10-26T06:52:58.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 10 client.16089 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:58.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55da73e35de0 con 0x55da740be500 2023-10-26T06:52:58.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f75fcff9700 1 -- 172.21.15.89:0/1666351569 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da7400fef0 con 0x55da740be500 2023-10-26T06:52:58.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f75fcff9700 1 -- 172.21.15.89:0/1666351569 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f760c09b710 con 0x55da740be500 2023-10-26T06:52:58.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:58.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:58.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 20 client.16089 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:58.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da740d5d70 0x55da740d8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 4 client.16089 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:52:58.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.165+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da740878b0 con 0x55da740d5d70 2023-10-26T06:52:58.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.166+0000 7f761919b700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da740d5d70 0x55da740d8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.166+0000 7f761919b700 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da740d5d70 0x55da740d8190 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726c579104224087 server_cookie=b30bd654d01c386b in_seq=0 out_seq=0 2023-10-26T06:52:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.166+0000 7f75fcff9700 10 client.16089 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f75fcff9700 1 -- 172.21.15.89:0/1666351569 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da740878b0 con 0x55da740d5d70 2023-10-26T06:52:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f75fcff9700 10 client.16089 handle_command_reply: tid=0 2023-10-26T06:52:58.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 client.16089 shutdown 2023-10-26T06:52:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 msgr2=0x55da740c1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740c1c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f760c097110 tx=0x7f760c09bb80 comp rx=0 tx=0).stop 2023-10-26T06:52:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 shutdown_connections 2023-10-26T06:52:58.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740be500 0x55da740c1c60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da740d5d70 0x55da740d8190 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 --2- 172.21.15.89:0/1666351569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f75fcff9700 0 client.16089 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 >> 172.21.15.89:0/1666351569 conn(0x55da7402c720 msgr2=0x55da740b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 shutdown_connections 2023-10-26T06:52:58.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 1 -- 172.21.15.89:0/1666351569 wait complete. 2023-10-26T06:52:58.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.167+0000 7f762ec34b80 20 client.16089 trim_cache size 0 max 16384 2023-10-26T06:52:58.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.184+0000 7f762ec34b80 1 Processor -- start 2023-10-26T06:52:58.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.184+0000 7f762ec34b80 1 -- start start 2023-10-26T06:52:58.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740bdbe0 0x55da740cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740be500 0x55da740b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f762ec34b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73e8b920 con 0x55da740bdbe0 2023-10-26T06:52:58.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f762ec34b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da740c1470 con 0x55da740b85d0 2023-10-26T06:52:58.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f762ec34b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da740b43e0 con 0x55da740be500 2023-10-26T06:52:58.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740bdbe0 0x55da740cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740bdbe0 0x55da740cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37494/0 (socket says 172.21.15.89:37494) 2023-10-26T06:52:58.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 -- 172.21.15.89:0/3068408003 learned_addr learned my addr 172.21.15.89:0/3068408003 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761919b700 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761999c700 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740be500 0x55da740b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 -- 172.21.15.89:0/3068408003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740be500 msgr2=0x55da740b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740be500 0x55da740b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 -- 172.21.15.89:0/3068408003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 msgr2=0x55da740c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761899a700 1 -- 172.21.15.89:0/3068408003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da73e1ec60 con 0x55da740bdbe0 2023-10-26T06:52:58.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.185+0000 7f761919b700 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:58.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.186+0000 7f761899a700 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740bdbe0 0x55da740cd340 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7604012c50 tx=0x7f760401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad17da333dccf2d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.186+0000 7f75fffff700 1 -- 172.21.15.89:0/3068408003 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604024070 con 0x55da740bdbe0 2023-10-26T06:52:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.186+0000 7f75fffff700 1 -- 172.21.15.89:0/3068408003 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7604003680 con 0x55da740bdbe0 2023-10-26T06:52:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.186+0000 7f75fffff700 1 -- 172.21.15.89:0/3068408003 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604017040 con 0x55da740bdbe0 2023-10-26T06:52:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.186+0000 7f762ec34b80 1 -- 172.21.15.89:0/3068408003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740bdbe0 msgr2=0x55da740cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.186+0000 7f762ec34b80 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740bdbe0 0x55da740cd340 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7604012c50 tx=0x7f760401d000 comp rx=0 tx=0).stop 2023-10-26T06:52:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 -- 172.21.15.89:0/3068408003 shutdown_connections 2023-10-26T06:52:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740be500 0x55da740b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740b85d0 0x55da740c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 --2- 172.21.15.89:0/3068408003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740bdbe0 0x55da740cd340 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 -- 172.21.15.89:0/3068408003 >> 172.21.15.89:0/3068408003 conn(0x55da7402c720 msgr2=0x55da740b89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 -- 172.21.15.89:0/3068408003 shutdown_connections 2023-10-26T06:52:58.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 -- 172.21.15.89:0/3068408003 wait complete. 2023-10-26T06:52:58.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 -- start start 2023-10-26T06:52:58.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.187+0000 7f762ec34b80 1 Processor -- start 2023-10-26T06:52:58.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740b85d0 0x55da740b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f762ec34b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740be500 0x55da740cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f762ec34b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da740ccdf0 con 0x55da740b85d0 2023-10-26T06:52:58.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f762ec34b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da740ccf30 con 0x55da740be500 2023-10-26T06:52:58.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f762ec34b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da740cd0c0 con 0x55da740bdbe0 2023-10-26T06:52:58.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f762ec34b80 10 client.0 _ll_register_callbacks cb 0x55da73ef1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:58.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740b85d0 0x55da740b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740b85d0 0x55da740b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37504/0 (socket says 172.21.15.89:37504) 2023-10-26T06:52:58.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 -- 172.21.15.89:0/887768315 learned_addr learned my addr 172.21.15.89:0/887768315 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761899a700 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761999c700 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740be500 0x55da740cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 -- 172.21.15.89:0/887768315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 msgr2=0x55da740cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 -- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740be500 msgr2=0x55da740cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740be500 0x55da740cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.188+0000 7f761919b700 1 -- 172.21.15.89:0/887768315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761001d040 con 0x55da740b85d0 2023-10-26T06:52:58.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f761919b700 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740b85d0 0x55da740b77c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7610009cb0 tx=0x7f7610009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f262401ad7382391 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f75fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:58.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f75fcff9700 1 -- 172.21.15.89:0/887768315 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610004150 con 0x55da740b85d0 2023-10-26T06:52:58.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f75fcff9700 1 -- 172.21.15.89:0/887768315 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f76100042b0 con 0x55da740b85d0 2023-10-26T06:52:58.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7610004510 con 0x55da740b85d0 2023-10-26T06:52:58.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f75fcff9700 1 -- 172.21.15.89:0/887768315 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7610026a90 con 0x55da740b85d0 2023-10-26T06:52:58.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f762ec34b80 10 client.16095 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:58.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f760401f500 con 0x55da740b85d0 2023-10-26T06:52:58.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f75fcff9700 1 -- 172.21.15.89:0/887768315 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7610004510 con 0x55da740b85d0 2023-10-26T06:52:58.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.189+0000 7f75fcff9700 1 -- 172.21.15.89:0/887768315 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f76100261b0 con 0x55da740b85d0 2023-10-26T06:52:58.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f762ec34b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:58.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f762ec34b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:58.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f762ec34b80 20 client.16095 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:58.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f762ec34b80 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da7412b090 0x55da7412d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f762ec34b80 4 client.16095 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:52:58.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da74003e50 con 0x55da7412b090 2023-10-26T06:52:58.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f761899a700 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da7412b090 0x55da7412d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f761899a700 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da7412b090 0x55da7412d4b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8d58081ee994fd server_cookie=59f54ae6063cbb5b in_seq=0 out_seq=0 2023-10-26T06:52:58.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.190+0000 7f75fcff9700 10 client.16095 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f75fcff9700 1 -- 172.21.15.89:0/887768315 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da74003e50 con 0x55da7412b090 2023-10-26T06:52:58.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f75fcff9700 10 client.16095 handle_command_reply: tid=0 2023-10-26T06:52:58.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 client.16095 shutdown 2023-10-26T06:52:58.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740b85d0 msgr2=0x55da740b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740b85d0 0x55da740b77c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7610009cb0 tx=0x7f7610009230 comp rx=0 tx=0).stop 2023-10-26T06:52:58.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 shutdown_connections 2023-10-26T06:52:58.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55da740b85d0 0x55da740b77c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55da7412b090 0x55da7412d4b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55da740be500 0x55da740cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 --2- 172.21.15.89:0/887768315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da740bdbe0 0x55da740cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f75fcff9700 0 client.16095 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 >> 172.21.15.89:0/887768315 conn(0x55da7402c720 msgr2=0x55da740b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 shutdown_connections 2023-10-26T06:52:58.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 1 -- 172.21.15.89:0/887768315 wait complete. 2023-10-26T06:52:58.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.191+0000 7f762ec34b80 20 client.16095 trim_cache size 0 max 16384 2023-10-26T06:52:58.224 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:52:58.224 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:52:58.224 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:52:58.225 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:52:58.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f76000a1670 msgr2=0x7f76000a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f76000a1670 0x7f76000a3b30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7604012500 tx=0x7f7604007f90 comp rx=0 tx=0).stop 2023-10-26T06:52:58.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7614106720 msgr2=0x7f7614196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7614106720 0x7f7614196960 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f760c00b070 tx=0x7f760c008ff0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 shutdown_connections 2023-10-26T06:52:58.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f76000a1670 0x7f76000a3b30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7614106720 0x7f7614196960 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7614100710 0x7f761419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 --2- 172.21.15.89:0/1388122331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76141010d0 0x7f761419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 >> 172.21.15.89:0/1388122331 conn(0x7f76140fc200 msgr2=0x7f76140fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 shutdown_connections 2023-10-26T06:52:58.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.192+0000 7f761b3ff700 1 -- 172.21.15.89:0/1388122331 wait complete. 2023-10-26T06:52:58.230 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:52:58.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:52:58.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 -- 172.21.15.89:0/799611467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc101060 msgr2=0x7fbefc10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 --2- 172.21.15.89:0/799611467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc101060 0x7fbefc10d570 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b0a0 tx=0x7fbef400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 -- 172.21.15.89:0/799611467 shutdown_connections 2023-10-26T06:52:58.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 --2- 172.21.15.89:0/799611467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc101060 0x7fbefc10d570 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 --2- 172.21.15.89:0/799611467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbefc1066b0 0x7fbefc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 --2- 172.21.15.89:0/799611467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbefc1006a0 0x7fbefc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 -- 172.21.15.89:0/799611467 >> 172.21.15.89:0/799611467 conn(0x7fbefc0fc170 msgr2=0x7fbefc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 -- 172.21.15.89:0/799611467 shutdown_connections 2023-10-26T06:52:58.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.428+0000 7fbf041ff700 1 -- 172.21.15.89:0/799611467 wait complete. 2023-10-26T06:52:58.434 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 Processor -- start 2023-10-26T06:52:58.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 -- start start 2023-10-26T06:52:58.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc1006a0 0x7fbefc19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbefc101060 0x7fbefc19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbefc1066b0 0x7fbefc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc1992a0 con 0x7fbefc1006a0 2023-10-26T06:52:58.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc199410 con 0x7fbefc1066b0 2023-10-26T06:52:58.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.429+0000 7fbf041ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc1995d0 con 0x7fbefc101060 2023-10-26T06:52:58.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc1006a0 0x7fbefc19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc1006a0 0x7fbefc19ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37524/0 (socket says 172.21.15.89:37524) 2023-10-26T06:52:58.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 -- 172.21.15.89:0/806061368 learned_addr learned my addr 172.21.15.89:0/806061368 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf0179a700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbefc101060 0x7fbefc19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf0279c700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbefc1066b0 0x7fbefc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 -- 172.21.15.89:0/806061368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbefc101060 msgr2=0x7fbefc19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbefc101060 0x7fbefc19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 -- 172.21.15.89:0/806061368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbefc1066b0 msgr2=0x7fbefc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbefc1066b0 0x7fbefc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 -- 172.21.15.89:0/806061368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4009d00 con 0x7fbefc1006a0 2023-10-26T06:52:58.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf0179a700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbefc101060 0x7fbefc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:58.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf0279c700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbefc1066b0 0x7fbefc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:58.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbf01f9b700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc1006a0 0x7fbefc19ea30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbef800e9a0 tx=0x7fbef800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.430+0000 7fbef2ffd700 1 -- 172.21.15.89:0/806061368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8009790 con 0x7fbefc1006a0 2023-10-26T06:52:58.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.431+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc1998b0 con 0x7fbefc1006a0 2023-10-26T06:52:58.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.431+0000 7fbef2ffd700 1 -- 172.21.15.89:0/806061368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef800cbd0 con 0x7fbefc1006a0 2023-10-26T06:52:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.431+0000 7fbef2ffd700 1 -- 172.21.15.89:0/806061368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8018680 con 0x7fbefc1006a0 2023-10-26T06:52:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.431+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc1a5a00 con 0x7fbefc1006a0 2023-10-26T06:52:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.432+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbefc101de0 con 0x7fbefc1006a0 2023-10-26T06:52:58.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.434+0000 7fbef2ffd700 1 -- 172.21.15.89:0/806061368 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbef800cd40 con 0x7fbefc1006a0 2023-10-26T06:52:58.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.435+0000 7fbef2ffd700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbee80a15f0 0x7fbee80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.435+0000 7fbef2ffd700 1 -- 172.21.15.89:0/806061368 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fbef8014070 con 0x7fbefc1006a0 2023-10-26T06:52:58.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.435+0000 7fbf0179a700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbee80a15f0 0x7fbee80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.436+0000 7fbf0179a700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbee80a15f0 0x7fbee80a3ab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbefc19a240 tx=0x7fbeec009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.437+0000 7fbef2ffd700 1 -- 172.21.15.89:0/806061368 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbef8096b80 con 0x7fbefc1006a0 2023-10-26T06:52:58.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.684+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbefc067530 con 0x7fbefc1006a0 2023-10-26T06:52:58.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.684+0000 7fbef2ffd700 1 -- 172.21.15.89:0/806061368 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fbef809a1a0 con 0x7fbefc1006a0 2023-10-26T06:52:58.686 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:52:58.686 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:52:58.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbee80a15f0 msgr2=0x7fbee80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbee80a15f0 0x7fbee80a3ab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbefc19a240 tx=0x7fbeec009260 comp rx=0 tx=0).stop 2023-10-26T06:52:58.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc1006a0 msgr2=0x7fbefc19ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc1006a0 0x7fbefc19ea30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbef800e9a0 tx=0x7fbef800ed60 comp rx=0 tx=0).stop 2023-10-26T06:52:58.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 shutdown_connections 2023-10-26T06:52:58.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbee80a15f0 0x7fbee80a3ab0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbefc1006a0 0x7fbefc19ea30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbefc1066b0 0x7fbefc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 --2- 172.21.15.89:0/806061368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbefc101060 0x7fbefc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 >> 172.21.15.89:0/806061368 conn(0x7fbefc0fc170 msgr2=0x7fbefc10bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 shutdown_connections 2023-10-26T06:52:58.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.687+0000 7fbf041ff700 1 -- 172.21.15.89:0/806061368 wait complete. 2023-10-26T06:52:58.692 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:52:58.707 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:52:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 -- 172.21.15.89:0/1945180828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90106720 msgr2=0x7f4f90106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 --2- 172.21.15.89:0/1945180828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90106720 0x7f4f90106b00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009990 tx=0x7f4f80009ca0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 -- 172.21.15.89:0/1945180828 shutdown_connections 2023-10-26T06:52:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 --2- 172.21.15.89:0/1945180828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90106720 0x7f4f90106b00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 --2- 172.21.15.89:0/1945180828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f901010d0 0x7f4f9010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 --2- 172.21.15.89:0/1945180828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f90100710 0x7f4f90100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 -- 172.21.15.89:0/1945180828 >> 172.21.15.89:0/1945180828 conn(0x7f4f900fc200 msgr2=0x7f4f900fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 -- 172.21.15.89:0/1945180828 shutdown_connections 2023-10-26T06:52:58.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.907+0000 7f4f95bff700 1 -- 172.21.15.89:0/1945180828 wait complete. 2023-10-26T06:52:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.908+0000 7f4f95bff700 1 Processor -- start 2023-10-26T06:52:58.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.908+0000 7f4f95bff700 1 -- start start 2023-10-26T06:52:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f95bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90100710 0x7f4f9019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f95bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f901010d0 0x7f4f9019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f95bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f90106720 0x7f4f901969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f95bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90197020 con 0x7f4f90100710 2023-10-26T06:52:58.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f95bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90197190 con 0x7f4f90106720 2023-10-26T06:52:58.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f95bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90197350 con 0x7f4f901010d0 2023-10-26T06:52:58.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90100710 0x7f4f9019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90100710 0x7f4f9019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37550/0 (socket says 172.21.15.89:37550) 2023-10-26T06:52:58.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/4250523532 learned_addr learned my addr 172.21.15.89:0/4250523532 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8effd700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f901010d0 0x7f4f9019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f90106720 0x7f4f901969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/4250523532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f901010d0 msgr2=0x7f4f9019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f901010d0 0x7f4f9019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/4250523532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f90106720 msgr2=0x7f4f901969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f90106720 0x7f4f901969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/4250523532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800095f0 con 0x7f4f90100710 2023-10-26T06:52:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.909+0000 7f4f8effd700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f901010d0 0x7f4f9019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.910+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90100710 0x7f4f9019c890 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4f80000c00 tx=0x7f4f800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.910+0000 7f4f8cff9700 1 -- 172.21.15.89:0/4250523532 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8001d070 con 0x7f4f90100710 2023-10-26T06:52:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.910+0000 7f4f8cff9700 1 -- 172.21.15.89:0/4250523532 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f80022470 con 0x7f4f90100710 2023-10-26T06:52:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.910+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f901975d0 con 0x7f4f90100710 2023-10-26T06:52:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.910+0000 7f4f8cff9700 1 -- 172.21.15.89:0/4250523532 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80017650 con 0x7f4f90100710 2023-10-26T06:52:58.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.910+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f901a3880 con 0x7f4f90100710 2023-10-26T06:52:58.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.913+0000 7f4f8cff9700 1 -- 172.21.15.89:0/4250523532 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4f800225e0 con 0x7f4f90100710 2023-10-26T06:52:58.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.914+0000 7f4fa959db80 1 Processor -- start 2023-10-26T06:52:58.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.914+0000 7f4f8cff9700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f780a15f0 0x7f4f780a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.914+0000 7f4fa959db80 1 -- start start 2023-10-26T06:52:58.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.914+0000 7f4f8cff9700 1 -- 172.21.15.89:0/4250523532 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4f800ce540 con 0x7f4f90100710 2023-10-26T06:52:58.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f780a15f0 0x7f4f780a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e891e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4fa959db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e8006e0 con 0x55cb7e8919e0 2023-10-26T06:52:58.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4fa959db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e7e6c60 con 0x55cb7e896ff0 2023-10-26T06:52:58.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4fa959db80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e7e4580 con 0x55cb7e897910 2023-10-26T06:52:58.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:40692/0 (socket says 172.21.15.89:40692) 2023-10-26T06:52:58.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2080505456 learned_addr learned my addr 172.21.15.89:0/2080505456 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e891e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8effd700 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f780a15f0 0x7f4f780a3ab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009c60 tx=0x7f4f880077e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2080505456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 msgr2=0x55cb7e8914a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.915+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8914a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2080505456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 msgr2=0x55cb7e891e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e891e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2080505456 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb7e79d3f0 con 0x55cb7e896ff0 2023-10-26T06:52:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e891e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:52:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8973d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4f80098000 tx=0x7f4f80022cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdd6b9ca615e56d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f75ffb700 1 -- 172.21.15.89:0/2080505456 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8009b590 con 0x55cb7e896ff0 2023-10-26T06:52:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f75ffb700 1 -- 172.21.15.89:0/2080505456 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8009f6e0 con 0x55cb7e896ff0 2023-10-26T06:52:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.916+0000 7f4f75ffb700 1 -- 172.21.15.89:0/2080505456 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8009aa80 con 0x55cb7e896ff0 2023-10-26T06:52:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 -- 172.21.15.89:0/2080505456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 msgr2=0x55cb7e8973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8973d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4f80098000 tx=0x7f4f80022cb0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 -- 172.21.15.89:0/2080505456 shutdown_connections 2023-10-26T06:52:58.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e891e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8973d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2080505456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 -- 172.21.15.89:0/2080505456 >> 172.21.15.89:0/2080505456 conn(0x55cb7e801720 msgr2=0x55cb7e887b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 -- 172.21.15.89:0/2080505456 shutdown_connections 2023-10-26T06:52:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 -- 172.21.15.89:0/2080505456 wait complete. 2023-10-26T06:52:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.917+0000 7f4fa959db80 1 -- start start 2023-10-26T06:52:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 1 Processor -- start 2023-10-26T06:52:58.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e89d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e896ff0 0x55cb7e89abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e897910 0x55cb7e89b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e7dfe70 con 0x55cb7e8919e0 2023-10-26T06:52:58.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e77c710 con 0x55cb7e897910 2023-10-26T06:52:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e77c3b0 con 0x55cb7e896ff0 2023-10-26T06:52:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4fa959db80 10 client.0 _ll_register_callbacks cb 0x55cb7e6c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.918+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e89d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e89d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37574/0 (socket says 172.21.15.89:37574) 2023-10-26T06:52:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1622046078 learned_addr learned my addr 172.21.15.89:0/1622046078 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8effd700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e896ff0 0x55cb7e89abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e897910 0x55cb7e89b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1622046078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e896ff0 msgr2=0x55cb7e89abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e896ff0 0x55cb7e89abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e897910 msgr2=0x55cb7e89b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e897910 0x55cb7e89b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1622046078 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f88009d40 con 0x55cb7e8919e0 2023-10-26T06:52:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e897910 0x55cb7e89b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:52:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e89d960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000f740 tx=0x7f4f800cbbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc4dfe96f811be server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f6effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:58.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f6effd700 1 -- 172.21.15.89:0/1622046078 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8009b590 con 0x55cb7e8919e0 2023-10-26T06:52:58.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f6effd700 1 -- 172.21.15.89:0/1622046078 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4f8009ac90 con 0x55cb7e8919e0 2023-10-26T06:52:58.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb7e7e4ef0 con 0x55cb7e8919e0 2023-10-26T06:52:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.919+0000 7f4f6effd700 1 -- 172.21.15.89:0/1622046078 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f8009b7f0 con 0x55cb7e8919e0 2023-10-26T06:52:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 10 client.16113 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f4f901a3880 con 0x55cb7e8919e0 2023-10-26T06:52:58.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4f6effd700 1 -- 172.21.15.89:0/1622046078 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb7e7e4ef0 con 0x55cb7e8919e0 2023-10-26T06:52:58.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4f6effd700 1 -- 172.21.15.89:0/1622046078 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4f800ce1c0 con 0x55cb7e8919e0 2023-10-26T06:52:58.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 10 client.16113 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:58.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:58.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 20 client.16113 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:58.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8aaab0 0x55cb7e8acea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 4 client.16113 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:52:58.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb7e85c8b0 con 0x55cb7e8aaab0 2023-10-26T06:52:58.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.920+0000 7f4f8effd700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8aaab0 0x55cb7e8acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.921+0000 7f4f6effd700 10 client.16113 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.921+0000 7f4f8effd700 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8aaab0 0x55cb7e8acea0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8262980a2477a50 server_cookie=df116cfae5d166db in_seq=0 out_seq=0 2023-10-26T06:52:58.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4f6effd700 1 -- 172.21.15.89:0/1622046078 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cb7e85c8b0 con 0x55cb7e8aaab0 2023-10-26T06:52:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4f6effd700 10 client.16113 handle_command_reply: tid=0 2023-10-26T06:52:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 client.16113 shutdown 2023-10-26T06:52:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 msgr2=0x55cb7e89d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e89d960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000f740 tx=0x7f4f800cbbe0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 shutdown_connections 2023-10-26T06:52:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e89d960 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8aaab0 0x55cb7e8acea0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e897910 0x55cb7e89b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1622046078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e896ff0 0x55cb7e89abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4f6effd700 0 client.16113 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 >> 172.21.15.89:0/1622046078 conn(0x55cb7e801720 msgr2=0x55cb7e890490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 shutdown_connections 2023-10-26T06:52:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 1 -- 172.21.15.89:0/1622046078 wait complete. 2023-10-26T06:52:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.922+0000 7f4fa959db80 20 client.16113 trim_cache size 0 max 16384 2023-10-26T06:52:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.935+0000 7f4fa959db80 1 Processor -- start 2023-10-26T06:52:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4fa959db80 1 -- start start 2023-10-26T06:52:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e8919e0 0x55cb7e8951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e897910 0x55cb7e89d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4fa959db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e660920 con 0x55cb7e897910 2023-10-26T06:52:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4fa959db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e884230 con 0x55cb7e896ff0 2023-10-26T06:52:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4fa959db80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e8893b0 con 0x55cb7e8919e0 2023-10-26T06:52:58.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e8919e0 0x55cb7e8951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4f8ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e897910 0x55cb7e89d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e8919e0 0x55cb7e8951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:52786/0 (socket says 172.21.15.89:52786) 2023-10-26T06:52:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1774523246 learned_addr learned my addr 172.21.15.89:0/1774523246 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.936+0000 7f4f8effd700 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1774523246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 msgr2=0x55cb7e8a2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1774523246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e897910 msgr2=0x55cb7e89d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e897910 0x55cb7e89d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/1774523246 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb7e60ade0 con 0x55cb7e8919e0 2023-10-26T06:52:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e8919e0 0x55cb7e8951d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f80098000 tx=0x7f4f80022cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f40a8449ecc54a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f767fc700 1 -- 172.21.15.89:0/1774523246 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8009b0d0 con 0x55cb7e8919e0 2023-10-26T06:52:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f767fc700 1 -- 172.21.15.89:0/1774523246 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8009b590 con 0x55cb7e8919e0 2023-10-26T06:52:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.937+0000 7f4f767fc700 1 -- 172.21.15.89:0/1774523246 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f800a3620 con 0x55cb7e8919e0 2023-10-26T06:52:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 -- 172.21.15.89:0/1774523246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e8919e0 msgr2=0x55cb7e8951d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e8919e0 0x55cb7e8951d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f80098000 tx=0x7f4f80022cb0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 -- 172.21.15.89:0/1774523246 shutdown_connections 2023-10-26T06:52:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e897910 0x55cb7e89d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e8919e0 0x55cb7e8951d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 --2- 172.21.15.89:0/1774523246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 -- 172.21.15.89:0/1774523246 >> 172.21.15.89:0/1774523246 conn(0x55cb7e801720 msgr2=0x55cb7e894ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 -- 172.21.15.89:0/1774523246 shutdown_connections 2023-10-26T06:52:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 -- 172.21.15.89:0/1774523246 wait complete. 2023-10-26T06:52:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 -- start start 2023-10-26T06:52:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.938+0000 7f4fa959db80 1 Processor -- start 2023-10-26T06:52:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e8a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4fa959db80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4fa959db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e8a1240 con 0x55cb7e8919e0 2023-10-26T06:52:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4fa959db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e8a13b0 con 0x55cb7e896ff0 2023-10-26T06:52:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4fa959db80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7e8fe880 con 0x55cb7e897910 2023-10-26T06:52:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4fa959db80 10 client.0 _ll_register_callbacks cb 0x55cb7e6c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:52:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e8a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e8a2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37590/0 (socket says 172.21.15.89:37590) 2023-10-26T06:52:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2654200268 learned_addr learned my addr 172.21.15.89:0/2654200268 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:52:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8effd700 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8ffff700 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2654200268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 msgr2=0x55cb7e8a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 msgr2=0x55cb7e8a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:52:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.939+0000 7f4f8f7fe700 1 -- 172.21.15.89:0/2654200268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f901a3880 con 0x55cb7e8919e0 2023-10-26T06:52:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4f8f7fe700 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e8a2060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4f800a3a50 tx=0x7f4f80097230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c47768b9e52fbaa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:52:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4f6effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:52:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4f6effd700 1 -- 172.21.15.89:0/2654200268 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8009b0d0 con 0x55cb7e8919e0 2023-10-26T06:52:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4f6effd700 1 -- 172.21.15.89:0/2654200268 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4f80097640 con 0x55cb7e8919e0 2023-10-26T06:52:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f8009f6e0 con 0x55cb7e8919e0 2023-10-26T06:52:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4f6effd700 1 -- 172.21.15.89:0/2654200268 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f800999d0 con 0x55cb7e8919e0 2023-10-26T06:52:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4fa959db80 10 client.16119 fetch_fsmap learned FSMap version 20 2023-10-26T06:52:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cb7e5f3c60 con 0x55cb7e8919e0 2023-10-26T06:52:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4f6effd700 1 -- 172.21.15.89:0/2654200268 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8009f6e0 con 0x55cb7e8919e0 2023-10-26T06:52:58.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4f6effd700 1 -- 172.21.15.89:0/2654200268 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4f800b9ab0 con 0x55cb7e8919e0 2023-10-26T06:52:58.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4fa959db80 10 client.16119 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:52:58.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4fa959db80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:52:58.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.940+0000 7f4fa959db80 20 client.16119 populate_metadata read hostname 'smithi089' 2023-10-26T06:52:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.941+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8ffa90 0x55cb7e901e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:52:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.941+0000 7f4fa959db80 4 client.16119 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:52:58.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.941+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb7e7d8e50 con 0x55cb7e8ffa90 2023-10-26T06:52:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.941+0000 7f4f8effd700 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8ffa90 0x55cb7e901e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:52:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.941+0000 7f4f8effd700 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8ffa90 0x55cb7e901e80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219acb9b6cefd192 server_cookie=de8e8eddc8545af3 in_seq=0 out_seq=0 2023-10-26T06:52:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.941+0000 7f4f6effd700 10 client.16119 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4f6effd700 1 -- 172.21.15.89:0/2654200268 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb7e7d8e50 con 0x55cb7e8ffa90 2023-10-26T06:52:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4f6effd700 10 client.16119 handle_command_reply: tid=0 2023-10-26T06:52:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 client.16119 shutdown 2023-10-26T06:52:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 msgr2=0x55cb7e8a2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e8a2060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4f800a3a50 tx=0x7f4f80097230 comp rx=0 tx=0).stop 2023-10-26T06:52:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 shutdown_connections 2023-10-26T06:52:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cb7e8919e0 0x55cb7e8a2060 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cb7e8ffa90 0x55cb7e901e80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb7e897910 0x55cb7e8a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 --2- 172.21.15.89:0/2654200268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb7e896ff0 0x55cb7e8a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4f6effd700 0 client.16119 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:52:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 >> 172.21.15.89:0/2654200268 conn(0x55cb7e801720 msgr2=0x55cb7e8951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 shutdown_connections 2023-10-26T06:52:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 1 -- 172.21.15.89:0/2654200268 wait complete. 2023-10-26T06:52:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.942+0000 7f4fa959db80 20 client.16119 trim_cache size 0 max 16384 2023-10-26T06:52:58.977 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:52:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f780a15f0 msgr2=0x7f4f780a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f780a15f0 0x7f4f780a3ab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009c60 tx=0x7f4f880077e0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90100710 msgr2=0x7f4f9019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:52:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90100710 0x7f4f9019c890 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4f80000c00 tx=0x7f4f800057f0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 shutdown_connections 2023-10-26T06:52:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4f780a15f0 0x7f4f780a3ab0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f90100710 0x7f4f9019c890 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f90106720 0x7f4f901969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 --2- 172.21.15.89:0/4250523532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f901010d0 0x7f4f9019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:52:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 >> 172.21.15.89:0/4250523532 conn(0x7f4f900fc200 msgr2=0x7f4f900fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:52:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 shutdown_connections 2023-10-26T06:52:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:52:58.943+0000 7f4f95bff700 1 -- 172.21.15.89:0/4250523532 wait complete. 2023-10-26T06:52:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:59 smithi192 ceph-mon[131661]: pgmap v492: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:52:59 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/806061368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:52:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:59 smithi135 ceph-mon[131403]: pgmap v492: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:52:59 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/806061368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:52:59.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:59 smithi089 ceph-mon[95031]: pgmap v492: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:52:59.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:52:59 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/806061368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:52:59.981 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:52:59.982 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:53:00.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3310607418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc106720 msgr2=0x7fb7dc106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3310607418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc106720 0x7fb7dc106b00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009990 tx=0x7fb7cc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3310607418 shutdown_connections 2023-10-26T06:53:00.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3310607418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc106720 0x7fb7dc106b00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3310607418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc1010d0 0x7fb7dc10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3310607418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc100710 0x7fb7dc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3310607418 >> 172.21.15.89:0/3310607418 conn(0x7fb7dc0fc200 msgr2=0x7fb7dc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3310607418 shutdown_connections 2023-10-26T06:53:00.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.193+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3310607418 wait complete. 2023-10-26T06:53:00.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.194+0000 7fb7e0bff700 1 Processor -- start 2023-10-26T06:53:00.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.194+0000 7fb7e0bff700 1 -- start start 2023-10-26T06:53:00.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7e0bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc100710 0x7fb7dc198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7e0bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc1010d0 0x7fb7dc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7e0bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc106720 0x7fb7dc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7e0bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19dc90 con 0x7fb7dc1010d0 2023-10-26T06:53:00.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7e0bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19de00 con 0x7fb7dc100710 2023-10-26T06:53:00.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7e0bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19dfc0 con 0x7fb7dc106720 2023-10-26T06:53:00.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7d9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc1010d0 0x7fb7dc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7dad9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc106720 0x7fb7dc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7da59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc100710 0x7fb7dc198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7d9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc1010d0 0x7fb7dc1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52120/0 (socket says 172.21.15.89:52120) 2023-10-26T06:53:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.195+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/3974751076 learned_addr learned my addr 172.21.15.89:0/3974751076 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/3974751076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc106720 msgr2=0x7fb7dc19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc106720 0x7fb7dc19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/3974751076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc100710 msgr2=0x7fb7dc198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc100710 0x7fb7dc198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/3974751076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0095f0 con 0x7fb7dc1010d0 2023-10-26T06:53:00.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc1010d0 0x7fb7dc1992b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000d950 tx=0x7fb7d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7cb7fe700 1 -- 172.21.15.89:0/3974751076 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000f890 con 0x7fb7dc1010d0 2023-10-26T06:53:00.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7cb7fe700 1 -- 172.21.15.89:0/3974751076 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d0010460 con 0x7fb7dc1010d0 2023-10-26T06:53:00.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7cb7fe700 1 -- 172.21.15.89:0/3974751076 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000e610 con 0x7fb7dc1010d0 2023-10-26T06:53:00.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc19e2a0 con 0x7fb7dc1010d0 2023-10-26T06:53:00.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.196+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc1a5ae0 con 0x7fb7dc1010d0 2023-10-26T06:53:00.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.199+0000 7fb7cb7fe700 1 -- 172.21.15.89:0/3974751076 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb7d00108d0 con 0x7fb7dc1010d0 2023-10-26T06:53:00.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7cb7fe700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7c40a1670 0x7fb7c40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 Processor -- start 2023-10-26T06:53:00.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7cb7fe700 1 -- 172.21.15.89:0/3974751076 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb7d00cce90 con 0x7fb7dc1010d0 2023-10-26T06:53:00.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 -- start start 2023-10-26T06:53:00.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7da59c700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7c40a1670 0x7fb7c40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed90e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90e580 0x55d6ed908110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed908ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed87b760 con 0x55d6ed908650 2023-10-26T06:53:00.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed861ce0 con 0x55d6ed90e580 2023-10-26T06:53:00.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.201+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed85f600 con 0x55d6ed90dc60 2023-10-26T06:53:00.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7dad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed908ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7dad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed908ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52128/0 (socket says 172.21.15.89:52128) 2023-10-26T06:53:00.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90e580 0x55d6ed908110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7dad9d700 1 -- 172.21.15.89:0/3350941243 learned_addr learned my addr 172.21.15.89:0/3350941243 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7da59c700 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed90e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7da59c700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7c40a1670 0x7fb7c40a3b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006010 tx=0x7fb7cc00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/3350941243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 msgr2=0x55d6ed90e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed90e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/3350941243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 msgr2=0x55d6ed908ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed908ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/3350941243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ed818470 con 0x55d6ed90e580 2023-10-26T06:53:00.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7dad9d700 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed908ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:00.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90e580 0x55d6ed908110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00ca5c0 tx=0x7fb7d009a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b24cc7f8d4ab5a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.202+0000 7fb7c8ff9700 1 -- 172.21.15.89:0/3350941243 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00a21e0 con 0x55d6ed90e580 2023-10-26T06:53:00.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7c8ff9700 1 -- 172.21.15.89:0/3350941243 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d0096560 con 0x55d6ed90e580 2023-10-26T06:53:00.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7c8ff9700 1 -- 172.21.15.89:0/3350941243 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00995a0 con 0x55d6ed90e580 2023-10-26T06:53:00.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3350941243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90e580 msgr2=0x55d6ed908110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90e580 0x55d6ed908110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00ca5c0 tx=0x7fb7d009a550 comp rx=0 tx=0).stop 2023-10-26T06:53:00.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3350941243 shutdown_connections 2023-10-26T06:53:00.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed908ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed90e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3350941243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90e580 0x55d6ed908110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3350941243 >> 172.21.15.89:0/3350941243 conn(0x55d6ed87c7a0 msgr2=0x55d6ed902be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3350941243 shutdown_connections 2023-10-26T06:53:00.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.203+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3350941243 wait complete. 2023-10-26T06:53:00.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.204+0000 7fb7f4543b80 1 -- start start 2023-10-26T06:53:00.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.204+0000 7fb7f4543b80 1 Processor -- start 2023-10-26T06:53:00.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed911950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90dc60 0x55d6ed911e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed9123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed85aef0 con 0x55d6ed908650 2023-10-26T06:53:00.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed7f7790 con 0x55d6ed90dc60 2023-10-26T06:53:00.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed7f7430 con 0x55d6ed90e580 2023-10-26T06:53:00.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7f4543b80 10 client.0 _ll_register_callbacks cb 0x55d6ed741b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:00.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7da59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed911950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7d9d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90dc60 0x55d6ed911e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7d9d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90dc60 0x55d6ed911e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:48762/0 (socket says 172.21.15.89:48762) 2023-10-26T06:53:00.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/1939544581 learned_addr learned my addr 172.21.15.89:0/1939544581 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.205+0000 7fb7dad9d700 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed9123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/1939544581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 msgr2=0x55d6ed9123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed9123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/1939544581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 msgr2=0x55d6ed911950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed911950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/1939544581 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d4003bd0 con 0x55d6ed90dc60 2023-10-26T06:53:00.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90dc60 0x55d6ed911e90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0099de0 tx=0x7fb7d0005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c870d49c2d6ca72f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:53:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/1939544581 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00a21e0 con 0x55d6ed90dc60 2023-10-26T06:53:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/1939544581 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb7d0099550 con 0x55d6ed90dc60 2023-10-26T06:53:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6ed85ff70 con 0x55d6ed90dc60 2023-10-26T06:53:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/1939544581 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d009a0b0 con 0x55d6ed90dc60 2023-10-26T06:53:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7f4543b80 10 client.26168 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.206+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fb7dc1a5ae0 con 0x55d6ed90dc60 2023-10-26T06:53:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/1939544581 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d6ed85ff70 con 0x55d6ed90dc60 2023-10-26T06:53:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/1939544581 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7d00a1de0 con 0x55d6ed90dc60 2023-10-26T06:53:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7f4543b80 10 client.26168 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7f4543b80 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:00.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7f4543b80 20 client.26168 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:00.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed921b30 0x55d6ed923f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7f4543b80 4 client.26168 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:53:00.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6ed8d7930 con 0x55d6ed921b30 2023-10-26T06:53:00.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.207+0000 7fb7da59c700 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed921b30 0x55d6ed923f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.208+0000 7fb7da59c700 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed921b30 0x55d6ed923f50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679c9dbe97ffc717 server_cookie=5b46e9ecac3549bc in_seq=0 out_seq=0 2023-10-26T06:53:00.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.208+0000 7fb7c1ffb700 10 client.26168 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.208+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/1939544581 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d6ed8d7930 con 0x55d6ed921b30 2023-10-26T06:53:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.208+0000 7fb7c1ffb700 10 client.26168 handle_command_reply: tid=0 2023-10-26T06:53:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.208+0000 7fb7f4543b80 1 client.26168 shutdown 2023-10-26T06:53:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90dc60 msgr2=0x55d6ed911e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90dc60 0x55d6ed911e90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0099de0 tx=0x7fb7d0005b40 comp rx=0 tx=0).stop 2023-10-26T06:53:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 shutdown_connections 2023-10-26T06:53:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed908650 0x55d6ed911950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed921b30 0x55d6ed923f50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed9123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/1939544581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed90dc60 0x55d6ed911e90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 >> 172.21.15.89:0/1939544581 conn(0x55d6ed87c7a0 msgr2=0x55d6ed907100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7c1ffb700 0 client.26168 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 shutdown_connections 2023-10-26T06:53:00.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 1 -- 172.21.15.89:0/1939544581 wait complete. 2023-10-26T06:53:00.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.209+0000 7fb7f4543b80 20 client.26168 trim_cache size 0 max 16384 2023-10-26T06:53:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.226+0000 7fb7f4543b80 1 Processor -- start 2023-10-26T06:53:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7f4543b80 1 -- start start 2023-10-26T06:53:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed90beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90dc60 0x55d6ed911ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed912400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed8fdb70 con 0x55d6ed90dc60 2023-10-26T06:53:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed6db9a0 con 0x55d6ed908650 2023-10-26T06:53:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed9036d0 con 0x55d6ed90e580 2023-10-26T06:53:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7d9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90dc60 0x55d6ed911ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7da59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed90beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7d9d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90dc60 0x55d6ed911ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52142/0 (socket says 172.21.15.89:52142) 2023-10-26T06:53:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.227+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/2717865437 learned_addr learned my addr 172.21.15.89:0/2717865437 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/2717865437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 msgr2=0x55d6ed912400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7dad9d700 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed912400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed912400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/2717865437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 msgr2=0x55d6ed90beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed90beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7d9d9b700 1 -- 172.21.15.89:0/2717865437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ed928b40 con 0x55d6ed90dc60 2023-10-26T06:53:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7dad9d700 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed912400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:53:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7da59c700 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed90beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:00.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90dc60 0x55d6ed911ec0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00cab40 tx=0x7fb7d0005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e0a22b93ecdf1a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7c97fa700 1 -- 172.21.15.89:0/2717865437 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00a21e0 con 0x55d6ed90dc60 2023-10-26T06:53:00.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7c97fa700 1 -- 172.21.15.89:0/2717865437 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d0099550 con 0x55d6ed90dc60 2023-10-26T06:53:00.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.228+0000 7fb7c97fa700 1 -- 172.21.15.89:0/2717865437 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00830d0 con 0x55d6ed90dc60 2023-10-26T06:53:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 -- 172.21.15.89:0/2717865437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90dc60 msgr2=0x55d6ed911ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90dc60 0x55d6ed911ec0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00cab40 tx=0x7fb7d0005bb0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 -- 172.21.15.89:0/2717865437 shutdown_connections 2023-10-26T06:53:00.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90e580 0x55d6ed912400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed90beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/2717865437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90dc60 0x55d6ed911ec0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 -- 172.21.15.89:0/2717865437 >> 172.21.15.89:0/2717865437 conn(0x55d6ed87c7a0 msgr2=0x55d6ed90b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 -- 172.21.15.89:0/2717865437 shutdown_connections 2023-10-26T06:53:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 -- 172.21.15.89:0/2717865437 wait complete. 2023-10-26T06:53:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 -- start start 2023-10-26T06:53:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.229+0000 7fb7f4543b80 1 Processor -- start 2023-10-26T06:53:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed918d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed917320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7f4543b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90e580 0x55d6ed917860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed917da0 con 0x55d6ed90e580 2023-10-26T06:53:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed917f10 con 0x55d6ed908650 2023-10-26T06:53:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7f4543b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ed9180d0 con 0x55d6ed90dc60 2023-10-26T06:53:00.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7f4543b80 10 client.0 _ll_register_callbacks cb 0x55d6ed741b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:00.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90e580 0x55d6ed917860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90e580 0x55d6ed917860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52150/0 (socket says 172.21.15.89:52150) 2023-10-26T06:53:00.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 -- 172.21.15.89:0/3545659913 learned_addr learned my addr 172.21.15.89:0/3545659913 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7da59c700 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed918d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7d9d9b700 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed917320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 -- 172.21.15.89:0/3545659913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 msgr2=0x55d6ed917320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed917320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 -- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 msgr2=0x55d6ed918d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed918d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.230+0000 7fb7dad9d700 1 -- 172.21.15.89:0/3545659913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc10dee0 con 0x55d6ed90e580 2023-10-26T06:53:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7dad9d700 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90e580 0x55d6ed917860 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4007b20 tx=0x7fb7d4007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d62d361d0ce32 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/3545659913 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4017770 con 0x55d6ed90e580 2023-10-26T06:53:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/3545659913 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fb7d40178d0 con 0x55d6ed90e580 2023-10-26T06:53:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7d0096560 con 0x55d6ed90e580 2023-10-26T06:53:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/3545659913 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d4017a90 con 0x55d6ed90e580 2023-10-26T06:53:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7f4543b80 10 client.16137 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fb7dc1a5ae0 con 0x55d6ed90e580 2023-10-26T06:53:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/3545659913 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d0096560 con 0x55d6ed90e580 2023-10-26T06:53:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.231+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/3545659913 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7d4018420 con 0x55d6ed90e580 2023-10-26T06:53:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7f4543b80 10 client.16137 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7f4543b80 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7f4543b80 20 client.16137 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed976f40 0x55d6ed979360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7f4543b80 4 client.16137 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:53:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d6ed853ed0 con 0x55d6ed976f40 2023-10-26T06:53:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7da59c700 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed976f40 0x55d6ed979360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7da59c700 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed976f40 0x55d6ed979360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29fa816b52a3d849 server_cookie=231a09adf87b7d9a in_seq=0 out_seq=0 2023-10-26T06:53:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.232+0000 7fb7c1ffb700 10 client.16137 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7c1ffb700 1 -- 172.21.15.89:0/3545659913 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d6ed853ed0 con 0x55d6ed976f40 2023-10-26T06:53:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7c1ffb700 10 client.16137 handle_command_reply: tid=0 2023-10-26T06:53:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 client.16137 shutdown 2023-10-26T06:53:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90e580 msgr2=0x55d6ed917860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90e580 0x55d6ed917860 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4007b20 tx=0x7fb7d4007c00 comp rx=0 tx=0).stop 2023-10-26T06:53:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 shutdown_connections 2023-10-26T06:53:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6ed90e580 0x55d6ed917860 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d6ed976f40 0x55d6ed979360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6ed908650 0x55d6ed918d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 --2- 172.21.15.89:0/3545659913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6ed90dc60 0x55d6ed917320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 >> 172.21.15.89:0/3545659913 conn(0x55d6ed87c7a0 msgr2=0x55d6ed90bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7c1ffb700 0 client.16137 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 shutdown_connections 2023-10-26T06:53:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 1 -- 172.21.15.89:0/3545659913 wait complete. 2023-10-26T06:53:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.234+0000 7fb7f4543b80 20 client.16137 trim_cache size 0 max 16384 2023-10-26T06:53:00.266 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:53:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:53:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "1aa7abb6-36ff-4888-9add-0f58d3625b29", 2023-10-26T06:53:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:53:00.268 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:53:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7c40a1670 msgr2=0x7fb7c40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7c40a1670 0x7fb7c40a3b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006010 tx=0x7fb7cc00b540 comp rx=0 tx=0).stop 2023-10-26T06:53:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc1010d0 msgr2=0x7fb7dc1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc1010d0 0x7fb7dc1992b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000d950 tx=0x7fb7d000dc60 comp rx=0 tx=0).stop 2023-10-26T06:53:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 shutdown_connections 2023-10-26T06:53:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fb7c40a1670 0x7fb7c40a3b30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc106720 0x7fb7dc19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc100710 0x7fb7dc198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 --2- 172.21.15.89:0/3974751076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb7dc1010d0 0x7fb7dc1992b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 >> 172.21.15.89:0/3974751076 conn(0x7fb7dc0fc200 msgr2=0x7fb7dc10bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 shutdown_connections 2023-10-26T06:53:00.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.235+0000 7fb7e0bff700 1 -- 172.21.15.89:0/3974751076 wait complete. 2023-10-26T06:53:00.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:53:00.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 -- 172.21.15.89:0/510129076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64106730 msgr2=0x7f3b64106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 --2- 172.21.15.89:0/510129076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64106730 0x7f3b64106b10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009990 tx=0x7f3b54009ca0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 -- 172.21.15.89:0/510129076 shutdown_connections 2023-10-26T06:53:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 --2- 172.21.15.89:0/510129076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64106730 0x7f3b64106b10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 --2- 172.21.15.89:0/510129076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b641010e0 0x7f3b6410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 --2- 172.21.15.89:0/510129076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b64100720 0x7f3b64100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 -- 172.21.15.89:0/510129076 >> 172.21.15.89:0/510129076 conn(0x7f3b640fc1b0 msgr2=0x7f3b640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 -- 172.21.15.89:0/510129076 shutdown_connections 2023-10-26T06:53:00.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.473+0000 7f3b69bff700 1 -- 172.21.15.89:0/510129076 wait complete. 2023-10-26T06:53:00.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.474+0000 7f3b69bff700 1 Processor -- start 2023-10-26T06:53:00.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.474+0000 7f3b69bff700 1 -- start start 2023-10-26T06:53:00.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b69bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64100720 0x7f3b64072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b69bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b641010e0 0x7f3b64073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b69bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b64106730 0x7f3b64073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b69bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64074ac0 con 0x7f3b64100720 2023-10-26T06:53:00.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b69bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b641a9510 con 0x7f3b64106730 2023-10-26T06:53:00.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b69bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b641a96d0 con 0x7f3b641010e0 2023-10-26T06:53:00.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64100720 0x7f3b64072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64100720 0x7f3b64072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52170/0 (socket says 172.21.15.89:52170) 2023-10-26T06:53:00.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 -- 172.21.15.89:0/592665480 learned_addr learned my addr 172.21.15.89:0/592665480 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b63fff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b64106730 0x7f3b64073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b641010e0 0x7f3b64073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 -- 172.21.15.89:0/592665480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b641010e0 msgr2=0x7f3b64073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b641010e0 0x7f3b64073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 -- 172.21.15.89:0/592665480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b64106730 msgr2=0x7f3b64073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b64106730 0x7f3b64073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.475+0000 7f3b637fe700 1 -- 172.21.15.89:0/592665480 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b540095f0 con 0x7f3b64100720 2023-10-26T06:53:00.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.476+0000 7f3b637fe700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64100720 0x7f3b64072af0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3b54005910 tx=0x7f3b54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.476+0000 7f3b60ff9700 1 -- 172.21.15.89:0/592665480 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5401d070 con 0x7f3b64100720 2023-10-26T06:53:00.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.476+0000 7f3b60ff9700 1 -- 172.21.15.89:0/592665480 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b54022470 con 0x7f3b64100720 2023-10-26T06:53:00.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.476+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b641a9950 con 0x7f3b64100720 2023-10-26T06:53:00.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.476+0000 7f3b60ff9700 1 -- 172.21.15.89:0/592665480 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54017650 con 0x7f3b64100720 2023-10-26T06:53:00.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.476+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b641a9e80 con 0x7f3b64100720 2023-10-26T06:53:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.479+0000 7f3b60ff9700 1 -- 172.21.15.89:0/592665480 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3b540225e0 con 0x7f3b64100720 2023-10-26T06:53:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.480+0000 7f3b7d4bab80 1 Processor -- start 2023-10-26T06:53:00.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.480+0000 7f3b7d4bab80 1 -- start start 2023-10-26T06:53:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b60ff9700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b4c0a1670 0x7f3b4c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ab9f96e0 con 0x5632aba8bbe0 2023-10-26T06:53:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ab9dfc60 con 0x5632aba865d0 2023-10-26T06:53:00.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ab9dd580 con 0x5632aba8c500 2023-10-26T06:53:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b60ff9700 1 -- 172.21.15.89:0/592665480 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3b540ce2d0 con 0x7f3b64100720 2023-10-26T06:53:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b63fff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b4c0a1670 0x7f3b4c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba86090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:36558/0 (socket says 172.21.15.89:36558) 2023-10-26T06:53:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 -- 172.21.15.89:0/1096032756 learned_addr learned my addr 172.21.15.89:0/1096032756 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b63fff700 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 -- 172.21.15.89:0/1096032756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 msgr2=0x5632aba86a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 -- 172.21.15.89:0/1096032756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 msgr2=0x5632aba8bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.481+0000 7f3b62ffd700 1 -- 172.21.15.89:0/1096032756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ab9963f0 con 0x5632aba8c500 2023-10-26T06:53:00.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.482+0000 7f3b63fff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b4c0a1670 0x7f3b4c0a3b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009ec0 tx=0x7f3b5c00fc40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.482+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba86090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009cf0 tx=0x7f3b58011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69b380c368f619d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.482+0000 7f3b49ffb700 1 -- 172.21.15.89:0/1096032756 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011d70 con 0x5632aba8c500 2023-10-26T06:53:00.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.482+0000 7f3b49ffb700 1 -- 172.21.15.89:0/1096032756 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b58004510 con 0x5632aba8c500 2023-10-26T06:53:00.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.482+0000 7f3b49ffb700 1 -- 172.21.15.89:0/1096032756 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58005280 con 0x5632aba8c500 2023-10-26T06:53:00.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/1096032756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 msgr2=0x5632aba86090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba86090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009cf0 tx=0x7f3b58011440 comp rx=0 tx=0).stop 2023-10-26T06:53:00.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/1096032756 shutdown_connections 2023-10-26T06:53:00.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/1096032756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba86090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/1096032756 >> 172.21.15.89:0/1096032756 conn(0x5632ab9fa720 msgr2=0x5632aba80b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/1096032756 shutdown_connections 2023-10-26T06:53:00.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/1096032756 wait complete. 2023-10-26T06:53:00.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.483+0000 7f3b7d4bab80 1 -- start start 2023-10-26T06:53:00.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 1 Processor -- start 2023-10-26T06:53:00.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba92420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba8fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ab9d8e70 con 0x5632aba8bbe0 2023-10-26T06:53:00.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ab975710 con 0x5632aba865d0 2023-10-26T06:53:00.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ab9753b0 con 0x5632aba8c500 2023-10-26T06:53:00.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b7d4bab80 10 client.0 _ll_register_callbacks cb 0x5632ab8bfa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:00.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba92420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba92420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:48814/0 (socket says 172.21.15.89:48814) 2023-10-26T06:53:00.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.484+0000 7f3b637fe700 1 -- 172.21.15.89:0/2100421368 learned_addr learned my addr 172.21.15.89:0/2100421368 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b63fff700 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba8fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b637fe700 1 -- 172.21.15.89:0/2100421368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 msgr2=0x5632aba8fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b637fe700 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba8fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b637fe700 1 -- 172.21.15.89:0/2100421368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 msgr2=0x5632aba8f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b637fe700 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b637fe700 1 -- 172.21.15.89:0/2100421368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b58009920 con 0x5632aba865d0 2023-10-26T06:53:00.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:00.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b637fe700 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba92420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b5409b020 tx=0x7f3b540cbf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d090e032ec9e4e2b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:53:00.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2100421368 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b540ad690 con 0x5632aba865d0 2023-10-26T06:53:00.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2100421368 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3b5409ba20 con 0x5632aba865d0 2023-10-26T06:53:00.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632ab9ddef0 con 0x5632aba865d0 2023-10-26T06:53:00.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.485+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2100421368 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b540ad460 con 0x5632aba865d0 2023-10-26T06:53:00.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 10 client.26186 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:00.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5632ab803de0 con 0x5632aba865d0 2023-10-26T06:53:00.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2100421368 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632ab9ddef0 con 0x5632aba865d0 2023-10-26T06:53:00.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2100421368 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b540ad870 con 0x5632aba865d0 2023-10-26T06:53:00.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 10 client.26186 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:00.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:00.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 20 client.26186 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:00.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632aba9f9e0 0x5632abaa1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 4 client.26186 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:53:00.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632aba558b0 con 0x5632aba9f9e0 2023-10-26T06:53:00.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.486+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632aba9f9e0 0x5632abaa1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.487+0000 7f3b42ffd700 10 client.26186 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.487+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632aba9f9e0 0x5632abaa1dd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6095856a71fca8 server_cookie=19577b4d176266e7 in_seq=0 out_seq=0 2023-10-26T06:53:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2100421368 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5632aba558b0 con 0x5632aba9f9e0 2023-10-26T06:53:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b42ffd700 10 client.26186 handle_command_reply: tid=0 2023-10-26T06:53:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 client.26186 shutdown 2023-10-26T06:53:00.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 msgr2=0x5632aba92420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba92420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b5409b020 tx=0x7f3b540cbf60 comp rx=0 tx=0).stop 2023-10-26T06:53:00.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 shutdown_connections 2023-10-26T06:53:00.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632aba9f9e0 0x5632abaa1dd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba8fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b42ffd700 0 client.26186 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba92420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.517 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2100421368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba8f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 >> 172.21.15.89:0/2100421368 conn(0x5632ab9fa720 msgr2=0x5632aba80ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 shutdown_connections 2023-10-26T06:53:00.518 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2100421368 wait complete. 2023-10-26T06:53:00.519 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.488+0000 7f3b7d4bab80 20 client.26186 trim_cache size 0 max 16384 2023-10-26T06:53:00.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 Processor -- start 2023-10-26T06:53:00.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 -- start start 2023-10-26T06:53:00.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba7b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba9b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ab859920 con 0x5632aba8bbe0 2023-10-26T06:53:00.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632aba92c90 con 0x5632aba865d0 2023-10-26T06:53:00.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.505+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632aba823b0 con 0x5632aba8c500 2023-10-26T06:53:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba7b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba7b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52192/0 (socket says 172.21.15.89:52192) 2023-10-26T06:53:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 -- 172.21.15.89:0/480467681 learned_addr learned my addr 172.21.15.89:0/480467681 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b63fff700 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba9b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 -- 172.21.15.89:0/480467681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 msgr2=0x5632aba9b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba9b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 -- 172.21.15.89:0/480467681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 msgr2=0x5632aba86090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 -- 172.21.15.89:0/480467681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632aba80d70 con 0x5632aba8bbe0 2023-10-26T06:53:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b637fe700 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.506+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba7b5d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5632aba827d0 tx=0x7f3b580046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a84c0d1790c4418 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b4a7fc700 1 -- 172.21.15.89:0/480467681 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58004050 con 0x5632aba8bbe0 2023-10-26T06:53:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b4a7fc700 1 -- 172.21.15.89:0/480467681 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b580042b0 con 0x5632aba8bbe0 2023-10-26T06:53:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b4a7fc700 1 -- 172.21.15.89:0/480467681 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011ea0 con 0x5632aba8bbe0 2023-10-26T06:53:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/480467681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 msgr2=0x5632aba7b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba7b5d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5632aba827d0 tx=0x7f3b580046a0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/480467681 shutdown_connections 2023-10-26T06:53:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8c500 0x5632aba9b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba86090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/480467681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8bbe0 0x5632aba7b5d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/480467681 >> 172.21.15.89:0/480467681 conn(0x5632ab9fa720 msgr2=0x5632aba8e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/480467681 shutdown_connections 2023-10-26T06:53:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.507+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/480467681 wait complete. 2023-10-26T06:53:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 -- start start 2023-10-26T06:53:00.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 Processor -- start 2023-10-26T06:53:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba97100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8bbe0 0x5632aba87cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8c500 0x5632aba88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632aba88ae0 con 0x5632aba8c500 2023-10-26T06:53:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632abaf3800 con 0x5632aba865d0 2023-10-26T06:53:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632abaf3940 con 0x5632aba8bbe0 2023-10-26T06:53:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.508+0000 7f3b7d4bab80 10 client.0 _ll_register_callbacks cb 0x5632ab8bfa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8c500 0x5632aba88400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8c500 0x5632aba88400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52202/0 (socket says 172.21.15.89:52202) 2023-10-26T06:53:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 -- 172.21.15.89:0/2471109697 learned_addr learned my addr 172.21.15.89:0/2471109697 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b62ffd700 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8bbe0 0x5632aba87cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b637fe700 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba97100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 -- 172.21.15.89:0/2471109697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8bbe0 msgr2=0x5632aba87cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8bbe0 0x5632aba87cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 -- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 msgr2=0x5632aba97100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba97100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 -- 172.21.15.89:0/2471109697 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632ab803de0 con 0x5632aba8c500 2023-10-26T06:53:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b63fff700 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8c500 0x5632aba88400 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c018b00 tx=0x7f3b5c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf9c7fdc78de97e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.509+0000 7f3b42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2471109697 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00e040 con 0x5632aba8c500 2023-10-26T06:53:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b58004510 con 0x5632aba8c500 2023-10-26T06:53:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2471109697 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3b5c008ba0 con 0x5632aba8c500 2023-10-26T06:53:00.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2471109697 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b5c008d60 con 0x5632aba8c500 2023-10-26T06:53:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 10 client.16155 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f3b5801d040 con 0x5632aba8c500 2023-10-26T06:53:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2471109697 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b58004510 con 0x5632aba8c500 2023-10-26T06:53:00.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2471109697 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b5c02c070 con 0x5632aba8c500 2023-10-26T06:53:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 10 client.16155 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 20 client.16155 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:00.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632abaf5210 0x5632abaf7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 4 client.16155 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:53:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.510+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632ab9d1e50 con 0x5632abaf5210 2023-10-26T06:53:00.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.511+0000 7f3b637fe700 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632abaf5210 0x5632abaf7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.511+0000 7f3b637fe700 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632abaf5210 0x5632abaf7630 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f2a899386ec748 server_cookie=898c1071eee96f96 in_seq=0 out_seq=0 2023-10-26T06:53:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.511+0000 7f3b42ffd700 10 client.16155 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b42ffd700 1 -- 172.21.15.89:0/2471109697 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5632ab9d1e50 con 0x5632abaf5210 2023-10-26T06:53:00.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b42ffd700 10 client.16155 handle_command_reply: tid=0 2023-10-26T06:53:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 client.16155 shutdown 2023-10-26T06:53:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8c500 msgr2=0x5632aba88400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8c500 0x5632aba88400 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c018b00 tx=0x7f3b5c004b40 comp rx=0 tx=0).stop 2023-10-26T06:53:00.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 shutdown_connections 2023-10-26T06:53:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5632aba8c500 0x5632aba88400 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5632abaf5210 0x5632abaf7630 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632aba865d0 0x5632aba97100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 --2- 172.21.15.89:0/2471109697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5632aba8bbe0 0x5632aba87cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b42ffd700 0 client.16155 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:00.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 >> 172.21.15.89:0/2471109697 conn(0x5632ab9fa720 msgr2=0x5632aba7a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 shutdown_connections 2023-10-26T06:53:00.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.512+0000 7f3b7d4bab80 1 -- 172.21.15.89:0/2471109697 wait complete. 2023-10-26T06:53:00.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b7d4bab80 20 client.16155 trim_cache size 0 max 16384 2023-10-26T06:53:00.546 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:53:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:53:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:53:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "1aa7abb6-36ff-4888-9add-0f58d3625b29": { 2023-10-26T06:53:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:53:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "1aa7abb6-36ff-4888-9add-0f58d3625b29", 2023-10-26T06:53:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:53:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:53:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:53:00.549 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:53:00.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b4c0a1670 msgr2=0x7f3b4c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b4c0a1670 0x7f3b4c0a3b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009ec0 tx=0x7f3b5c00fc40 comp rx=0 tx=0).stop 2023-10-26T06:53:00.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64100720 msgr2=0x7f3b64072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:00.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64100720 0x7f3b64072af0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3b54005910 tx=0x7f3b54004730 comp rx=0 tx=0).stop 2023-10-26T06:53:00.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 shutdown_connections 2023-10-26T06:53:00.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3b4c0a1670 0x7f3b4c0a3b30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b64100720 0x7f3b64072af0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b64106730 0x7f3b64073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 --2- 172.21.15.89:0/592665480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b641010e0 0x7f3b64073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:00.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 >> 172.21.15.89:0/592665480 conn(0x7f3b640fc1b0 msgr2=0x7f3b640fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:00.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 shutdown_connections 2023-10-26T06:53:00.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:00.513+0000 7f3b69bff700 1 -- 172.21.15.89:0/592665480 wait complete. 2023-10-26T06:53:00.553 INFO:tasks.cephfs.filesystem:scrub status for tag:1aa7abb6-36ff-4888-9add-0f58d3625b29 - {'path': '/', 'tag': '1aa7abb6-36ff-4888-9add-0f58d3625b29', 'options': 'recursive,force'} 2023-10-26T06:53:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:01 smithi192 ceph-mon[131661]: pgmap v493: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:01.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:01 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:53:01.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:01 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:01.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:01 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:53:01.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:01 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:01 smithi135 ceph-mon[131403]: pgmap v493: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:01.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:01 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:53:01.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:01 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:01.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:01 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:53:01.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:01 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:01 smithi089 ceph-mon[95031]: pgmap v493: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:01 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:53:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:01 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:01.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:01 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:53:01.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:01 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:03 smithi192 ceph-mon[131661]: pgmap v494: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:53:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:03 smithi135 ceph-mon[131403]: pgmap v494: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:53:03.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:03 smithi089 ceph-mon[95031]: pgmap v494: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:53:04.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:04 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:53:04.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:04 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:53:04.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:04 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:53:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:05 smithi135 ceph-mon[131403]: pgmap v495: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-26T06:53:05.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:05 smithi089 ceph-mon[95031]: pgmap v495: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-26T06:53:05.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:05 smithi192 ceph-mon[131661]: pgmap v495: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-26T06:53:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:07 smithi135 ceph-mon[131403]: pgmap v496: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:53:07.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:07 smithi089 ceph-mon[95031]: pgmap v496: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:53:07.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:07 smithi192 ceph-mon[131661]: pgmap v496: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:53:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:09 smithi135 ceph-mon[131403]: pgmap v497: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:53:09.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:09 smithi089 ceph-mon[95031]: pgmap v497: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:53:09.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:09 smithi192 ceph-mon[131661]: pgmap v497: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:53:10.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:10 smithi135 ceph-mon[131403]: osdmap e87: 12 total, 12 up, 12 in 2023-10-26T06:53:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:10 smithi089 ceph-mon[95031]: osdmap e87: 12 total, 12 up, 12 in 2023-10-26T06:53:10.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:10 smithi192 ceph-mon[131661]: osdmap e87: 12 total, 12 up, 12 in 2023-10-26T06:53:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:11 smithi135 ceph-mon[131403]: pgmap v499: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:53:11.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:11 smithi089 ceph-mon[95031]: pgmap v499: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:53:11.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:11 smithi192 ceph-mon[131661]: pgmap v499: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:53:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:13 smithi135 ceph-mon[131403]: pgmap v500: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:53:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:13 smithi089 ceph-mon[95031]: pgmap v500: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:53:13.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:13 smithi192 ceph-mon[131661]: pgmap v500: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:53:15.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:15 smithi135 ceph-mon[131403]: pgmap v501: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.2 KiB/s rd, 27 KiB/s wr, 4 op/s 2023-10-26T06:53:15.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:15 smithi135 ceph-mon[131403]: osdmap e88: 12 total, 12 up, 12 in 2023-10-26T06:53:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:15 smithi089 ceph-mon[95031]: pgmap v501: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.2 KiB/s rd, 27 KiB/s wr, 4 op/s 2023-10-26T06:53:15.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:15 smithi089 ceph-mon[95031]: osdmap e88: 12 total, 12 up, 12 in 2023-10-26T06:53:15.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:15 smithi192 ceph-mon[131661]: pgmap v501: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.2 KiB/s rd, 27 KiB/s wr, 4 op/s 2023-10-26T06:53:15.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:15 smithi192 ceph-mon[131661]: osdmap e88: 12 total, 12 up, 12 in 2023-10-26T06:53:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:17 smithi135 ceph-mon[131403]: pgmap v503: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2023-10-26T06:53:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:17 smithi089 ceph-mon[95031]: pgmap v503: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2023-10-26T06:53:17.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:17 smithi192 ceph-mon[131661]: pgmap v503: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2023-10-26T06:53:19.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:19 smithi135 ceph-mon[131403]: pgmap v504: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 240 B/s wr, 2 op/s 2023-10-26T06:53:19.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:19 smithi089 ceph-mon[95031]: pgmap v504: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 240 B/s wr, 2 op/s 2023-10-26T06:53:19.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:19 smithi192 ceph-mon[131661]: pgmap v504: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 240 B/s wr, 2 op/s 2023-10-26T06:53:21.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:21 smithi135 ceph-mon[131403]: pgmap v505: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:53:21.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:21 smithi089 ceph-mon[95031]: pgmap v505: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:53:21.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:21 smithi192 ceph-mon[131661]: pgmap v505: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:53:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:23 smithi135 ceph-mon[131403]: pgmap v506: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:53:23.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:23 smithi089 ceph-mon[95031]: pgmap v506: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:53:23.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:23 smithi192 ceph-mon[131661]: pgmap v506: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:53:25.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:25 smithi135 ceph-mon[131403]: pgmap v507: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:53:25.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:25 smithi089 ceph-mon[95031]: pgmap v507: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:53:25.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:25 smithi192 ceph-mon[131661]: pgmap v507: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:53:27.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:27 smithi135 ceph-mon[131403]: pgmap v508: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 95 B/s wr, 2 op/s 2023-10-26T06:53:27.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:27 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:53:27.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:27 smithi089 ceph-mon[95031]: pgmap v508: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 95 B/s wr, 2 op/s 2023-10-26T06:53:27.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:27 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:53:27.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:27 smithi192 ceph-mon[131661]: pgmap v508: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 95 B/s wr, 2 op/s 2023-10-26T06:53:27.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:27 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:53:28.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:53:28.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:53:28.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:53:28.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:53:28.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:53:28.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:53:28.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:53:28.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:53:28.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:53:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:29 smithi135 ceph-mon[131403]: pgmap v509: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:53:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:29 smithi089 ceph-mon[95031]: pgmap v509: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:53:29.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:29 smithi192 ceph-mon[131661]: pgmap v509: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:53:30.554 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:53:30.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 -- 172.21.15.89:0/1386012439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d501010e0 msgr2=0x7f1d5010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 --2- 172.21.15.89:0/1386012439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d501010e0 0x7f1d5010d5f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800b0a0 tx=0x7f1d4800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 -- 172.21.15.89:0/1386012439 shutdown_connections 2023-10-26T06:53:30.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 --2- 172.21.15.89:0/1386012439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d501010e0 0x7f1d5010d5f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 --2- 172.21.15.89:0/1386012439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50106730 0x7f1d50106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 --2- 172.21.15.89:0/1386012439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d50100720 0x7f1d50100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 -- 172.21.15.89:0/1386012439 >> 172.21.15.89:0/1386012439 conn(0x7f1d500fc1b0 msgr2=0x7f1d500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 -- 172.21.15.89:0/1386012439 shutdown_connections 2023-10-26T06:53:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.763+0000 7f1d553ff700 1 -- 172.21.15.89:0/1386012439 wait complete. 2023-10-26T06:53:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.764+0000 7f1d553ff700 1 Processor -- start 2023-10-26T06:53:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d553ff700 1 -- start start 2023-10-26T06:53:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d553ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50100720 0x7f1d5019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d553ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d501010e0 0x7f1d5019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d553ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d50106730 0x7f1d50196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d553ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50197070 con 0x7f1d50106730 2023-10-26T06:53:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d553ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d501971e0 con 0x7f1d501010e0 2023-10-26T06:53:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d553ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d501973a0 con 0x7f1d50100720 2023-10-26T06:53:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d50106730 0x7f1d50196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d50106730 0x7f1d50196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60908/0 (socket says 172.21.15.89:60908) 2023-10-26T06:53:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2106392870 learned_addr learned my addr 172.21.15.89:0/2106392870 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d4effd700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50100720 0x7f1d5019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.765+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d501010e0 0x7f1d5019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2106392870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50100720 msgr2=0x7f1d5019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50100720 0x7f1d5019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2106392870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d501010e0 msgr2=0x7f1d5019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d501010e0 0x7f1d5019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2106392870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d48009d00 con 0x7f1d50106730 2023-10-26T06:53:30.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d50106730 0x7f1d50196a10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800b070 tx=0x7f1d480077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:30.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d37fff700 1 -- 172.21.15.89:0/2106392870 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4800e040 con 0x7f1d50106730 2023-10-26T06:53:30.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d37fff700 1 -- 172.21.15.89:0/2106392870 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d480094c0 con 0x7f1d50106730 2023-10-26T06:53:30.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d50197620 con 0x7f1d50106730 2023-10-26T06:53:30.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d37fff700 1 -- 172.21.15.89:0/2106392870 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480043a0 con 0x7f1d50106730 2023-10-26T06:53:30.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.766+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d501a38d0 con 0x7f1d50106730 2023-10-26T06:53:30.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.769+0000 7f1d37fff700 1 -- 172.21.15.89:0/2106392870 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1d48004500 con 0x7f1d50106730 2023-10-26T06:53:30.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.771+0000 7f1d68d40b80 1 Processor -- start 2023-10-26T06:53:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.771+0000 7f1d68d40b80 1 -- start start 2023-10-26T06:53:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.771+0000 7f1d37fff700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1d380a1670 0x7f1d380a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d37fff700 1 -- 172.21.15.89:0/2106392870 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1d480cdf30 con 0x7f1d50106730 2023-10-26T06:53:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1d380a1670 0x7f1d380a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f38f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f38646e0 con 0x55f6f38f15d0 2023-10-26T06:53:30.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f384ac60 con 0x55f6f38f6be0 2023-10-26T06:53:30.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f3848580 con 0x55f6f38f7500 2023-10-26T06:53:30.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f38f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f38f1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60916/0 (socket says 172.21.15.89:60916) 2023-10-26T06:53:30.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2425684176 learned_addr learned my addr 172.21.15.89:0/2425684176 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:30.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d4effd700 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.772+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1d380a1670 0x7f1d380a3b30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1d50197bd0 tx=0x7f1d440098d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:30.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2425684176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 msgr2=0x55f6f38f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2425684176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 msgr2=0x55f6f38f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/2425684176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6f38013f0 con 0x55f6f38f15d0 2023-10-26T06:53:30.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f38f1a50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1d480a2d20 tx=0x7f1d4809aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbea8cc157fdbea server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:30.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d357fa700 1 -- 172.21.15.89:0/2425684176 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4809b650 con 0x55f6f38f15d0 2023-10-26T06:53:30.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d357fa700 1 -- 172.21.15.89:0/2425684176 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d4809af50 con 0x55f6f38f15d0 2023-10-26T06:53:30.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.773+0000 7f1d357fa700 1 -- 172.21.15.89:0/2425684176 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d480cb8b0 con 0x55f6f38f15d0 2023-10-26T06:53:30.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 -- 172.21.15.89:0/2425684176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 msgr2=0x55f6f38f1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f38f1a50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1d480a2d20 tx=0x7f1d4809aa90 comp rx=0 tx=0).stop 2023-10-26T06:53:30.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 -- 172.21.15.89:0/2425684176 shutdown_connections 2023-10-26T06:53:30.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f38f1a50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/2425684176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 -- 172.21.15.89:0/2425684176 >> 172.21.15.89:0/2425684176 conn(0x55f6f3865720 msgr2=0x55f6f38ebb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:30.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 -- 172.21.15.89:0/2425684176 shutdown_connections 2023-10-26T06:53:30.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.774+0000 7f1d68d40b80 1 -- 172.21.15.89:0/2425684176 wait complete. 2023-10-26T06:53:30.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.775+0000 7f1d68d40b80 1 -- start start 2023-10-26T06:53:30.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.775+0000 7f1d68d40b80 1 Processor -- start 2023-10-26T06:53:30.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f15d0 0x55f6f38fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f7500 0x55f6f38faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f3843e70 con 0x55f6f38f7500 2023-10-26T06:53:30.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f37e0710 con 0x55f6f38f6be0 2023-10-26T06:53:30.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f37e03b0 con 0x55f6f38f15d0 2023-10-26T06:53:30.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d68d40b80 10 client.0 _ll_register_callbacks cb 0x55f6f372aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:30.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f7500 0x55f6f38faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d4e7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f7500 0x55f6f38faf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60924/0 (socket says 172.21.15.89:60924) 2023-10-26T06:53:30.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/178365105 learned_addr learned my addr 172.21.15.89:0/178365105 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:30.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d4effd700 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f15d0 0x55f6f38fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.776+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/178365105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f15d0 msgr2=0x55f6f38fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f15d0 0x55f6f38fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 msgr2=0x55f6f38fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38fa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d4f7fe700 1 -- 172.21.15.89:0/178365105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d501a38d0 con 0x55f6f38f7500 2023-10-26T06:53:30.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:30.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f7500 0x55f6f38faf50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d4809ae90 tx=0x7f1d480cb620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faf0e90bd0ecb11 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:30.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:30.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d32ff5700 1 -- 172.21.15.89:0/178365105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4809f0a0 con 0x55f6f38f7500 2023-10-26T06:53:30.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d32ff5700 1 -- 172.21.15.89:0/178365105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1d480a3250 con 0x55f6f38f7500 2023-10-26T06:53:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6f3848ef0 con 0x55f6f38f7500 2023-10-26T06:53:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d32ff5700 1 -- 172.21.15.89:0/178365105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d480047b0 con 0x55f6f38f7500 2023-10-26T06:53:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d68d40b80 10 client.16167 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55f6f366ede0 con 0x55f6f38f7500 2023-10-26T06:53:30.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.777+0000 7f1d32ff5700 1 -- 172.21.15.89:0/178365105 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6f3848ef0 con 0x55f6f38f7500 2023-10-26T06:53:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d32ff5700 1 -- 172.21.15.89:0/178365105 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d48024360 con 0x55f6f38f7500 2023-10-26T06:53:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d68d40b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d68d40b80 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:30.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d68d40b80 20 client.16167 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:30.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f390a7a0 0x55f6f390cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d68d40b80 4 client.16167 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:53:30.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6f38c08b0 con 0x55f6f390a7a0 2023-10-26T06:53:30.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.778+0000 7f1d4effd700 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f390a7a0 0x55f6f390cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.779+0000 7f1d4effd700 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f390a7a0 0x55f6f390cb90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16fae2bb868bd5ec server_cookie=aeb5d1ea0afc2c0a in_seq=0 out_seq=0 2023-10-26T06:53:30.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.779+0000 7f1d32ff5700 10 client.16167 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:30.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.779+0000 7f1d32ff5700 1 -- 172.21.15.89:0/178365105 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6f38c08b0 con 0x55f6f390a7a0 2023-10-26T06:53:30.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.779+0000 7f1d32ff5700 10 client.16167 handle_command_reply: tid=0 2023-10-26T06:53:30.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.779+0000 7f1d68d40b80 1 client.16167 shutdown 2023-10-26T06:53:30.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f7500 msgr2=0x55f6f38faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f7500 0x55f6f38faf50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d4809ae90 tx=0x7f1d480cb620 comp rx=0 tx=0).stop 2023-10-26T06:53:30.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 shutdown_connections 2023-10-26T06:53:30.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f7500 0x55f6f38faf50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f390a7a0 0x55f6f390cb90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f15d0 0x55f6f38fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/178365105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f6be0 0x55f6f38fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d32ff5700 0 client.16167 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:30.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 >> 172.21.15.89:0/178365105 conn(0x55f6f3865720 msgr2=0x55f6f38f0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:30.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 shutdown_connections 2023-10-26T06:53:30.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 1 -- 172.21.15.89:0/178365105 wait complete. 2023-10-26T06:53:30.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.780+0000 7f1d68d40b80 20 client.16167 trim_cache size 0 max 16384 2023-10-26T06:53:30.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.797+0000 7f1d68d40b80 1 Processor -- start 2023-10-26T06:53:30.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.797+0000 7f1d68d40b80 1 -- start start 2023-10-26T06:53:30.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.797+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f15d0 0x55f6f3901cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.797+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f6be0 0x55f6f38eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38fa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f36c4920 con 0x55f6f38f6be0 2023-10-26T06:53:30.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f3848ef0 con 0x55f6f38f15d0 2023-10-26T06:53:30.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f38e82e0 con 0x55f6f38f7500 2023-10-26T06:53:30.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f6be0 0x55f6f38eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f6be0 0x55f6f38eb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60932/0 (socket says 172.21.15.89:60932) 2023-10-26T06:53:30.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 -- 172.21.15.89:0/1737328755 learned_addr learned my addr 172.21.15.89:0/1737328755 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:30.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4effd700 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f15d0 0x55f6f3901cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 -- 172.21.15.89:0/1737328755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 msgr2=0x55f6f38fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 -- 172.21.15.89:0/1737328755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f15d0 msgr2=0x55f6f3901cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f15d0 0x55f6f3901cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.798+0000 7f1d4e7fc700 1 -- 172.21.15.89:0/1737328755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6f39070f0 con 0x55f6f38f6be0 2023-10-26T06:53:30.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f6be0 0x55f6f38eb810 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1d44013ad0 tx=0x7f1d44016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17564216e2c1393d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:30.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d35ffb700 1 -- 172.21.15.89:0/1737328755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44003b10 con 0x55f6f38f6be0 2023-10-26T06:53:30.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d35ffb700 1 -- 172.21.15.89:0/1737328755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d44003c70 con 0x55f6f38f6be0 2023-10-26T06:53:30.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d35ffb700 1 -- 172.21.15.89:0/1737328755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44010430 con 0x55f6f38f6be0 2023-10-26T06:53:30.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 -- 172.21.15.89:0/1737328755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f6be0 msgr2=0x55f6f38eb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f6be0 0x55f6f38eb810 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1d44013ad0 tx=0x7f1d44016000 comp rx=0 tx=0).stop 2023-10-26T06:53:30.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 -- 172.21.15.89:0/1737328755 shutdown_connections 2023-10-26T06:53:30.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f7500 0x55f6f38fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f15d0 0x55f6f3901cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/1737328755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f6be0 0x55f6f38eb810 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.816 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 -- 172.21.15.89:0/1737328755 >> 172.21.15.89:0/1737328755 conn(0x55f6f3865720 msgr2=0x55f6f38fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:30.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 -- 172.21.15.89:0/1737328755 shutdown_connections 2023-10-26T06:53:30.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.799+0000 7f1d68d40b80 1 -- 172.21.15.89:0/1737328755 wait complete. 2023-10-26T06:53:30.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.800+0000 7f1d68d40b80 1 -- start start 2023-10-26T06:53:30.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.800+0000 7f1d68d40b80 1 Processor -- start 2023-10-26T06:53:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.800+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f3900e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.800+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f6be0 0x55f6f39013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d68d40b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f7500 0x55f6f39577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f3957d50 con 0x55f6f38f15d0 2023-10-26T06:53:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f3957ec0 con 0x55f6f38f7500 2023-10-26T06:53:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d68d40b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6f3958080 con 0x55f6f38f6be0 2023-10-26T06:53:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d68d40b80 10 client.0 _ll_register_callbacks cb 0x55f6f372aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:30.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f3900e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f3900e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60946/0 (socket says 172.21.15.89:60946) 2023-10-26T06:53:30.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 -- 172.21.15.89:0/3035800956 learned_addr learned my addr 172.21.15.89:0/3035800956 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:30.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f6be0 0x55f6f39013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f7500 0x55f6f39577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 -- 172.21.15.89:0/3035800956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f6be0 msgr2=0x55f6f39013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f6be0 0x55f6f39013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 -- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f7500 msgr2=0x55f6f39577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f7500 0x55f6f39577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4effd700 1 -- 172.21.15.89:0/3035800956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d44019050 con 0x55f6f38f15d0 2023-10-26T06:53:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.801+0000 7f1d4f7fe700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f7500 0x55f6f39577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:30.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d4effd700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f3900e90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55f6f390d440 tx=0x7f1d40010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35a144a906f55e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d32ff5700 1 -- 172.21.15.89:0/3035800956 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40017070 con 0x55f6f38f15d0 2023-10-26T06:53:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d4809af50 con 0x55f6f38f15d0 2023-10-26T06:53:30.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d32ff5700 1 -- 172.21.15.89:0/3035800956 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1d4001fbf0 con 0x55f6f38f15d0 2023-10-26T06:53:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d32ff5700 1 -- 172.21.15.89:0/3035800956 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d4001fdb0 con 0x55f6f38f15d0 2023-10-26T06:53:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d68d40b80 10 client.16173 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:30.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55f6f3657c60 con 0x55f6f38f15d0 2023-10-26T06:53:30.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d32ff5700 1 -- 172.21.15.89:0/3035800956 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d4809af50 con 0x55f6f38f15d0 2023-10-26T06:53:30.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.802+0000 7f1d32ff5700 1 -- 172.21.15.89:0/3035800956 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d4002e070 con 0x55f6f38f15d0 2023-10-26T06:53:30.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d68d40b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:30.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d68d40b80 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:30.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d68d40b80 20 client.16173 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:30.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f3960b60 0x55f6f3962f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:30.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d68d40b80 4 client.16173 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:53:30.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f6f383ce50 con 0x55f6f3960b60 2023-10-26T06:53:30.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f3960b60 0x55f6f3962f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:30.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d32ff5700 10 client.16173 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:30.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.803+0000 7f1d4e7fc700 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f3960b60 0x55f6f3962f80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75857cb34794c95 server_cookie=964fec0b58e6838d in_seq=0 out_seq=0 2023-10-26T06:53:30.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d32ff5700 1 -- 172.21.15.89:0/3035800956 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f6f383ce50 con 0x55f6f3960b60 2023-10-26T06:53:30.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d32ff5700 10 client.16173 handle_command_reply: tid=0 2023-10-26T06:53:30.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 client.16173 shutdown 2023-10-26T06:53:30.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 msgr2=0x55f6f3900e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f3900e90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55f6f390d440 tx=0x7f1d40010710 comp rx=0 tx=0).stop 2023-10-26T06:53:30.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 shutdown_connections 2023-10-26T06:53:30.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f6f38f15d0 0x55f6f3900e90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55f6f3960b60 0x55f6f3962f80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6f38f7500 0x55f6f39577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 --2- 172.21.15.89:0/3035800956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6f38f6be0 0x55f6f39013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d32ff5700 0 client.16173 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:30.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 >> 172.21.15.89:0/3035800956 conn(0x55f6f3865720 msgr2=0x55f6f38f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:30.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 shutdown_connections 2023-10-26T06:53:30.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 1 -- 172.21.15.89:0/3035800956 wait complete. 2023-10-26T06:53:30.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.804+0000 7f1d68d40b80 20 client.16173 trim_cache size 0 max 16384 2023-10-26T06:53:30.832 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:53:30.832 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:53:30.832 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:53:30.832 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:53:30.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1d380a1670 msgr2=0x7f1d380a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1d380a1670 0x7f1d380a3b30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1d50197bd0 tx=0x7f1d440098d0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d50106730 msgr2=0x7f1d50196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:30.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d50106730 0x7f1d50196a10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800b070 tx=0x7f1d480077f0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 shutdown_connections 2023-10-26T06:53:30.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1d380a1670 0x7f1d380a3b30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d50106730 0x7f1d50196a10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50100720 0x7f1d5019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 --2- 172.21.15.89:0/2106392870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d501010e0 0x7f1d5019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:30.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 >> 172.21.15.89:0/2106392870 conn(0x7f1d500fc1b0 msgr2=0x7f1d500fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:30.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 shutdown_connections 2023-10-26T06:53:30.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:30.805+0000 7f1d553ff700 1 -- 172.21.15.89:0/2106392870 wait complete. 2023-10-26T06:53:30.837 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:53:30.837 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:53:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 -- 172.21.15.89:0/32421679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981006a0 msgr2=0x7fdb98100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 --2- 172.21.15.89:0/32421679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981006a0 0x7fdb98100b20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009990 tx=0x7fdb88009ca0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 -- 172.21.15.89:0/32421679 shutdown_connections 2023-10-26T06:53:31.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 --2- 172.21.15.89:0/32421679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb98101060 0x7fdb9810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 --2- 172.21.15.89:0/32421679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb981066b0 0x7fdb98106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 --2- 172.21.15.89:0/32421679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981006a0 0x7fdb98100b20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 -- 172.21.15.89:0/32421679 >> 172.21.15.89:0/32421679 conn(0x7fdb980fc170 msgr2=0x7fdb980fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 -- 172.21.15.89:0/32421679 shutdown_connections 2023-10-26T06:53:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.034+0000 7fdba03ff700 1 -- 172.21.15.89:0/32421679 wait complete. 2023-10-26T06:53:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.035+0000 7fdba03ff700 1 Processor -- start 2023-10-26T06:53:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.035+0000 7fdba03ff700 1 -- start start 2023-10-26T06:53:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdba03ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb981006a0 0x7fdb9819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdba03ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb98101060 0x7fdb9819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdba03ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981066b0 0x7fdb98196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdba03ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98196fd0 con 0x7fdb981066b0 2023-10-26T06:53:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdba03ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98197140 con 0x7fdb981006a0 2023-10-26T06:53:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdba03ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98197300 con 0x7fdb98101060 2023-10-26T06:53:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981066b0 0x7fdb98196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9d99a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb98101060 0x7fdb9819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9d99a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb98101060 0x7fdb9819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:43402/0 (socket says 172.21.15.89:43402) 2023-10-26T06:53:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981066b0 0x7fdb98196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60964/0 (socket says 172.21.15.89:60964) 2023-10-26T06:53:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9d99a700 1 -- 172.21.15.89:0/3497014841 learned_addr learned my addr 172.21.15.89:0/3497014841 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e19b700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb981006a0 0x7fdb9819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 -- 172.21.15.89:0/3497014841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb98101060 msgr2=0x7fdb9819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb98101060 0x7fdb9819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 -- 172.21.15.89:0/3497014841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb981006a0 msgr2=0x7fdb9819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb981006a0 0x7fdb9819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 -- 172.21.15.89:0/3497014841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94009520 con 0x7fdb981066b0 2023-10-26T06:53:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.036+0000 7fdb9e99c700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981066b0 0x7fdb98196970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000efc0 tx=0x7fdb9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.037+0000 7fdb8f7fe700 1 -- 172.21.15.89:0/3497014841 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90018070 con 0x7fdb981066b0 2023-10-26T06:53:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.037+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb880095f0 con 0x7fdb981066b0 2023-10-26T06:53:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.037+0000 7fdb8f7fe700 1 -- 172.21.15.89:0/3497014841 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb9000f040 con 0x7fdb981066b0 2023-10-26T06:53:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.037+0000 7fdb8f7fe700 1 -- 172.21.15.89:0/3497014841 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000cd40 con 0x7fdb981066b0 2023-10-26T06:53:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.037+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb981a3650 con 0x7fdb981066b0 2023-10-26T06:53:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.038+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb98108c30 con 0x7fdb981066b0 2023-10-26T06:53:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.040+0000 7fdb8f7fe700 1 -- 172.21.15.89:0/3497014841 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdb90003a00 con 0x7fdb981066b0 2023-10-26T06:53:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.041+0000 7fdb8f7fe700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdb840a15f0 0x7fdb840a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.041+0000 7fdb8f7fe700 1 -- 172.21.15.89:0/3497014841 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fdb900ccd40 con 0x7fdb981066b0 2023-10-26T06:53:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.041+0000 7fdb9e19b700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdb840a15f0 0x7fdb840a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.042+0000 7fdb9e19b700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdb840a15f0 0x7fdb840a3ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009d70 tx=0x7fdb94009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.046+0000 7fdb8f7fe700 1 -- 172.21.15.89:0/3497014841 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb900960a0 con 0x7fdb981066b0 2023-10-26T06:53:31.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.293+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb98197e30 con 0x7fdb981066b0 2023-10-26T06:53:31.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.294+0000 7fdb8f7fe700 1 -- 172.21.15.89:0/3497014841 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fdb900996c0 con 0x7fdb981066b0 2023-10-26T06:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:53:31.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdb840a15f0 msgr2=0x7fdb840a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdb840a15f0 0x7fdb840a3ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009d70 tx=0x7fdb94009260 comp rx=0 tx=0).stop 2023-10-26T06:53:31.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981066b0 msgr2=0x7fdb98196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981066b0 0x7fdb98196970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000efc0 tx=0x7fdb9000c3a0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 shutdown_connections 2023-10-26T06:53:31.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdb840a15f0 0x7fdb840a3ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb981066b0 0x7fdb98196970 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb981006a0 0x7fdb9819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 --2- 172.21.15.89:0/3497014841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb98101060 0x7fdb9819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 >> 172.21.15.89:0/3497014841 conn(0x7fdb980fc170 msgr2=0x7fdb980fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.296+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 shutdown_connections 2023-10-26T06:53:31.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.297+0000 7fdba03ff700 1 -- 172.21.15.89:0/3497014841 wait complete. 2023-10-26T06:53:31.302 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:53:31.317 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:53:31.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 -- 172.21.15.89:0/1042194674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105b30 msgr2=0x7fbf1c10d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.520 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 --2- 172.21.15.89:0/1042194674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105b30 0x7fbf1c10d540 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009fc0 tx=0x7fbf1400b330 comp rx=0 tx=0).stop 2023-10-26T06:53:31.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 -- 172.21.15.89:0/1042194674 shutdown_connections 2023-10-26T06:53:31.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 --2- 172.21.15.89:0/1042194674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105b30 0x7fbf1c10d540 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 --2- 172.21.15.89:0/1042194674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf1c067d40 0x7fbf1c068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 --2- 172.21.15.89:0/1042194674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf1c105170 0x7fbf1c1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 -- 172.21.15.89:0/1042194674 >> 172.21.15.89:0/1042194674 conn(0x7fbf1c0767f0 msgr2=0x7fbf1c076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 -- 172.21.15.89:0/1042194674 shutdown_connections 2023-10-26T06:53:31.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.517+0000 7fbf229ff700 1 -- 172.21.15.89:0/1042194674 wait complete. 2023-10-26T06:53:31.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.518+0000 7fbf229ff700 1 Processor -- start 2023-10-26T06:53:31.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.518+0000 7fbf229ff700 1 -- start start 2023-10-26T06:53:31.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf229ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf1c067d40 0x7fbf1c0ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf229ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105170 0x7fbf1c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf229ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf1c105b30 0x7fbf1c100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf229ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1a50a0 con 0x7fbf1c105170 2023-10-26T06:53:31.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf229ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1a5210 con 0x7fbf1c105b30 2023-10-26T06:53:31.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf229ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1a53d0 con 0x7fbf1c067d40 2023-10-26T06:53:31.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105170 0x7fbf1c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105170 0x7fbf1c0ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60976/0 (socket says 172.21.15.89:60976) 2023-10-26T06:53:31.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf13fff700 1 -- 172.21.15.89:0/4137142234 learned_addr learned my addr 172.21.15.89:0/4137142234 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:31.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf20f9c700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf1c105b30 0x7fbf1c100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.519+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf1c067d40 0x7fbf1c0ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf13fff700 1 -- 172.21.15.89:0/4137142234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf1c067d40 msgr2=0x7fbf1c0ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf13fff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf1c067d40 0x7fbf1c0ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf13fff700 1 -- 172.21.15.89:0/4137142234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf1c105b30 msgr2=0x7fbf1c100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf13fff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf1c105b30 0x7fbf1c100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf13fff700 1 -- 172.21.15.89:0/4137142234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf14009c40 con 0x7fbf1c105170 2023-10-26T06:53:31.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf13fff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105170 0x7fbf1c0ffbf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400e9c0 tx=0x7fbf0400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf19ffb700 1 -- 172.21.15.89:0/4137142234 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04004a30 con 0x7fbf1c105170 2023-10-26T06:53:31.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf19ffb700 1 -- 172.21.15.89:0/4137142234 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf0400cc80 con 0x7fbf1c105170 2023-10-26T06:53:31.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c1a56b0 con 0x7fbf1c105170 2023-10-26T06:53:31.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf19ffb700 1 -- 172.21.15.89:0/4137142234 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040054e0 con 0x7fbf1c105170 2023-10-26T06:53:31.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.520+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c1a5bc0 con 0x7fbf1c105170 2023-10-26T06:53:31.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.524+0000 7fbf19ffb700 1 -- 172.21.15.89:0/4137142234 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbf04004b90 con 0x7fbf1c105170 2023-10-26T06:53:31.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 Processor -- start 2023-10-26T06:53:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf19ffb700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf080a15f0 0x7fbf080a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 -- start start 2023-10-26T06:53:31.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf19ffb700 1 -- 172.21.15.89:0/4137142234 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fbf04014070 con 0x7fbf1c105170 2023-10-26T06:53:31.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf20f9c700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf080a15f0 0x7fbf080a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c1623fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c161e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c161ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c15916e0 con 0x55e0c1623be0 2023-10-26T06:53:31.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c1577c60 con 0x55e0c1624500 2023-10-26T06:53:31.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.525+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c1575580 con 0x55e0c161e5d0 2023-10-26T06:53:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c1623fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c1623fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60986/0 (socket says 172.21.15.89:60986) 2023-10-26T06:53:31.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 -- 172.21.15.89:0/1900657200 learned_addr learned my addr 172.21.15.89:0/1900657200 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf13fff700 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c161e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf20f9c700 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c161ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 -- 172.21.15.89:0/1900657200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 msgr2=0x55e0c161ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c161ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 -- 172.21.15.89:0/1900657200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 msgr2=0x55e0c161e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c161e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 -- 172.21.15.89:0/1900657200 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0c152e3f0 con 0x55e0c1623be0 2023-10-26T06:53:31.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf13fff700 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c161e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:31.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf20f9c700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf080a15f0 0x7fbf080a3ab0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b6c0 tx=0x7fbf140074c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.526+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c1623fc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009d80 tx=0x7fbf0c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5be41d824d75fb server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf127fc700 1 -- 172.21.15.89:0/1900657200 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c004230 con 0x55e0c1623be0 2023-10-26T06:53:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf127fc700 1 -- 172.21.15.89:0/1900657200 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf0c004390 con 0x55e0c1623be0 2023-10-26T06:53:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf127fc700 1 -- 172.21.15.89:0/1900657200 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c019880 con 0x55e0c1623be0 2023-10-26T06:53:31.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1900657200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 msgr2=0x55e0c1623fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c1623fc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009d80 tx=0x7fbf0c011720 comp rx=0 tx=0).stop 2023-10-26T06:53:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1900657200 shutdown_connections 2023-10-26T06:53:31.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c1623fc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c161ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1900657200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c161e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1900657200 >> 172.21.15.89:0/1900657200 conn(0x55e0c1592720 msgr2=0x55e0c1618b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1900657200 shutdown_connections 2023-10-26T06:53:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.527+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1900657200 wait complete. 2023-10-26T06:53:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.528+0000 7fbf363cbb80 1 -- start start 2023-10-26T06:53:31.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.528+0000 7fbf363cbb80 1 Processor -- start 2023-10-26T06:53:31.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c162a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c16277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c1627db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c1570e70 con 0x55e0c1623be0 2023-10-26T06:53:31.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c150d710 con 0x55e0c1624500 2023-10-26T06:53:31.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c150d3b0 con 0x55e0c161e5d0 2023-10-26T06:53:31.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf363cbb80 10 client.0 _ll_register_callbacks cb 0x55e0c1457a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:31.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c16277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf1bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c162a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c16277a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60996/0 (socket says 172.21.15.89:60996) 2023-10-26T06:53:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 -- 172.21.15.89:0/1024562449 learned_addr learned my addr 172.21.15.89:0/1024562449 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf20f9c700 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c1627db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 -- 172.21.15.89:0/1024562449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 msgr2=0x55e0c162a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c162a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 -- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 msgr2=0x55e0c1627db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c1627db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.529+0000 7fbf13fff700 1 -- 172.21.15.89:0/1024562449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c1a5bc0 con 0x55e0c1623be0 2023-10-26T06:53:31.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbf13fff700 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c16277a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbf040ca910 tx=0x7fbf0400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a92aaf2dbe07236 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbefb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:31.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbefb7fe700 1 -- 172.21.15.89:0/1024562449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04016530 con 0x55e0c1623be0 2023-10-26T06:53:31.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbefb7fe700 1 -- 172.21.15.89:0/1024562449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fbf0409a140 con 0x55e0c1623be0 2023-10-26T06:53:31.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0c1575ef0 con 0x55e0c1623be0 2023-10-26T06:53:31.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbefb7fe700 1 -- 172.21.15.89:0/1024562449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf0409e290 con 0x55e0c1623be0 2023-10-26T06:53:31.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbf363cbb80 10 client.16191 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:31.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fbf0c0099e0 con 0x55e0c1623be0 2023-10-26T06:53:31.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbefb7fe700 1 -- 172.21.15.89:0/1024562449 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0c1575ef0 con 0x55e0c1623be0 2023-10-26T06:53:31.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbefb7fe700 1 -- 172.21.15.89:0/1024562449 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf04099630 con 0x55e0c1623be0 2023-10-26T06:53:31.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbf363cbb80 10 client.16191 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:31.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbf363cbb80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:31.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.530+0000 7fbf363cbb80 20 client.16191 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.531+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c1637e20 0x55e0c163a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.531+0000 7fbf363cbb80 4 client.16191 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:53:31.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.531+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0c15ed8b0 con 0x55e0c1637e20 2023-10-26T06:53:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.531+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c1637e20 0x55e0c163a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.531+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c1637e20 0x55e0c163a240 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8748daefe31a6fb0 server_cookie=25bcc4950129bc14 in_seq=0 out_seq=0 2023-10-26T06:53:31.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.531+0000 7fbefb7fe700 10 client.16191 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbefb7fe700 1 -- 172.21.15.89:0/1024562449 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e0c15ed8b0 con 0x55e0c1637e20 2023-10-26T06:53:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbefb7fe700 10 client.16191 handle_command_reply: tid=0 2023-10-26T06:53:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 client.16191 shutdown 2023-10-26T06:53:31.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 msgr2=0x55e0c16277a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c16277a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbf040ca910 tx=0x7fbf0400cf30 comp rx=0 tx=0).stop 2023-10-26T06:53:31.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 shutdown_connections 2023-10-26T06:53:31.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c1637e20 0x55e0c163a240 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1624500 0x55e0c1627db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c161e5d0 0x55e0c162a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/1024562449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c1623be0 0x55e0c16277a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 >> 172.21.15.89:0/1024562449 conn(0x55e0c1592720 msgr2=0x55e0c161d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbefb7fe700 0 client.16191 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:31.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.532+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 shutdown_connections 2023-10-26T06:53:31.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.533+0000 7fbf363cbb80 1 -- 172.21.15.89:0/1024562449 wait complete. 2023-10-26T06:53:31.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.533+0000 7fbf363cbb80 20 client.16191 trim_cache size 0 max 16384 2023-10-26T06:53:31.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 Processor -- start 2023-10-26T06:53:31.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 -- start start 2023-10-26T06:53:31.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c162eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c16216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c1621bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c13f1920 con 0x55e0c161e5d0 2023-10-26T06:53:31.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c1623ff0 con 0x55e0c1623be0 2023-10-26T06:53:31.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c161e090 con 0x55e0c1624500 2023-10-26T06:53:31.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c162eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c162eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32772/0 (socket says 172.21.15.89:32772) 2023-10-26T06:53:31.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.546+0000 7fbf1bfff700 1 -- 172.21.15.89:0/3355890116 learned_addr learned my addr 172.21.15.89:0/3355890116 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:31.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf13fff700 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c16216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf1bfff700 1 -- 172.21.15.89:0/3355890116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 msgr2=0x55e0c1621bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c1621bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf1bfff700 1 -- 172.21.15.89:0/3355890116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 msgr2=0x55e0c16216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c16216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf1bfff700 1 -- 172.21.15.89:0/3355890116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0c163a5f0 con 0x55e0c161e5d0 2023-10-26T06:53:31.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c162eeb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c01b040 tx=0x7fbf0c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb297c22f4ad200 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf12ffd700 1 -- 172.21.15.89:0/3355890116 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c029040 con 0x55e0c161e5d0 2023-10-26T06:53:31.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf12ffd700 1 -- 172.21.15.89:0/3355890116 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf0c004500 con 0x55e0c161e5d0 2023-10-26T06:53:31.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.547+0000 7fbf12ffd700 1 -- 172.21.15.89:0/3355890116 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c02aad0 con 0x55e0c161e5d0 2023-10-26T06:53:31.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 -- 172.21.15.89:0/3355890116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 msgr2=0x55e0c162eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c162eeb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c01b040 tx=0x7fbf0c011720 comp rx=0 tx=0).stop 2023-10-26T06:53:31.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 -- 172.21.15.89:0/3355890116 shutdown_connections 2023-10-26T06:53:31.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c162eeb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c1621bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/3355890116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c16216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 -- 172.21.15.89:0/3355890116 >> 172.21.15.89:0/3355890116 conn(0x55e0c1592720 msgr2=0x55e0c16260b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 -- 172.21.15.89:0/3355890116 shutdown_connections 2023-10-26T06:53:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 -- 172.21.15.89:0/3355890116 wait complete. 2023-10-26T06:53:31.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 -- start start 2023-10-26T06:53:31.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.548+0000 7fbf363cbb80 1 Processor -- start 2023-10-26T06:53:31.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c1625340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c162e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf363cbb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c162e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c162ee40 con 0x55e0c161e5d0 2023-10-26T06:53:31.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c162efb0 con 0x55e0c1623be0 2023-10-26T06:53:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf363cbb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0c168bc60 con 0x55e0c1624500 2023-10-26T06:53:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf363cbb80 10 client.0 _ll_register_callbacks cb 0x55e0c1457a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c1625340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf1bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c1625340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32784/0 (socket says 172.21.15.89:32784) 2023-10-26T06:53:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf1bfff700 1 -- 172.21.15.89:0/2853761274 learned_addr learned my addr 172.21.15.89:0/2853761274 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf13fff700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c162e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf20f9c700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c162e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf1bfff700 1 -- 172.21.15.89:0/2853761274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 msgr2=0x55e0c162e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c162e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf1bfff700 1 -- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 msgr2=0x55e0c162e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.549+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c162e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbf1bfff700 1 -- 172.21.15.89:0/2853761274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0c152e3f0 con 0x55e0c161e5d0 2023-10-26T06:53:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbf20f9c700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c162e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbf1bfff700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c1625340 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009750 tx=0x7fbf0c0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f306ee254be1d2c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbefb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:31.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbefb7fe700 1 -- 172.21.15.89:0/2853761274 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c029070 con 0x55e0c161e5d0 2023-10-26T06:53:31.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbefb7fe700 1 -- 172.21.15.89:0/2853761274 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fbf0c004670 con 0x55e0c161e5d0 2023-10-26T06:53:31.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0c161f040 con 0x55e0c161e5d0 2023-10-26T06:53:31.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbefb7fe700 1 -- 172.21.15.89:0/2853761274 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf0c004830 con 0x55e0c161e5d0 2023-10-26T06:53:31.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbf363cbb80 10 client.16197 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:31.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e0c139bde0 con 0x55e0c161e5d0 2023-10-26T06:53:31.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.550+0000 7fbefb7fe700 1 -- 172.21.15.89:0/2853761274 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf0c048020 con 0x55e0c161e5d0 2023-10-26T06:53:31.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbefb7fe700 1 -- 172.21.15.89:0/2853761274 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf0c011440 con 0x55e0c161e5d0 2023-10-26T06:53:31.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbf363cbb80 10 client.16197 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:31.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbf363cbb80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:31.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbf363cbb80 20 client.16197 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:31.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c168d130 0x55e0c168f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:31.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbf363cbb80 4 client.16197 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:53:31.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fbf0c004390 con 0x55e0c168d130 2023-10-26T06:53:31.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.551+0000 7fbf13fff700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c168d130 0x55e0c168f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:31.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.552+0000 7fbf13fff700 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c168d130 0x55e0c168f520 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e20a40d699a191 server_cookie=34d5256bd7c3ceb6 in_seq=0 out_seq=0 2023-10-26T06:53:31.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.552+0000 7fbefb7fe700 10 client.16197 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:31.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.552+0000 7fbefb7fe700 1 -- 172.21.15.89:0/2853761274 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fbf0c004390 con 0x55e0c168d130 2023-10-26T06:53:31.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.552+0000 7fbefb7fe700 10 client.16197 handle_command_reply: tid=0 2023-10-26T06:53:31.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.552+0000 7fbf363cbb80 1 client.16197 shutdown 2023-10-26T06:53:31.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.552+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 msgr2=0x55e0c1625340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.552+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c1625340 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009750 tx=0x7fbf0c0050d0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 shutdown_connections 2023-10-26T06:53:31.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0c161e5d0 0x55e0c1625340 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e0c168d130 0x55e0c168f520 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0c1624500 0x55e0c162e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 --2- 172.21.15.89:0/2853761274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0c1623be0 0x55e0c162e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbefb7fe700 0 client.16197 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:31.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 >> 172.21.15.89:0/2853761274 conn(0x55e0c1592720 msgr2=0x55e0c16124b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 shutdown_connections 2023-10-26T06:53:31.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 1 -- 172.21.15.89:0/2853761274 wait complete. 2023-10-26T06:53:31.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf363cbb80 20 client.16197 trim_cache size 0 max 16384 2023-10-26T06:53:31.598 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:53:31.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf080a15f0 msgr2=0x7fbf080a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf229ff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf080a15f0 0x7fbf080a3ab0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b6c0 tx=0x7fbf140074c0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105170 msgr2=0x7fbf1c0ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:31.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.553+0000 7fbf229ff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105170 0x7fbf1c0ffbf0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400e9c0 tx=0x7fbf0400ecd0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 shutdown_connections 2023-10-26T06:53:31.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fbf080a15f0 0x7fbf080a3ab0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf1c105b30 0x7fbf1c100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf1c067d40 0x7fbf1c0ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 --2- 172.21.15.89:0/4137142234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf1c105170 0x7fbf1c0ffbf0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:31.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 >> 172.21.15.89:0/4137142234 conn(0x7fbf1c0767f0 msgr2=0x7fbf1c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:31.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 shutdown_connections 2023-10-26T06:53:31.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:31.554+0000 7fbf229ff700 1 -- 172.21.15.89:0/4137142234 wait complete. 2023-10-26T06:53:31.603 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:31 smithi089 ceph-mon[95031]: pgmap v510: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:31.603 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:31 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3497014841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:53:31.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:31 smithi135 ceph-mon[131403]: pgmap v510: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:31 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3497014841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:53:31.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:31 smithi192 ceph-mon[131661]: pgmap v510: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:31.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:31 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3497014841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:53:32.605 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:53:32.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:53:32.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 -- 172.21.15.89:0/3281574560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7808106730 msgr2=0x7f7808106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3281574560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7808106730 0x7f7808106b10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009990 tx=0x7f77f8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 -- 172.21.15.89:0/3281574560 shutdown_connections 2023-10-26T06:53:32.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3281574560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78081010e0 0x7f780810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3281574560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7808106730 0x7f7808106b10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3281574560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7808100720 0x7f7808100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 -- 172.21.15.89:0/3281574560 >> 172.21.15.89:0/3281574560 conn(0x7f78080fc1b0 msgr2=0x7f78080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:32.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 -- 172.21.15.89:0/3281574560 shutdown_connections 2023-10-26T06:53:32.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.818+0000 7f780d1ff700 1 -- 172.21.15.89:0/3281574560 wait complete. 2023-10-26T06:53:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 Processor -- start 2023-10-26T06:53:32.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 -- start start 2023-10-26T06:53:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7808100720 0x7f7808198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78081010e0 0x7f78081993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7808106730 0x7f780819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819dc90 con 0x7f7808106730 2023-10-26T06:53:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819de00 con 0x7f78081010e0 2023-10-26T06:53:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.819+0000 7f780d1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819dfc0 con 0x7f7808100720 2023-10-26T06:53:32.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7808106730 0x7f780819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7808106730 0x7f780819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32798/0 (socket says 172.21.15.89:32798) 2023-10-26T06:53:32.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 -- 172.21.15.89:0/3015423217 learned_addr learned my addr 172.21.15.89:0/3015423217 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:32.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780659c700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78081010e0 0x7f78081993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f7806d9d700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7808100720 0x7f7808198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 -- 172.21.15.89:0/3015423217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7808100720 msgr2=0x7f7808198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7808100720 0x7f7808198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 -- 172.21.15.89:0/3015423217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78081010e0 msgr2=0x7f78081993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78081010e0 0x7f78081993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 -- 172.21.15.89:0/3015423217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80095f0 con 0x7f7808106730 2023-10-26T06:53:32.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f780759e700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7808106730 0x7f780819d650 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f780000ef30 tx=0x7f780000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.820+0000 7f77effff700 1 -- 172.21.15.89:0/3015423217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7800018070 con 0x7f7808106730 2023-10-26T06:53:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.821+0000 7f77effff700 1 -- 172.21.15.89:0/3015423217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780000f040 con 0x7f7808106730 2023-10-26T06:53:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.821+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780819e240 con 0x7f7808106730 2023-10-26T06:53:32.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.821+0000 7f77effff700 1 -- 172.21.15.89:0/3015423217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780000cd00 con 0x7f7808106730 2023-10-26T06:53:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.821+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78081a5ae0 con 0x7f7808106730 2023-10-26T06:53:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.824+0000 7f77effff700 1 -- 172.21.15.89:0/3015423217 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7800003a00 con 0x7f7808106730 2023-10-26T06:53:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 Processor -- start 2023-10-26T06:53:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f77effff700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77f00a15f0 0x7f77f00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 -- start start 2023-10-26T06:53:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f77effff700 1 -- 172.21.15.89:0/3015423217 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f78000ccc20 con 0x7f7808106730 2023-10-26T06:53:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f780659c700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77f00a15f0 0x7f77f00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e8dc60 0x5602d0e8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e88110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e88650 0x5602d0e88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0dfb760 con 0x5602d0e8dc60 2023-10-26T06:53:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0de1ce0 con 0x5602d0e88650 2023-10-26T06:53:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.825+0000 7f7820b16b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0ddf600 con 0x5602d0e8e580 2023-10-26T06:53:32.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e8dc60 0x5602d0e8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780759e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e88650 0x5602d0e88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e8dc60 0x5602d0e8e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32806/0 (socket says 172.21.15.89:32806) 2023-10-26T06:53:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f7806d9d700 1 -- 172.21.15.89:0/2440381413 learned_addr learned my addr 172.21.15.89:0/2440381413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780659c700 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e88110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780659c700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77f00a15f0 0x7f77f00a3ab0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f780819a360 tx=0x7f77fc007f80 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780759e700 1 -- 172.21.15.89:0/2440381413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 msgr2=0x5602d0e88110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780759e700 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e88110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780759e700 1 -- 172.21.15.89:0/2440381413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e8dc60 msgr2=0x5602d0e8e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780759e700 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e8dc60 0x5602d0e8e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780759e700 1 -- 172.21.15.89:0/2440381413 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602d0d98470 con 0x5602d0e88650 2023-10-26T06:53:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f780759e700 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e88650 0x5602d0e88ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f78000ca320 tx=0x7f7800099780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76322016d0eacd22 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f77ed7fa700 1 -- 172.21.15.89:0/2440381413 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7800004ad0 con 0x5602d0e88650 2023-10-26T06:53:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f77ed7fa700 1 -- 172.21.15.89:0/2440381413 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780001b320 con 0x5602d0e88650 2023-10-26T06:53:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.826+0000 7f77ed7fa700 1 -- 172.21.15.89:0/2440381413 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7800099ea0 con 0x5602d0e88650 2023-10-26T06:53:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 -- 172.21.15.89:0/2440381413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e88650 msgr2=0x5602d0e88ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e88650 0x5602d0e88ad0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f78000ca320 tx=0x7f7800099780 comp rx=0 tx=0).stop 2023-10-26T06:53:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 -- 172.21.15.89:0/2440381413 shutdown_connections 2023-10-26T06:53:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e8dc60 0x5602d0e8e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e88650 0x5602d0e88ad0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 --2- 172.21.15.89:0/2440381413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e88110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 -- 172.21.15.89:0/2440381413 >> 172.21.15.89:0/2440381413 conn(0x5602d0dfc7a0 msgr2=0x5602d0e82be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 -- 172.21.15.89:0/2440381413 shutdown_connections 2023-10-26T06:53:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.827+0000 7f7820b16b80 1 -- 172.21.15.89:0/2440381413 wait complete. 2023-10-26T06:53:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.828+0000 7f7820b16b80 1 -- start start 2023-10-26T06:53:32.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.828+0000 7f7820b16b80 1 Processor -- start 2023-10-26T06:53:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8dc60 0x5602d0e917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8e580 0x5602d0e91e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7820b16b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0ddaef0 con 0x5602d0e88650 2023-10-26T06:53:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7820b16b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0d77790 con 0x5602d0e8e580 2023-10-26T06:53:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7820b16b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0d77430 con 0x5602d0e8dc60 2023-10-26T06:53:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7820b16b80 10 client.0 _ll_register_callbacks cb 0x5602d0cc1b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e944c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32812/0 (socket says 172.21.15.89:32812) 2023-10-26T06:53:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 -- 172.21.15.89:0/3252483420 learned_addr learned my addr 172.21.15.89:0/3252483420 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:32.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f780759e700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8e580 0x5602d0e91e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f780659c700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8dc60 0x5602d0e917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 -- 172.21.15.89:0/3252483420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8dc60 msgr2=0x5602d0e917b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.853 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8dc60 0x5602d0e917b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 -- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8e580 msgr2=0x5602d0e91e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8e580 0x5602d0e91e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.854 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f7806d9d700 1 -- 172.21.15.89:0/3252483420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78081a5ae0 con 0x5602d0e88650 2023-10-26T06:53:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.829+0000 7f780659c700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8dc60 0x5602d0e917b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f7806d9d700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e944c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009990 tx=0x7f77f8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11711d51ecb3c33f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f77eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f77eaff5700 1 -- 172.21.15.89:0/3252483420 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f801d070 con 0x5602d0e88650 2023-10-26T06:53:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f77eaff5700 1 -- 172.21.15.89:0/3252483420 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f77f8004500 con 0x5602d0e88650 2023-10-26T06:53:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602d0ddff70 con 0x5602d0e88650 2023-10-26T06:53:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f77eaff5700 1 -- 172.21.15.89:0/3252483420 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77f800bec0 con 0x5602d0e88650 2023-10-26T06:53:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f7820b16b80 10 client.16209 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5602d0c05e60 con 0x5602d0e88650 2023-10-26T06:53:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f77eaff5700 1 -- 172.21.15.89:0/3252483420 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602d0ddff70 con 0x5602d0e88650 2023-10-26T06:53:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.830+0000 7f77eaff5700 1 -- 172.21.15.89:0/3252483420 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77f800f850 con 0x5602d0e88650 2023-10-26T06:53:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.831+0000 7f7820b16b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.831+0000 7f7820b16b80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.831+0000 7f7820b16b80 20 client.16209 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.831+0000 7f7820b16b80 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ea2010 0x5602d0ea4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.831+0000 7f7820b16b80 4 client.16209 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:53:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.831+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602d0e57930 con 0x5602d0ea2010 2023-10-26T06:53:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.831+0000 7f780659c700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ea2010 0x5602d0ea4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.832+0000 7f780659c700 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ea2010 0x5602d0ea4430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe8475115a4bb09 server_cookie=c6a8a1843768986b in_seq=0 out_seq=0 2023-10-26T06:53:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.832+0000 7f77eaff5700 10 client.16209 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.832+0000 7f77eaff5700 1 -- 172.21.15.89:0/3252483420 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5602d0e57930 con 0x5602d0ea2010 2023-10-26T06:53:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.832+0000 7f77eaff5700 10 client.16209 handle_command_reply: tid=0 2023-10-26T06:53:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.832+0000 7f7820b16b80 1 client.16209 shutdown 2023-10-26T06:53:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 msgr2=0x5602d0e944c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e944c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009990 tx=0x7f77f8005bc0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 shutdown_connections 2023-10-26T06:53:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e944c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ea2010 0x5602d0ea4430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8e580 0x5602d0e91e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 --2- 172.21.15.89:0/3252483420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8dc60 0x5602d0e917b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 >> 172.21.15.89:0/3252483420 conn(0x5602d0dfc7a0 msgr2=0x5602d0e87100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f77eaff5700 0 client.16209 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 shutdown_connections 2023-10-26T06:53:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 1 -- 172.21.15.89:0/3252483420 wait complete. 2023-10-26T06:53:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.833+0000 7f7820b16b80 20 client.16209 trim_cache size 0 max 16384 2023-10-26T06:53:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.850+0000 7f7820b16b80 1 Processor -- start 2023-10-26T06:53:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.850+0000 7f7820b16b80 1 -- start start 2023-10-26T06:53:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.850+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e98f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e8b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e8bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7820b16b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0c5b9a0 con 0x5602d0e88650 2023-10-26T06:53:32.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7820b16b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0e83920 con 0x5602d0e8dc60 2023-10-26T06:53:32.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7820b16b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0e83120 con 0x5602d0e8e580 2023-10-26T06:53:32.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e98f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e98f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32828/0 (socket says 172.21.15.89:32828) 2023-10-26T06:53:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 -- 172.21.15.89:0/1726539037 learned_addr learned my addr 172.21.15.89:0/1726539037 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f780659c700 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e8b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f780759e700 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e8bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 -- 172.21.15.89:0/1726539037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 msgr2=0x5602d0e8bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e8bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 -- 172.21.15.89:0/1726539037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 msgr2=0x5602d0e8b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e8b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 -- 172.21.15.89:0/1726539037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602d0beece0 con 0x5602d0e88650 2023-10-26T06:53:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f780659c700 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e8b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.851+0000 7f7806d9d700 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e98f30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009340 tx=0x7f77f80244b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6229e8f5d291297 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f77edffb700 1 -- 172.21.15.89:0/1726539037 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f801d070 con 0x5602d0e88650 2023-10-26T06:53:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f77edffb700 1 -- 172.21.15.89:0/1726539037 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77f801e040 con 0x5602d0e88650 2023-10-26T06:53:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f77edffb700 1 -- 172.21.15.89:0/1726539037 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8024ab0 con 0x5602d0e88650 2023-10-26T06:53:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 -- 172.21.15.89:0/1726539037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 msgr2=0x5602d0e98f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e98f30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009340 tx=0x7f77f80244b0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 -- 172.21.15.89:0/1726539037 shutdown_connections 2023-10-26T06:53:32.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e98f30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0e8bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1726539037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e8b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 -- 172.21.15.89:0/1726539037 >> 172.21.15.89:0/1726539037 conn(0x5602d0dfc7a0 msgr2=0x5602d0e900c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:32.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 -- 172.21.15.89:0/1726539037 shutdown_connections 2023-10-26T06:53:32.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.852+0000 7f7820b16b80 1 -- 172.21.15.89:0/1726539037 wait complete. 2023-10-26T06:53:32.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 -- start start 2023-10-26T06:53:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 Processor -- start 2023-10-26T06:53:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e98930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0eefe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0ef0380 con 0x5602d0e88650 2023-10-26T06:53:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0ef04f0 con 0x5602d0e8dc60 2023-10-26T06:53:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d0ef06b0 con 0x5602d0e8e580 2023-10-26T06:53:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.853+0000 7f7820b16b80 10 client.0 _ll_register_callbacks cb 0x5602d0cc1b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32830/0 (socket says 172.21.15.89:32830) 2023-10-26T06:53:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 -- 172.21.15.89:0/1304741 learned_addr learned my addr 172.21.15.89:0/1304741 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f780659c700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e98930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f780759e700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0eefe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 -- 172.21.15.89:0/1304741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 msgr2=0x5602d0eefe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0eefe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 -- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 msgr2=0x5602d0e98930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:32.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e98930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 -- 172.21.15.89:0/1304741 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602d0c05e60 con 0x5602d0e88650 2023-10-26T06:53:32.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f780759e700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0eefe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:32.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7806d9d700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e983f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009990 tx=0x7f77f8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff798764096d68be server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:32.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f77eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:32.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f77eaff5700 1 -- 172.21.15.89:0/1304741 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f801d070 con 0x5602d0e88650 2023-10-26T06:53:32.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f77eaff5700 1 -- 172.21.15.89:0/1304741 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f77f8000e40 con 0x5602d0e88650 2023-10-26T06:53:32.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.854+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f780001b320 con 0x5602d0e88650 2023-10-26T06:53:32.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f77eaff5700 1 -- 172.21.15.89:0/1304741 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77f8005960 con 0x5602d0e88650 2023-10-26T06:53:32.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 10 client.16215 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5602d0c2c2d0 con 0x5602d0e88650 2023-10-26T06:53:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f77eaff5700 1 -- 172.21.15.89:0/1304741 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77f804e020 con 0x5602d0e88650 2023-10-26T06:53:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f77eaff5700 1 -- 172.21.15.89:0/1304741 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77f8004590 con 0x5602d0e88650 2023-10-26T06:53:32.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 20 client.16215 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ef7b90 0x5602d0ef9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:32.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 4 client.16215 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:53:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5602d0dd3ed0 con 0x5602d0ef7b90 2023-10-26T06:53:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.855+0000 7f780659c700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ef7b90 0x5602d0ef9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.856+0000 7f77eaff5700 10 client.16215 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:32.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.856+0000 7f780659c700 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ef7b90 0x5602d0ef9fb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d29c6a64ebaff32 server_cookie=e083839c32609786 in_seq=0 out_seq=0 2023-10-26T06:53:32.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.857+0000 7f77eaff5700 1 -- 172.21.15.89:0/1304741 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5602d0dd3ed0 con 0x5602d0ef7b90 2023-10-26T06:53:32.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.857+0000 7f77eaff5700 10 client.16215 handle_command_reply: tid=0 2023-10-26T06:53:32.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.857+0000 7f7820b16b80 1 client.16215 shutdown 2023-10-26T06:53:32.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.857+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 msgr2=0x5602d0e983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.857+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e983f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009990 tx=0x7f77f8005bc0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 shutdown_connections 2023-10-26T06:53:32.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602d0e88650 0x5602d0e983f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5602d0ef7b90 0x5602d0ef9fb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5602d0e8e580 0x5602d0eefe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 --2- 172.21.15.89:0/1304741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5602d0e8dc60 0x5602d0e98930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f77eaff5700 0 client.16215 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:32.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 >> 172.21.15.89:0/1304741 conn(0x5602d0dfc7a0 msgr2=0x5602d0e7c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:32.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 shutdown_connections 2023-10-26T06:53:32.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 1 -- 172.21.15.89:0/1304741 wait complete. 2023-10-26T06:53:32.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.858+0000 7f7820b16b80 20 client.16215 trim_cache size 0 max 16384 2023-10-26T06:53:32.894 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:53:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:53:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "f7611be0-a613-4896-92a9-3e0f7152b275", 2023-10-26T06:53:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:53:32.895 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:53:32.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77f00a15f0 msgr2=0x7f77f00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77f00a15f0 0x7f77f00a3ab0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f780819a360 tx=0x7f77fc007f80 comp rx=0 tx=0).stop 2023-10-26T06:53:32.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7808106730 msgr2=0x7f780819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:32.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7808106730 0x7f780819d650 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f780000ef30 tx=0x7f780000c3a0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 shutdown_connections 2023-10-26T06:53:32.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77f00a15f0 0x7f77f00a3ab0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7808106730 0x7f780819d650 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7808100720 0x7f7808198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 --2- 172.21.15.89:0/3015423217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78081010e0 0x7f78081993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:32.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 >> 172.21.15.89:0/3015423217 conn(0x7f78080fc1b0 msgr2=0x7f78080fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:32.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 shutdown_connections 2023-10-26T06:53:32.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:32.859+0000 7f780d1ff700 1 -- 172.21.15.89:0/3015423217 wait complete. 2023-10-26T06:53:32.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:53:33.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 -- 172.21.15.89:0/3023660355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c41010e0 msgr2=0x7fa0c410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/3023660355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c41010e0 0x7fa0c410d5f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b0a0 tx=0x7fa0bc00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 -- 172.21.15.89:0/3023660355 shutdown_connections 2023-10-26T06:53:33.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/3023660355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c41010e0 0x7fa0c410d5f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/3023660355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c4106730 0x7fa0c4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/3023660355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 0x7fa0c4100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 -- 172.21.15.89:0/3023660355 >> 172.21.15.89:0/3023660355 conn(0x7fa0c40fc1b0 msgr2=0x7fa0c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:33.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 -- 172.21.15.89:0/3023660355 shutdown_connections 2023-10-26T06:53:33.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.099+0000 7fa0c89ff700 1 -- 172.21.15.89:0/3023660355 wait complete. 2023-10-26T06:53:33.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.100+0000 7fa0c89ff700 1 Processor -- start 2023-10-26T06:53:33.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.100+0000 7fa0c89ff700 1 -- start start 2023-10-26T06:53:33.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c89ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 0x7fa0c4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c89ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c41010e0 0x7fa0c4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c89ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c4106730 0x7fa0c4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c89ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41a4f70 con 0x7fa0c4106730 2023-10-26T06:53:33.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c89ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41a50e0 con 0x7fa0c4100720 2023-10-26T06:53:33.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c89ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41a52a0 con 0x7fa0c41010e0 2023-10-26T06:53:33.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 0x7fa0c4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 0x7fa0c4072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42760/0 (socket says 172.21.15.89:42760) 2023-10-26T06:53:33.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c4106730 0x7fa0c4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 -- 172.21.15.89:0/412467896 learned_addr learned my addr 172.21.15.89:0/412467896 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:33.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c41010e0 0x7fa0c4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 -- 172.21.15.89:0/412467896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c41010e0 msgr2=0x7fa0c4073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c41010e0 0x7fa0c4073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 -- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c4106730 msgr2=0x7fa0c4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c4106730 0x7fa0c4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c259c700 1 -- 172.21.15.89:0/412467896 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009d00 con 0x7fa0c4100720 2023-10-26T06:53:33.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c41010e0 0x7fa0c4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:33.111 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.101+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c4106730 0x7fa0c4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:33.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.102+0000 7fa0c259c700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 0x7fa0c4072af0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400d980 tx=0x7fa0b400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:33.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.102+0000 7fa0bb7fe700 1 -- 172.21.15.89:0/412467896 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4004220 con 0x7fa0c4100720 2023-10-26T06:53:33.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.102+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c41a5580 con 0x7fa0c4100720 2023-10-26T06:53:33.112 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.102+0000 7fa0bb7fe700 1 -- 172.21.15.89:0/412467896 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b4007640 con 0x7fa0c4100720 2023-10-26T06:53:33.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.102+0000 7fa0bb7fe700 1 -- 172.21.15.89:0/412467896 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4005610 con 0x7fa0c4100720 2023-10-26T06:53:33.113 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.102+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41a5ab0 con 0x7fa0c4100720 2023-10-26T06:53:33.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.105+0000 7fa0bb7fe700 1 -- 172.21.15.89:0/412467896 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa0b4004750 con 0x7fa0c4100720 2023-10-26T06:53:33.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.106+0000 7fa0dc24ab80 1 Processor -- start 2023-10-26T06:53:33.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.106+0000 7fa0dc24ab80 1 -- start start 2023-10-26T06:53:33.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.106+0000 7fa0bb7fe700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa0b00a15f0 0x7fa0b00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bb7fe700 1 -- 172.21.15.89:0/412467896 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fa0b40ccf10 con 0x7fa0c4100720 2023-10-26T06:53:33.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d8500 0x564cf04d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d25d0 0x564cf04d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf04456e0 con 0x564cf04d8500 2023-10-26T06:53:33.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf042bc60 con 0x564cf04d7be0 2023-10-26T06:53:33.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf0429580 con 0x564cf04d25d0 2023-10-26T06:53:33.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa0b00a15f0 0x7fa0b00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d8500 0x564cf04d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d8500 0x564cf04d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32848/0 (socket says 172.21.15.89:32848) 2023-10-26T06:53:33.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 -- 172.21.15.89:0/2163048653 learned_addr learned my addr 172.21.15.89:0/2163048653 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:33.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d25d0 0x564cf04d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 -- 172.21.15.89:0/2163048653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d25d0 msgr2=0x564cf04d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d25d0 0x564cf04d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 -- 172.21.15.89:0/2163048653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 msgr2=0x564cf04d7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04d7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0bbfff700 1 -- 172.21.15.89:0/2163048653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf03e23f0 con 0x564cf04d8500 2023-10-26T06:53:33.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.107+0000 7fa0c259c700 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04d7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:33.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.108+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d8500 0x564cf04d2090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009d40 tx=0x7fa0ac0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38dc43452650e94 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:33.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.108+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa0b00a15f0 0x7fa0b00a3ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b780 tx=0x7fa0bc00bab0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:33.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.108+0000 7fa0b8ff9700 1 -- 172.21.15.89:0/2163048653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac011e20 con 0x564cf04d8500 2023-10-26T06:53:33.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.108+0000 7fa0b8ff9700 1 -- 172.21.15.89:0/2163048653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac009e90 con 0x564cf04d8500 2023-10-26T06:53:33.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.108+0000 7fa0b8ff9700 1 -- 172.21.15.89:0/2163048653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac012450 con 0x564cf04d8500 2023-10-26T06:53:33.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.108+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/2163048653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d8500 msgr2=0x564cf04d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.108+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d8500 0x564cf04d2090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009d40 tx=0x7fa0ac0116c0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/2163048653 shutdown_connections 2023-10-26T06:53:33.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d25d0 0x564cf04d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04d7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/2163048653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d8500 0x564cf04d2090 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/2163048653 >> 172.21.15.89:0/2163048653 conn(0x564cf0446720 msgr2=0x564cf04ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:33.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/2163048653 shutdown_connections 2023-10-26T06:53:33.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/2163048653 wait complete. 2023-10-26T06:53:33.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 -- start start 2023-10-26T06:53:33.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.109+0000 7fa0dc24ab80 1 Processor -- start 2023-10-26T06:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04de530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf0424e70 con 0x564cf04d25d0 2023-10-26T06:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf03c1710 con 0x564cf04d7be0 2023-10-26T06:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf03c13b0 con 0x564cf04d8500 2023-10-26T06:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0dc24ab80 10 client.0 _ll_register_callbacks cb 0x564cf030ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:33.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0bbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0bbfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04db820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42774/0 (socket says 172.21.15.89:42774) 2023-10-26T06:53:33.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0bbfff700 1 -- 172.21.15.89:0/1755663248 learned_addr learned my addr 172.21.15.89:0/1755663248 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04de530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32858/0 (socket says 172.21.15.89:32858) 2023-10-26T06:53:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c259c700 1 -- 172.21.15.89:0/1755663248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 msgr2=0x564cf04dbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c259c700 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c259c700 1 -- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 msgr2=0x564cf04db820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c259c700 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04db820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.110+0000 7fa0c259c700 1 -- 172.21.15.89:0/1755663248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41a5ab0 con 0x564cf04d25d0 2023-10-26T06:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04db820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0c259c700 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04de530 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ca610 tx=0x7fa0b4099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9145d03eb7c08e1f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1755663248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40b1070 con 0x564cf04d25d0 2023-10-26T06:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1755663248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fa0b4004380 con 0x564cf04d25d0 2023-10-26T06:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cf0429ef0 con 0x564cf04d25d0 2023-10-26T06:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1755663248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0b40b8170 con 0x564cf04d25d0 2023-10-26T06:53:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0dc24ab80 10 client.16227 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fa0ac009970 con 0x564cf04d25d0 2023-10-26T06:53:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.111+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1755663248 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b4063080 con 0x564cf04d25d0 2023-10-26T06:53:33.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1755663248 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa0b40a1da0 con 0x564cf04d25d0 2023-10-26T06:53:33.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0dc24ab80 10 client.16227 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:33.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0dc24ab80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:33.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0dc24ab80 20 client.16227 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:33.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf04ebf00 0x564cf04ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0dc24ab80 4 client.16227 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:53:33.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cf04a18b0 con 0x564cf04ebf00 2023-10-26T06:53:33.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.112+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf04ebf00 0x564cf04ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.113+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf04ebf00 0x564cf04ee2f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b578a3c3b0af69fe server_cookie=be0263cda5c613a4 in_seq=0 out_seq=0 2023-10-26T06:53:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.113+0000 7fa0a9ffb700 10 client.16227 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.113+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1755663248 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564cf04a18b0 con 0x564cf04ebf00 2023-10-26T06:53:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.113+0000 7fa0a9ffb700 10 client.16227 handle_command_reply: tid=0 2023-10-26T06:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.113+0000 7fa0dc24ab80 1 client.16227 shutdown 2023-10-26T06:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 msgr2=0x564cf04de530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04de530 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ca610 tx=0x7fa0b4099f70 comp rx=0 tx=0).stop 2023-10-26T06:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 shutdown_connections 2023-10-26T06:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04de530 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf04ebf00 0x564cf04ee2f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1755663248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04db820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 >> 172.21.15.89:0/1755663248 conn(0x564cf0446720 msgr2=0x564cf04ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0a9ffb700 0 client.16227 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 shutdown_connections 2023-10-26T06:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1755663248 wait complete. 2023-10-26T06:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.114+0000 7fa0dc24ab80 20 client.16227 trim_cache size 0 max 16384 2023-10-26T06:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.131+0000 7fa0dc24ab80 1 Processor -- start 2023-10-26T06:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.131+0000 7fa0dc24ab80 1 -- start start 2023-10-26T06:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.131+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.131+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04da130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf02a5920 con 0x564cf04d25d0 2023-10-26T06:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf04de9d0 con 0x564cf04d7be0 2023-10-26T06:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf04e3420 con 0x564cf04d8500 2023-10-26T06:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32874/0 (socket says 172.21.15.89:32874) 2023-10-26T06:53:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 -- 172.21.15.89:0/3992945155 learned_addr learned my addr 172.21.15.89:0/3992945155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04da130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 -- 172.21.15.89:0/3992945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 msgr2=0x564cf04da130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04da130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 -- 172.21.15.89:0/3992945155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 msgr2=0x564cf04e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 -- 172.21.15.89:0/3992945155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf024fde0 con 0x564cf04d25d0 2023-10-26T06:53:33.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:33.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.132+0000 7fa0c259c700 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04d2090 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40963a0 tx=0x7fa0b4098450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3df4ad341ffdd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:33.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0b97fa700 1 -- 172.21.15.89:0/3992945155 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40b1070 con 0x564cf04d25d0 2023-10-26T06:53:33.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0b97fa700 1 -- 172.21.15.89:0/3992945155 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564cf0429ef0 con 0x564cf04d25d0 2023-10-26T06:53:33.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0b97fa700 1 -- 172.21.15.89:0/3992945155 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b409a360 con 0x564cf04d25d0 2023-10-26T06:53:33.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/3992945155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 msgr2=0x564cf04d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04d2090 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40963a0 tx=0x7fa0b4098450 comp rx=0 tx=0).stop 2023-10-26T06:53:33.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/3992945155 shutdown_connections 2023-10-26T06:53:33.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04d2090 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf04da130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/3992945155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/3992945155 >> 172.21.15.89:0/3992945155 conn(0x564cf0446720 msgr2=0x564cf04d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:33.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/3992945155 shutdown_connections 2023-10-26T06:53:33.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.133+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/3992945155 wait complete. 2023-10-26T06:53:33.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 -- start start 2023-10-26T06:53:33.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 Processor -- start 2023-10-26T06:53:33.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04e1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf053fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf04e28a0 con 0x564cf04d25d0 2023-10-26T06:53:33.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf0540250 con 0x564cf04d7be0 2023-10-26T06:53:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf0540410 con 0x564cf04d8500 2023-10-26T06:53:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.134+0000 7fa0dc24ab80 10 client.0 _ll_register_callbacks cb 0x564cf030ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:53:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04e1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04e1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32876/0 (socket says 172.21.15.89:32876) 2023-10-26T06:53:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 -- 172.21.15.89:0/1803158798 learned_addr learned my addr 172.21.15.89:0/1803158798 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:53:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf053fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 -- 172.21.15.89:0/1803158798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 msgr2=0x564cf053fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf053fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 -- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 msgr2=0x564cf04e2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:53:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 -- 172.21.15.89:0/1803158798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac009970 con 0x564cf04d25d0 2023-10-26T06:53:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:53:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c2d9d700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf053fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T06:53:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0c259c700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04e1bf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4099da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a80b38b852d43f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:53:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:53:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1803158798 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40b1070 con 0x564cf04d25d0 2023-10-26T06:53:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1803158798 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fa0b409a100 con 0x564cf04d25d0 2023-10-26T06:53:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.135+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0ac009e90 con 0x564cf04d25d0 2023-10-26T06:53:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1803158798 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0b409a300 con 0x564cf04d25d0 2023-10-26T06:53:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 10 client.16233 fetch_fsmap learned FSMap version 20 2023-10-26T06:53:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564cf0238c60 con 0x564cf04d25d0 2023-10-26T06:53:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1803158798 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac009e90 con 0x564cf04d25d0 2023-10-26T06:53:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1803158798 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa0b409ed30 con 0x564cf04d25d0 2023-10-26T06:53:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 10 client.16233 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:53:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:53:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 20 client.16233 populate_metadata read hostname 'smithi089' 2023-10-26T06:53:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf0541760 0x564cf0543b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:53:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 4 client.16233 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:53:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564cf041de50 con 0x564cf0541760 2023-10-26T06:53:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.136+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf0541760 0x564cf0543b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:53:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0bbfff700 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf0541760 0x564cf0543b50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6416df40a6d22f server_cookie=55c827ddb8437ec3 in_seq=0 out_seq=0 2023-10-26T06:53:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0a9ffb700 10 client.16233 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0a9ffb700 1 -- 172.21.15.89:0/1803158798 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564cf041de50 con 0x564cf0541760 2023-10-26T06:53:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0a9ffb700 10 client.16233 handle_command_reply: tid=0 2023-10-26T06:53:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 client.16233 shutdown 2023-10-26T06:53:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 msgr2=0x564cf04e1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04e1bf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4099da0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 shutdown_connections 2023-10-26T06:53:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564cf04d25d0 0x564cf04e1bf0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564cf0541760 0x564cf0543b50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf04d8500 0x564cf053fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 --2- 172.21.15.89:0/1803158798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564cf04d7be0 0x564cf04e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0a9ffb700 0 client.16233 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:53:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 >> 172.21.15.89:0/1803158798 conn(0x564cf0446720 msgr2=0x564cf04c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 shutdown_connections 2023-10-26T06:53:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.137+0000 7fa0dc24ab80 1 -- 172.21.15.89:0/1803158798 wait complete. 2023-10-26T06:53:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0dc24ab80 20 client.16233 trim_cache size 0 max 16384 2023-10-26T06:53:33.172 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:53:33.172 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:53:33.173 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:53:33.173 INFO:teuthology.orchestra.run.smithi089.stdout: "f7611be0-a613-4896-92a9-3e0f7152b275": { 2023-10-26T06:53:33.173 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:53:33.173 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "f7611be0-a613-4896-92a9-3e0f7152b275", 2023-10-26T06:53:33.174 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:53:33.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:53:33.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:53:33.175 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:53:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa0b00a15f0 msgr2=0x7fa0b00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa0b00a15f0 0x7fa0b00a3ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b780 tx=0x7fa0bc00bab0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 msgr2=0x7fa0c4072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:53:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 0x7fa0c4072af0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400d980 tx=0x7fa0b400dc90 comp rx=0 tx=0).stop 2023-10-26T06:53:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 shutdown_connections 2023-10-26T06:53:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa0b00a15f0 0x7fa0b00a3ab0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa0c4106730 0x7fa0c4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0c4100720 0x7fa0c4072af0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 --2- 172.21.15.89:0/412467896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c41010e0 0x7fa0c4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:53:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 >> 172.21.15.89:0/412467896 conn(0x7fa0c40fc1b0 msgr2=0x7fa0c40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:53:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 shutdown_connections 2023-10-26T06:53:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:53:33.138+0000 7fa0c89ff700 1 -- 172.21.15.89:0/412467896 wait complete. 2023-10-26T06:53:33.179 INFO:tasks.cephfs.filesystem:scrub status for tag:f7611be0-a613-4896-92a9-3e0f7152b275 - {'path': '/', 'tag': 'f7611be0-a613-4896-92a9-3e0f7152b275', 'options': 'recursive,force'} 2023-10-26T06:53:33.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:33 smithi135 ceph-mon[131403]: pgmap v511: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:33.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:33 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:53:33.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:33 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:33.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:33 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:53:33.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:33 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:33.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:33 smithi089 ceph-mon[95031]: pgmap v511: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:33.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:33 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:53:33.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:33 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:33.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:33 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:53:33.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:33 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:33.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:33 smithi192 ceph-mon[131661]: pgmap v511: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:33.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:33 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:53:33.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:33 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:33.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:33 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:53:33.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:33 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:53:34.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:34 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:53:34.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:34 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:53:34.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:34 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:53:35.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:35 smithi089 ceph-mon[95031]: pgmap v512: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 2 op/s 2023-10-26T06:53:35.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:35 smithi192 ceph-mon[131661]: pgmap v512: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 2 op/s 2023-10-26T06:53:35.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:35 smithi135 ceph-mon[131403]: pgmap v512: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 2 op/s 2023-10-26T06:53:37.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:37 smithi089 ceph-mon[95031]: pgmap v513: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:53:37.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:37 smithi192 ceph-mon[131661]: pgmap v513: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:53:37.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:37 smithi135 ceph-mon[131403]: pgmap v513: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:53:39.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:39 smithi089 ceph-mon[95031]: pgmap v514: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:53:39.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:39 smithi192 ceph-mon[131661]: pgmap v514: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:53:39.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:39 smithi135 ceph-mon[131403]: pgmap v514: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:53:41.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:41 smithi089 ceph-mon[95031]: pgmap v515: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:53:41.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:41 smithi192 ceph-mon[131661]: pgmap v515: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:53:41.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:41 smithi135 ceph-mon[131403]: pgmap v515: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:53:43.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:43 smithi089 ceph-mon[95031]: pgmap v516: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:53:43.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:43 smithi192 ceph-mon[131661]: pgmap v516: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:53:43.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:43 smithi135 ceph-mon[131403]: pgmap v516: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:53:45.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:45 smithi089 ceph-mon[95031]: pgmap v517: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:53:45.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:45 smithi192 ceph-mon[131661]: pgmap v517: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:53:45.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:45 smithi135 ceph-mon[131403]: pgmap v517: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:53:47.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:47 smithi089 ceph-mon[95031]: pgmap v518: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:53:47.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:47 smithi192 ceph-mon[131661]: pgmap v518: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:53:47.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:47 smithi135 ceph-mon[131403]: pgmap v518: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:53:49.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:49 smithi089 ceph-mon[95031]: pgmap v519: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:53:49.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:49 smithi192 ceph-mon[131661]: pgmap v519: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:53:49.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:49 smithi135 ceph-mon[131403]: pgmap v519: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:53:51.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:51 smithi089 ceph-mon[95031]: pgmap v520: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:51.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:51 smithi192 ceph-mon[131661]: pgmap v520: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:51.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:51 smithi135 ceph-mon[131403]: pgmap v520: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:53 smithi089 ceph-mon[95031]: pgmap v521: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:53.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:53 smithi192 ceph-mon[131661]: pgmap v521: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:53.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:53 smithi135 ceph-mon[131403]: pgmap v521: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:55.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:55 smithi089 ceph-mon[95031]: pgmap v522: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:53:55.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:55 smithi192 ceph-mon[131661]: pgmap v522: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:53:55.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:55 smithi135 ceph-mon[131403]: pgmap v522: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:53:57.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:57 smithi089 ceph-mon[95031]: pgmap v523: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:57.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:57 smithi192 ceph-mon[131661]: pgmap v523: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:57.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:57 smithi135 ceph-mon[131403]: pgmap v523: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:53:59.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:53:59 smithi089 ceph-mon[95031]: pgmap v524: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:59.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:53:59 smithi192 ceph-mon[131661]: pgmap v524: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:53:59.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:53:59 smithi135 ceph-mon[131403]: pgmap v524: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:01 smithi089 ceph-mon[95031]: pgmap v525: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:01.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:01 smithi192 ceph-mon[131661]: pgmap v525: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:01.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:01 smithi135 ceph-mon[131403]: pgmap v525: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:03.180 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:54:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 -- 172.21.15.89:0/3247182136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105170 msgr2=0x7f12901055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 --2- 172.21.15.89:0/3247182136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105170 0x7f12901055f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f12780099c0 tx=0x7f1278009cd0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 -- 172.21.15.89:0/3247182136 shutdown_connections 2023-10-26T06:54:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 --2- 172.21.15.89:0/3247182136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1290105b30 0x7f129010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 --2- 172.21.15.89:0/3247182136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1290067d40 0x7f1290068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 --2- 172.21.15.89:0/3247182136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105170 0x7f12901055f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 -- 172.21.15.89:0/3247182136 >> 172.21.15.89:0/3247182136 conn(0x7f12900767f0 msgr2=0x7f1290076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 -- 172.21.15.89:0/3247182136 shutdown_connections 2023-10-26T06:54:03.395 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.391+0000 7f1295fff700 1 -- 172.21.15.89:0/3247182136 wait complete. 2023-10-26T06:54:03.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.392+0000 7f1295fff700 1 Processor -- start 2023-10-26T06:54:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.392+0000 7f1295fff700 1 -- start start 2023-10-26T06:54:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f1295fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1290067d40 0x7f129019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f1295fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1290105170 0x7f129019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f1295fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105b30 0x7f1290196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f1295fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290197070 con 0x7f1290105b30 2023-10-26T06:54:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f1295fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12901971e0 con 0x7f1290105170 2023-10-26T06:54:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f1295fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12901973a0 con 0x7f1290067d40 2023-10-26T06:54:03.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105b30 0x7f1290196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105b30 0x7f1290196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44818/0 (socket says 172.21.15.89:44818) 2023-10-26T06:54:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 -- 172.21.15.89:0/3263663397 learned_addr learned my addr 172.21.15.89:0/3263663397 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:03.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128f7fe700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1290067d40 0x7f129019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f1287fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1290105170 0x7f129019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 -- 172.21.15.89:0/3263663397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1290067d40 msgr2=0x7f129019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1290067d40 0x7f129019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 -- 172.21.15.89:0/3263663397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1290105170 msgr2=0x7f129019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1290105170 0x7f129019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.393+0000 7f128ffff700 1 -- 172.21.15.89:0/3263663397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12780095f0 con 0x7f1290105b30 2023-10-26T06:54:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.394+0000 7f128ffff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105b30 0x7f1290196a10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f128800efc0 tx=0x7f128800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.394+0000 7f128d7fa700 1 -- 172.21.15.89:0/3263663397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288018070 con 0x7f1290105b30 2023-10-26T06:54:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.394+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1290197620 con 0x7f1290105b30 2023-10-26T06:54:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.394+0000 7f128d7fa700 1 -- 172.21.15.89:0/3263663397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128800f040 con 0x7f1290105b30 2023-10-26T06:54:03.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.394+0000 7f128d7fa700 1 -- 172.21.15.89:0/3263663397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800cc30 con 0x7f1290105b30 2023-10-26T06:54:03.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.394+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12901a38d0 con 0x7f1290105b30 2023-10-26T06:54:03.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.397+0000 7f128d7fa700 1 -- 172.21.15.89:0/3263663397 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f128800cd90 con 0x7f1290105b30 2023-10-26T06:54:03.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.398+0000 7f12a9893b80 1 Processor -- start 2023-10-26T06:54:03.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.398+0000 7f128d7fa700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f127c0a15f0 0x7f127c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f12a9893b80 1 -- start start 2023-10-26T06:54:03.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f128d7fa700 1 -- 172.21.15.89:0/3263663397 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f12880ccb00 con 0x7f1290105b30 2023-10-26T06:54:03.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f1287fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f127c0a15f0 0x7f127c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b39690 0x564f25b39a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b3e100 0x564f25b3e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f12a9893b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25aa86e0 con 0x564f25b39fb0 2023-10-26T06:54:03.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f12a9893b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25a8ec60 con 0x564f25b39690 2023-10-26T06:54:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f12a9893b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25a8c580 con 0x564f25b3e100 2023-10-26T06:54:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f1287fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.399+0000 7f128ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b3e100 0x564f25b3e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f1287fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44824/0 (socket says 172.21.15.89:44824) 2023-10-26T06:54:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f1287fff700 1 -- 172.21.15.89:0/2902859145 learned_addr learned my addr 172.21.15.89:0/2902859145 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f128f7fe700 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b39690 0x564f25b39a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f128f7fe700 1 -- 172.21.15.89:0/2902859145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b3e100 msgr2=0x564f25b3e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f128f7fe700 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b3e100 0x564f25b3e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f128f7fe700 1 -- 172.21.15.89:0/2902859145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 msgr2=0x564f25b3dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f128f7fe700 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f128f7fe700 1 -- 172.21.15.89:0/2902859145 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f25a453f0 con 0x564f25b39690 2023-10-26T06:54:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f1287fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f127c0a15f0 0x7f127c0a3ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1278000c00 tx=0x7f127800b9b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f128f7fe700 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b39690 0x564f25b39a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1280009cb0 tx=0x7f1280011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3005384d0d5261d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.400+0000 7f1285ffb700 1 -- 172.21.15.89:0/2902859145 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280011c50 con 0x564f25b39690 2023-10-26T06:54:03.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f1285ffb700 1 -- 172.21.15.89:0/2902859145 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1280004510 con 0x564f25b39690 2023-10-26T06:54:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f1285ffb700 1 -- 172.21.15.89:0/2902859145 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280005240 con 0x564f25b39690 2023-10-26T06:54:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 -- 172.21.15.89:0/2902859145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b39690 msgr2=0x564f25b39a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b39690 0x564f25b39a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1280009cb0 tx=0x7f1280011440 comp rx=0 tx=0).stop 2023-10-26T06:54:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 -- 172.21.15.89:0/2902859145 shutdown_connections 2023-10-26T06:54:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b3e100 0x564f25b3e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b39690 0x564f25b39a70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 --2- 172.21.15.89:0/2902859145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 -- 172.21.15.89:0/2902859145 >> 172.21.15.89:0/2902859145 conn(0x564f25aa9720 msgr2=0x564f25b2fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 -- 172.21.15.89:0/2902859145 shutdown_connections 2023-10-26T06:54:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.401+0000 7f12a9893b80 1 -- 172.21.15.89:0/2902859145 wait complete. 2023-10-26T06:54:03.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.402+0000 7f12a9893b80 1 -- start start 2023-10-26T06:54:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.402+0000 7f12a9893b80 1 Processor -- start 2023-10-26T06:54:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39690 0x564f25b45840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39fb0 0x564f25b42b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f12a9893b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25a87e70 con 0x564f25b39690 2023-10-26T06:54:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f12a9893b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25a24710 con 0x564f25b3e100 2023-10-26T06:54:03.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f12a9893b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25a243b0 con 0x564f25b39fb0 2023-10-26T06:54:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f12a9893b80 10 client.0 _ll_register_callbacks cb 0x564f2596ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39690 0x564f25b45840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39690 0x564f25b45840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44830/0 (socket says 172.21.15.89:44830) 2023-10-26T06:54:03.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 -- 172.21.15.89:0/3907475860 learned_addr learned my addr 172.21.15.89:0/3907475860 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f1287fff700 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39fb0 0x564f25b42b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128ffff700 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.423 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 -- 172.21.15.89:0/3907475860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39fb0 msgr2=0x564f25b42b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39fb0 0x564f25b42b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 -- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 msgr2=0x564f25b430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.424 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 -- 172.21.15.89:0/3907475860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12800098e0 con 0x564f25b39690 2023-10-26T06:54:03.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.403+0000 7f128f7fe700 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39690 0x564f25b45840 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1280005c50 tx=0x7f12800047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46880a4c80996e8f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f126effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:03.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f126effd700 1 -- 172.21.15.89:0/3907475860 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280011c60 con 0x564f25b39690 2023-10-26T06:54:03.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f25a8cef0 con 0x564f25b39690 2023-10-26T06:54:03.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f126effd700 1 -- 172.21.15.89:0/3907475860 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1280011dc0 con 0x564f25b39690 2023-10-26T06:54:03.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f126effd700 1 -- 172.21.15.89:0/3907475860 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1280003680 con 0x564f25b39690 2023-10-26T06:54:03.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 10 client.16245 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:03.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564f258b2de0 con 0x564f25b39690 2023-10-26T06:54:03.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f126effd700 1 -- 172.21.15.89:0/3907475860 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f25a8cef0 con 0x564f25b39690 2023-10-26T06:54:03.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f126effd700 1 -- 172.21.15.89:0/3907475860 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1280005260 con 0x564f25b39690 2023-10-26T06:54:03.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:03.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:03.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 20 client.16245 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:03.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25b53070 0x564f25b55490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 4 client.16245 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:03.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.404+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f25b048b0 con 0x564f25b53070 2023-10-26T06:54:03.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.405+0000 7f1287fff700 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25b53070 0x564f25b55490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.405+0000 7f126effd700 10 client.16245 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:03.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.405+0000 7f1287fff700 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25b53070 0x564f25b55490 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7e9fca285f3f54 server_cookie=73b90686a1d47553 in_seq=0 out_seq=0 2023-10-26T06:54:03.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.406+0000 7f126effd700 1 -- 172.21.15.89:0/3907475860 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f25b048b0 con 0x564f25b53070 2023-10-26T06:54:03.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.406+0000 7f126effd700 10 client.16245 handle_command_reply: tid=0 2023-10-26T06:54:03.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.406+0000 7f12a9893b80 1 client.16245 shutdown 2023-10-26T06:54:03.430 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.406+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39690 msgr2=0x564f25b45840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.406+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39690 0x564f25b45840 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1280005c50 tx=0x7f12800047d0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 shutdown_connections 2023-10-26T06:54:03.431 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39690 0x564f25b45840 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25b53070 0x564f25b55490 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3907475860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39fb0 0x564f25b42b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f126effd700 0 client.16245 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:03.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 >> 172.21.15.89:0/3907475860 conn(0x564f25aa9720 msgr2=0x564f25b34080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 shutdown_connections 2023-10-26T06:54:03.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 1 -- 172.21.15.89:0/3907475860 wait complete. 2023-10-26T06:54:03.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.407+0000 7f12a9893b80 20 client.16245 trim_cache size 0 max 16384 2023-10-26T06:54:03.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.424+0000 7f12a9893b80 1 Processor -- start 2023-10-26T06:54:03.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.424+0000 7f12a9893b80 1 -- start start 2023-10-26T06:54:03.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b45370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f12a9893b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25908920 con 0x564f25b39fb0 2023-10-26T06:54:03.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f12a9893b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25b44130 con 0x564f25b3e100 2023-10-26T06:54:03.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f12a9893b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25b2c2f0 con 0x564f25b39690 2023-10-26T06:54:03.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f1287fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f1287fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44842/0 (socket says 172.21.15.89:44842) 2023-10-26T06:54:03.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57252/0 (socket says 172.21.15.89:57252) 2023-10-26T06:54:03.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f1287fff700 1 -- 172.21.15.89:0/3570544413 learned_addr learned my addr 172.21.15.89:0/3570544413 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:03.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128ffff700 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b45370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128f7fe700 1 -- 172.21.15.89:0/3570544413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 msgr2=0x564f25b45370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128f7fe700 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b45370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128f7fe700 1 -- 172.21.15.89:0/3570544413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 msgr2=0x564f25b3e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128f7fe700 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f128f7fe700 1 -- 172.21.15.89:0/3570544413 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2589bc60 con 0x564f25b39690 2023-10-26T06:54:03.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.425+0000 7f1287fff700 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:03.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f128f7fe700 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4a2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12800044e0 tx=0x7f1280011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16576f82b3569d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12867fc700 1 -- 172.21.15.89:0/3570544413 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280022400 con 0x564f25b39690 2023-10-26T06:54:03.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12867fc700 1 -- 172.21.15.89:0/3570544413 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12800048f0 con 0x564f25b39690 2023-10-26T06:54:03.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12867fc700 1 -- 172.21.15.89:0/3570544413 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280003970 con 0x564f25b39690 2023-10-26T06:54:03.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 -- 172.21.15.89:0/3570544413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 msgr2=0x564f25b4a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4a2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f12800044e0 tx=0x7f1280011440 comp rx=0 tx=0).stop 2023-10-26T06:54:03.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 -- 172.21.15.89:0/3570544413 shutdown_connections 2023-10-26T06:54:03.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b45370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4a2c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 --2- 172.21.15.89:0/3570544413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b3e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 -- 172.21.15.89:0/3570544413 >> 172.21.15.89:0/3570544413 conn(0x564f25aa9720 msgr2=0x564f25b34080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.426+0000 7f12a9893b80 1 -- 172.21.15.89:0/3570544413 shutdown_connections 2023-10-26T06:54:03.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.427+0000 7f12a9893b80 1 -- 172.21.15.89:0/3570544413 wait complete. 2023-10-26T06:54:03.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.427+0000 7f12a9893b80 1 -- start start 2023-10-26T06:54:03.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.427+0000 7f12a9893b80 1 Processor -- start 2023-10-26T06:54:03.448 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f12a9893b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b49a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f12a9893b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25b4a140 con 0x564f25b39fb0 2023-10-26T06:54:03.449 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f12a9893b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25ba6ed0 con 0x564f25b3e100 2023-10-26T06:54:03.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f12a9893b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f25ba7010 con 0x564f25b39690 2023-10-26T06:54:03.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f12a9893b80 10 client.0 _ll_register_callbacks cb 0x564f2596ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:03.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.450 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44852/0 (socket says 172.21.15.89:44852) 2023-10-26T06:54:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f128f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57264/0 (socket says 172.21.15.89:57264) 2023-10-26T06:54:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 -- 172.21.15.89:0/1088464185 learned_addr learned my addr 172.21.15.89:0/1088464185 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:03.451 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f128ffff700 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b49a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 -- 172.21.15.89:0/1088464185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 msgr2=0x564f25b4f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 -- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 msgr2=0x564f25b49a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:03.452 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b49a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 -- 172.21.15.89:0/1088464185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127801f040 con 0x564f25b39fb0 2023-10-26T06:54:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f128f7fe700 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:03.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f1287fff700 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b492f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f12780058e0 tx=0x7f1278009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191bc82079746d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.428+0000 7f126effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f126effd700 1 -- 172.21.15.89:0/1088464185 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127801c040 con 0x564f25b39fb0 2023-10-26T06:54:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f126effd700 1 -- 172.21.15.89:0/1088464185 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f12780344c0 con 0x564f25b39fb0 2023-10-26T06:54:03.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1280004510 con 0x564f25b39fb0 2023-10-26T06:54:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f126effd700 1 -- 172.21.15.89:0/1088464185 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1278034a00 con 0x564f25b39fb0 2023-10-26T06:54:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 10 client.16251 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x564f258b2de0 con 0x564f25b39fb0 2023-10-26T06:54:03.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f126effd700 1 -- 172.21.15.89:0/1088464185 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1278049080 con 0x564f25b39fb0 2023-10-26T06:54:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f126effd700 1 -- 172.21.15.89:0/1088464185 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f127803d4c0 con 0x564f25b39fb0 2023-10-26T06:54:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:03.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 20 client.16251 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25ba8740 0x564f25baab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 4 client.16251 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:54:03.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.429+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f25a80e50 con 0x564f25ba8740 2023-10-26T06:54:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.430+0000 7f128f7fe700 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25ba8740 0x564f25baab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.430+0000 7f126effd700 10 client.16251 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.430+0000 7f128f7fe700 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25ba8740 0x564f25baab60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3e6c6285858176 server_cookie=cb9a69cddeca60e2 in_seq=0 out_seq=0 2023-10-26T06:54:03.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f126effd700 1 -- 172.21.15.89:0/1088464185 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564f25a80e50 con 0x564f25ba8740 2023-10-26T06:54:03.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f126effd700 10 client.16251 handle_command_reply: tid=0 2023-10-26T06:54:03.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 client.16251 shutdown 2023-10-26T06:54:03.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 msgr2=0x564f25b492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b492f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f12780058e0 tx=0x7f1278009cd0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 shutdown_connections 2023-10-26T06:54:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x564f25ba8740 0x564f25baab60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564f25b3e100 0x564f25b49a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f25b39690 0x564f25b4f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 --2- 172.21.15.89:0/1088464185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564f25b39fb0 0x564f25b492f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f126effd700 0 client.16251 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:03.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 >> 172.21.15.89:0/1088464185 conn(0x564f25aa9720 msgr2=0x564f25b34080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 shutdown_connections 2023-10-26T06:54:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 1 -- 172.21.15.89:0/1088464185 wait complete. 2023-10-26T06:54:03.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.431+0000 7f12a9893b80 20 client.16251 trim_cache size 0 max 16384 2023-10-26T06:54:03.463 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:54:03.463 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:54:03.463 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:54:03.464 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:54:03.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f127c0a15f0 msgr2=0x7f127c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f127c0a15f0 0x7f127c0a3ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1278000c00 tx=0x7f127800b9b0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105b30 msgr2=0x7f1290196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105b30 0x7f1290196a10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f128800efc0 tx=0x7f128800c3a0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 shutdown_connections 2023-10-26T06:54:03.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f127c0a15f0 0x7f127c0a3ab0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1290105b30 0x7f1290196a10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1290067d40 0x7f129019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 --2- 172.21.15.89:0/3263663397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1290105170 0x7f129019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 >> 172.21.15.89:0/3263663397 conn(0x7f12900767f0 msgr2=0x7f12900fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 shutdown_connections 2023-10-26T06:54:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.432+0000 7f1295fff700 1 -- 172.21.15.89:0/3263663397 wait complete. 2023-10-26T06:54:03.468 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:54:03.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:54:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 -- 172.21.15.89:0/3482781234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 msgr2=0x7f3e7c10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 --2- 172.21.15.89:0/3482781234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 0x7f3e7c10d570 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b0a0 tx=0x7f3e7400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 -- 172.21.15.89:0/3482781234 shutdown_connections 2023-10-26T06:54:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 --2- 172.21.15.89:0/3482781234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 0x7f3e7c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 --2- 172.21.15.89:0/3482781234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 0x7f3e7c10d570 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 --2- 172.21.15.89:0/3482781234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e7c1006a0 0x7f3e7c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 -- 172.21.15.89:0/3482781234 >> 172.21.15.89:0/3482781234 conn(0x7f3e7c0fc170 msgr2=0x7f3e7c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 -- 172.21.15.89:0/3482781234 shutdown_connections 2023-10-26T06:54:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.667+0000 7f3e83fff700 1 -- 172.21.15.89:0/3482781234 wait complete. 2023-10-26T06:54:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 Processor -- start 2023-10-26T06:54:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 -- start start 2023-10-26T06:54:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e7c1006a0 0x7f3e7c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 0x7f3e7c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 0x7f3e7c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19dc50 con 0x7f3e7c1066b0 2023-10-26T06:54:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19ddc0 con 0x7f3e7c1006a0 2023-10-26T06:54:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.668+0000 7f3e83fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19df80 con 0x7f3e7c101060 2023-10-26T06:54:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 0x7f3e7c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e81d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e7c1006a0 0x7f3e7c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 0x7f3e7c19d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44862/0 (socket says 172.21.15.89:44862) 2023-10-26T06:54:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 -- 172.21.15.89:0/1874561890 learned_addr learned my addr 172.21.15.89:0/1874561890 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8159a700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 0x7f3e7c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 -- 172.21.15.89:0/1874561890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 msgr2=0x7f3e7c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 0x7f3e7c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 -- 172.21.15.89:0/1874561890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e7c1006a0 msgr2=0x7f3e7c198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e7c1006a0 0x7f3e7c198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 -- 172.21.15.89:0/1874561890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74009d00 con 0x7f3e7c1066b0 2023-10-26T06:54:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8159a700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 0x7f3e7c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e81d9b700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e7c1006a0 0x7f3e7c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.669+0000 7f3e8259c700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 0x7f3e7c19d610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b070 tx=0x7f3e740077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.670+0000 7f3e72ffd700 1 -- 172.21.15.89:0/1874561890 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400e040 con 0x7f3e7c1066b0 2023-10-26T06:54:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.670+0000 7f3e72ffd700 1 -- 172.21.15.89:0/1874561890 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e740094c0 con 0x7f3e7c1066b0 2023-10-26T06:54:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.670+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c19e200 con 0x7f3e7c1066b0 2023-10-26T06:54:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.670+0000 7f3e72ffd700 1 -- 172.21.15.89:0/1874561890 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740044e0 con 0x7f3e7c1066b0 2023-10-26T06:54:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.670+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c1a5aa0 con 0x7f3e7c1066b0 2023-10-26T06:54:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.671+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c101de0 con 0x7f3e7c1066b0 2023-10-26T06:54:03.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.673+0000 7f3e72ffd700 1 -- 172.21.15.89:0/1874561890 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3e74004640 con 0x7f3e7c1066b0 2023-10-26T06:54:03.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.675+0000 7f3e72ffd700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3e680a15f0 0x7f3e680a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.675+0000 7f3e72ffd700 1 -- 172.21.15.89:0/1874561890 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f3e740cdff0 con 0x7f3e7c1066b0 2023-10-26T06:54:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.675+0000 7f3e81d9b700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3e680a15f0 0x7f3e680a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.676+0000 7f3e81d9b700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3e680a15f0 0x7f3e680a3ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009fd0 tx=0x7f3e78009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.679+0000 7f3e72ffd700 1 -- 172.21.15.89:0/1874561890 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e74097390 con 0x7f3e7c1066b0 2023-10-26T06:54:03.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:03 smithi089 ceph-mon[95031]: pgmap v526: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 22 KiB/s wr, 4 op/s 2023-10-26T06:54:03.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:03 smithi192 ceph-mon[131661]: pgmap v526: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 22 KiB/s wr, 4 op/s 2023-10-26T06:54:03.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.924+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e7c067530 con 0x7f3e7c1066b0 2023-10-26T06:54:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.925+0000 7f3e72ffd700 1 -- 172.21.15.89:0/1874561890 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f3e7409a9b0 con 0x7f3e7c1066b0 2023-10-26T06:54:03.927 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:54:03.928 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:54:03.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3e680a15f0 msgr2=0x7f3e680a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3e680a15f0 0x7f3e680a3ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009fd0 tx=0x7f3e78009260 comp rx=0 tx=0).stop 2023-10-26T06:54:03.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 msgr2=0x7f3e7c19d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:03.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 0x7f3e7c19d610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b070 tx=0x7f3e740077f0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 shutdown_connections 2023-10-26T06:54:03.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f3e680a15f0 0x7f3e680a3ab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e7c1066b0 0x7f3e7c19d610 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e7c1006a0 0x7f3e7c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 --2- 172.21.15.89:0/1874561890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c101060 0x7f3e7c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:03.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.927+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 >> 172.21.15.89:0/1874561890 conn(0x7f3e7c0fc170 msgr2=0x7f3e7c10b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:03.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.928+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 shutdown_connections 2023-10-26T06:54:03.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:03.928+0000 7f3e83fff700 1 -- 172.21.15.89:0/1874561890 wait complete. 2023-10-26T06:54:03.935 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:54:03.949 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:54:03.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:03 smithi135 ceph-mon[131403]: pgmap v526: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 22 KiB/s wr, 4 op/s 2023-10-26T06:54:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 -- 172.21.15.89:0/2715433323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 msgr2=0x7f1ef8100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 --2- 172.21.15.89:0/2715433323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 0x7f1ef8100bc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009990 tx=0x7f1eec009ca0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 -- 172.21.15.89:0/2715433323 shutdown_connections 2023-10-26T06:54:04.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 --2- 172.21.15.89:0/2715433323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 0x7f1ef810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 --2- 172.21.15.89:0/2715433323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ef8106750 0x7f1ef8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 --2- 172.21.15.89:0/2715433323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 0x7f1ef8100bc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 -- 172.21.15.89:0/2715433323 >> 172.21.15.89:0/2715433323 conn(0x7f1ef80fc230 msgr2=0x7f1ef80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:04.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 -- 172.21.15.89:0/2715433323 shutdown_connections 2023-10-26T06:54:04.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.147+0000 7f1efddff700 1 -- 172.21.15.89:0/2715433323 wait complete. 2023-10-26T06:54:04.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.148+0000 7f1efddff700 1 Processor -- start 2023-10-26T06:54:04.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.148+0000 7f1efddff700 1 -- start start 2023-10-26T06:54:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1efddff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 0x7f1ef8198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1efddff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 0x7f1ef8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1efddff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ef8106750 0x7f1ef819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1efddff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819dce0 con 0x7f1ef8100740 2023-10-26T06:54:04.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1efddff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819de50 con 0x7f1ef8106750 2023-10-26T06:54:04.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1efddff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819e010 con 0x7f1ef8101100 2023-10-26T06:54:04.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 0x7f1ef8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 0x7f1ef8198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 0x7f1ef8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57306/0 (socket says 172.21.15.89:57306) 2023-10-26T06:54:04.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/3035111849 learned_addr learned my addr 172.21.15.89:0/3035111849 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:04.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef7fff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ef8106750 0x7f1ef819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/3035111849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ef8106750 msgr2=0x7f1ef819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ef8106750 0x7f1ef819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 msgr2=0x7f1ef8198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 0x7f1ef8198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/3035111849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0095f0 con 0x7f1ef8101100 2023-10-26T06:54:04.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.149+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 0x7f1ef8199300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1eec005910 tx=0x7f1eec004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:04.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.150+0000 7f1ef4ff9700 1 -- 172.21.15.89:0/3035111849 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec01d070 con 0x7f1ef8101100 2023-10-26T06:54:04.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.150+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef819e290 con 0x7f1ef8101100 2023-10-26T06:54:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.150+0000 7f1ef4ff9700 1 -- 172.21.15.89:0/3035111849 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eec022470 con 0x7f1ef8101100 2023-10-26T06:54:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.150+0000 7f1ef4ff9700 1 -- 172.21.15.89:0/3035111849 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec017650 con 0x7f1ef8101100 2023-10-26T06:54:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.150+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81a5b30 con 0x7f1ef8101100 2023-10-26T06:54:04.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.153+0000 7f1ef4ff9700 1 -- 172.21.15.89:0/3035111849 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1eec0225e0 con 0x7f1ef8101100 2023-10-26T06:54:04.166 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 Processor -- start 2023-10-26T06:54:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 -- start start 2023-10-26T06:54:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e54777ebe0 0x55e54777efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547779090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.167 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5477795d0 0x55e547779a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476ec6e0 con 0x55e54777ebe0 2023-10-26T06:54:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476d2c60 con 0x55e5477795d0 2023-10-26T06:54:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.155+0000 7f1f116eab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476d0580 con 0x55e54777f500 2023-10-26T06:54:04.168 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef4ff9700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ee00a15f0 0x7f1ee00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547779090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef6ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5477795d0 0x55e547779a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547779090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57320/0 (socket says 172.21.15.89:57320) 2023-10-26T06:54:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 -- 172.21.15.89:0/3664339094 learned_addr learned my addr 172.21.15.89:0/3664339094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef7fff700 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e54777ebe0 0x55e54777efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef4ff9700 1 -- 172.21.15.89:0/3035111849 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1eec0ce480 con 0x7f1ef8101100 2023-10-26T06:54:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef7fff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ee00a15f0 0x7f1ee00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 -- 172.21.15.89:0/3664339094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5477795d0 msgr2=0x55e547779a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5477795d0 0x55e547779a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 -- 172.21.15.89:0/3664339094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e54777ebe0 msgr2=0x55e54777efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e54777ebe0 0x55e54777efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 -- 172.21.15.89:0/3664339094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5476893f0 con 0x55e54777f500 2023-10-26T06:54:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef7fff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ee00a15f0 0x7f1ee00a3ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009fd0 tx=0x7f1ef000b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.156+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547779090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009d00 tx=0x7f1ee8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed9754e11e3171e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:04.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1eddffb700 1 -- 172.21.15.89:0/3664339094 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8004230 con 0x55e54777f500 2023-10-26T06:54:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1eddffb700 1 -- 172.21.15.89:0/3664339094 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ee8009e50 con 0x55e54777f500 2023-10-26T06:54:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1eddffb700 1 -- 172.21.15.89:0/3664339094 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8019740 con 0x55e54777f500 2023-10-26T06:54:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 -- 172.21.15.89:0/3664339094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 msgr2=0x55e547779090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547779090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009d00 tx=0x7f1ee8011720 comp rx=0 tx=0).stop 2023-10-26T06:54:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 -- 172.21.15.89:0/3664339094 shutdown_connections 2023-10-26T06:54:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e54777ebe0 0x55e54777efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5477795d0 0x55e547779a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 --2- 172.21.15.89:0/3664339094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547779090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 -- 172.21.15.89:0/3664339094 >> 172.21.15.89:0/3664339094 conn(0x55e5476ed720 msgr2=0x55e547773b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.157+0000 7f1f116eab80 1 -- 172.21.15.89:0/3664339094 shutdown_connections 2023-10-26T06:54:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.158+0000 7f1f116eab80 1 -- 172.21.15.89:0/3664339094 wait complete. 2023-10-26T06:54:04.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.158+0000 7f1f116eab80 1 -- start start 2023-10-26T06:54:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.158+0000 7f1f116eab80 1 Processor -- start 2023-10-26T06:54:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547782df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1f116eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476cbe70 con 0x55e5477795d0 2023-10-26T06:54:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1f116eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e547668710 con 0x55e54777ebe0 2023-10-26T06:54:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1f116eab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5476683b0 con 0x55e54777f500 2023-10-26T06:54:04.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1f116eab80 10 client.0 _ll_register_callbacks cb 0x55e5475b2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44910/0 (socket says 172.21.15.89:44910) 2023-10-26T06:54:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 -- 172.21.15.89:0/59389708 learned_addr learned my addr 172.21.15.89:0/59389708 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef7fff700 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547782df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 -- 172.21.15.89:0/59389708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 msgr2=0x55e547782df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547782df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 -- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 msgr2=0x55e5477827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.159+0000 7f1ef77fe700 1 -- 172.21.15.89:0/59389708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee8009960 con 0x55e5477795d0 2023-10-26T06:54:04.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477854f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009230 tx=0x7f1ee8004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb15461f770b30b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:04.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1ed6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:04.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/59389708 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8005240 con 0x55e5477795d0 2023-10-26T06:54:04.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/59389708 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1ee8005ce0 con 0x55e5477795d0 2023-10-26T06:54:04.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5476d0ef0 con 0x55e5477795d0 2023-10-26T06:54:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/59389708 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee8003cc0 con 0x55e5477795d0 2023-10-26T06:54:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1f116eab80 10 client.16269 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e5474f6de0 con 0x55e5477795d0 2023-10-26T06:54:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/59389708 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5476d0ef0 con 0x55e5477795d0 2023-10-26T06:54:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/59389708 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1ee8031070 con 0x55e5477795d0 2023-10-26T06:54:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1f116eab80 10 client.16269 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.160+0000 7f1f116eab80 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.161+0000 7f1f116eab80 20 client.16269 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.161+0000 7f1f116eab80 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e547792dd0 0x55e5477951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.161+0000 7f1f116eab80 4 client.16269 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.161+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5477488b0 con 0x55e547792dd0 2023-10-26T06:54:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.161+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e547792dd0 0x55e5477951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.162+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e547792dd0 0x55e5477951c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de28add088b7a5c9 server_cookie=f4aa70e6f10ea4e0 in_seq=0 out_seq=0 2023-10-26T06:54:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.162+0000 7f1ed6ffd700 10 client.16269 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.162+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/59389708 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e5477488b0 con 0x55e547792dd0 2023-10-26T06:54:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.162+0000 7f1ed6ffd700 10 client.16269 handle_command_reply: tid=0 2023-10-26T06:54:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.162+0000 7f1f116eab80 1 client.16269 shutdown 2023-10-26T06:54:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 msgr2=0x55e5477854f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477854f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009230 tx=0x7f1ee8004130 comp rx=0 tx=0).stop 2023-10-26T06:54:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 shutdown_connections 2023-10-26T06:54:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477854f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e547792dd0 0x55e5477951c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.187 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547782df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 --2- 172.21.15.89:0/59389708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1ed6ffd700 0 client.16269 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 >> 172.21.15.89:0/59389708 conn(0x55e5476ed720 msgr2=0x55e547778080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 shutdown_connections 2023-10-26T06:54:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 1 -- 172.21.15.89:0/59389708 wait complete. 2023-10-26T06:54:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.163+0000 7f1f116eab80 20 client.16269 trim_cache size 0 max 16384 2023-10-26T06:54:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.176+0000 7f1f116eab80 1 Processor -- start 2023-10-26T06:54:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.176+0000 7f1f116eab80 1 -- start start 2023-10-26T06:54:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.176+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e547789f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547781610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1f116eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54754c920 con 0x55e5477795d0 2023-10-26T06:54:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1f116eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5477833f0 con 0x55e54777ebe0 2023-10-26T06:54:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1f116eab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e547785c90 con 0x55e54777f500 2023-10-26T06:54:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e547789f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e547789f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44914/0 (socket says 172.21.15.89:44914) 2023-10-26T06:54:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 -- 172.21.15.89:0/4160101680 learned_addr learned my addr 172.21.15.89:0/4160101680 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef7fff700 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547781610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 -- 172.21.15.89:0/4160101680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 msgr2=0x55e547781610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547781610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 -- 172.21.15.89:0/4160101680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 msgr2=0x55e5477810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 -- 172.21.15.89:0/4160101680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5474dfc60 con 0x55e5477795d0 2023-10-26T06:54:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.177+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e547789f10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80095f0 tx=0x7f1ee8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8855647f9b1ea2be server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:04.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1ede7fc700 1 -- 172.21.15.89:0/4160101680 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8004250 con 0x55e5477795d0 2023-10-26T06:54:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1ede7fc700 1 -- 172.21.15.89:0/4160101680 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ee8005c20 con 0x55e5477795d0 2023-10-26T06:54:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1ede7fc700 1 -- 172.21.15.89:0/4160101680 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8004e00 con 0x55e5477795d0 2023-10-26T06:54:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 -- 172.21.15.89:0/4160101680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 msgr2=0x55e547789f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e547789f10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80095f0 tx=0x7f1ee8005a00 comp rx=0 tx=0).stop 2023-10-26T06:54:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 -- 172.21.15.89:0/4160101680 shutdown_connections 2023-10-26T06:54:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e547789f10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777f500 0x55e547781610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 --2- 172.21.15.89:0/4160101680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777ebe0 0x55e5477810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 -- 172.21.15.89:0/4160101680 >> 172.21.15.89:0/4160101680 conn(0x55e5476ed720 msgr2=0x55e5477789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 -- 172.21.15.89:0/4160101680 shutdown_connections 2023-10-26T06:54:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.178+0000 7f1f116eab80 1 -- 172.21.15.89:0/4160101680 wait complete. 2023-10-26T06:54:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.179+0000 7f1f116eab80 1 -- start start 2023-10-26T06:54:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.179+0000 7f1f116eab80 1 Processor -- start 2023-10-26T06:54:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777ebe0 0x55e547788620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1f116eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777f500 0x55e5477dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1f116eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5477e0170 con 0x55e5477795d0 2023-10-26T06:54:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1f116eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5477e02e0 con 0x55e54777f500 2023-10-26T06:54:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1f116eab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5477e04a0 con 0x55e54777ebe0 2023-10-26T06:54:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1f116eab80 10 client.0 _ll_register_callbacks cb 0x55e5475b2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777ebe0 0x55e547788620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777ebe0 0x55e547788620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57354/0 (socket says 172.21.15.89:57354) 2023-10-26T06:54:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/1507586155 learned_addr learned my addr 172.21.15.89:0/1507586155 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef7fff700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777f500 0x55e5477dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777f500 msgr2=0x55e5477dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777f500 0x55e5477dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/1507586155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 msgr2=0x55e5477880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef6ffd700 1 -- 172.21.15.89:0/1507586155 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5474f6de0 con 0x55e54777ebe0 2023-10-26T06:54:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.180+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1ef6ffd700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777ebe0 0x55e547788620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0cbbe0 tx=0x7f1eec0c5aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb21e8d2721a8a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1ed6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:54:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/1507586155 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec09ae00 con 0x55e54777ebe0 2023-10-26T06:54:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/1507586155 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1eec09bba0 con 0x55e54777ebe0 2023-10-26T06:54:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ee8009e50 con 0x55e54777ebe0 2023-10-26T06:54:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/1507586155 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eec0a37a0 con 0x55e54777ebe0 2023-10-26T06:54:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 10 client.26317 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e54751d250 con 0x55e54777ebe0 2023-10-26T06:54:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/1507586155 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ee8009e50 con 0x55e54777ebe0 2023-10-26T06:54:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/1507586155 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1eec09b700 con 0x55e54777ebe0 2023-10-26T06:54:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 10 client.26317 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 10 client.26317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 20 client.26317 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e5477e7a80 0x55e5477e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 4 client.26317 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:54:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.181+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5476c4e50 con 0x55e5477e7a80 2023-10-26T06:54:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.182+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e5477e7a80 0x55e5477e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.182+0000 7f1ed6ffd700 10 client.26317 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.182+0000 7f1ef77fe700 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e5477e7a80 0x55e5477e9e70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9b5cd2236afb9c server_cookie=5939389212807f9 in_seq=0 out_seq=0 2023-10-26T06:54:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.182+0000 7f1ed6ffd700 1 -- 172.21.15.89:0/1507586155 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5476c4e50 con 0x55e5477e7a80 2023-10-26T06:54:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.182+0000 7f1ed6ffd700 10 client.26317 handle_command_reply: tid=0 2023-10-26T06:54:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.182+0000 7f1f116eab80 1 client.26317 shutdown 2023-10-26T06:54:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777ebe0 msgr2=0x55e547788620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777ebe0 0x55e547788620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0cbbe0 tx=0x7f1eec0c5aa0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 shutdown_connections 2023-10-26T06:54:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5477795d0 0x55e5477880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e5477e7a80 0x55e5477e9e70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e54777f500 0x55e5477dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 --2- 172.21.15.89:0/1507586155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e54777ebe0 0x55e547788620 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1ed6ffd700 0 client.26317 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:04.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 >> 172.21.15.89:0/1507586155 conn(0x55e5476ed720 msgr2=0x55e547781090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 shutdown_connections 2023-10-26T06:54:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 1 -- 172.21.15.89:0/1507586155 wait complete. 2023-10-26T06:54:04.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1f116eab80 20 client.26317 trim_cache size 0 max 16384 2023-10-26T06:54:04.216 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:54:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ee00a15f0 msgr2=0x7f1ee00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1efddff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ee00a15f0 0x7f1ee00a3ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009fd0 tx=0x7f1ef000b040 comp rx=0 tx=0).stop 2023-10-26T06:54:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 msgr2=0x7f1ef8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.183+0000 7f1efddff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 0x7f1ef8199300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1eec005910 tx=0x7f1eec004730 comp rx=0 tx=0).stop 2023-10-26T06:54:04.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 shutdown_connections 2023-10-26T06:54:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1ee00a15f0 0x7f1ee00a3ab0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ef8100740 0x7f1ef8198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ef8106750 0x7f1ef819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 --2- 172.21.15.89:0/3035111849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ef8101100 0x7f1ef8199300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 >> 172.21.15.89:0/3035111849 conn(0x7f1ef80fc230 msgr2=0x7f1ef810bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 shutdown_connections 2023-10-26T06:54:04.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:04.184+0000 7f1efddff700 1 -- 172.21.15.89:0/3035111849 wait complete. 2023-10-26T06:54:04.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:04 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1874561890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:54:04.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:04 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1874561890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:54:04.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:04 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1874561890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:54:05.223 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:54:05.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:54:05.435 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 -- 172.21.15.89:0/929916481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f801010e0 msgr2=0x7f2f8010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 --2- 172.21.15.89:0/929916481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f801010e0 0x7f2f8010d5f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009f90 tx=0x7f2f7800b330 comp rx=0 tx=0).stop 2023-10-26T06:54:05.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 -- 172.21.15.89:0/929916481 shutdown_connections 2023-10-26T06:54:05.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 --2- 172.21.15.89:0/929916481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f801010e0 0x7f2f8010d5f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 --2- 172.21.15.89:0/929916481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f80106730 0x7f2f80106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 --2- 172.21.15.89:0/929916481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f80100720 0x7f2f80100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 -- 172.21.15.89:0/929916481 >> 172.21.15.89:0/929916481 conn(0x7f2f800fc1b0 msgr2=0x7f2f800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 -- 172.21.15.89:0/929916481 shutdown_connections 2023-10-26T06:54:05.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.433+0000 7f2f86bff700 1 -- 172.21.15.89:0/929916481 wait complete. 2023-10-26T06:54:05.440 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.434+0000 7f2f86bff700 1 Processor -- start 2023-10-26T06:54:05.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.434+0000 7f2f86bff700 1 -- start start 2023-10-26T06:54:05.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f86bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f80100720 0x7f2f8019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f86bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f801010e0 0x7f2f8019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.441 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f86bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f80106730 0x7f2f80198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f86bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80199340 con 0x7f2f80100720 2023-10-26T06:54:05.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f86bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801994b0 con 0x7f2f80106730 2023-10-26T06:54:05.442 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f86bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80199670 con 0x7f2f801010e0 2023-10-26T06:54:05.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f8499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f80100720 0x7f2f8019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f8499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f80100720 0x7f2f8019ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44952/0 (socket says 172.21.15.89:44952) 2023-10-26T06:54:05.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f8499b700 1 -- 172.21.15.89:0/1566086493 learned_addr learned my addr 172.21.15.89:0/1566086493 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.443 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f801010e0 0x7f2f8019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.435+0000 7f2f8519c700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f80106730 0x7f2f80198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f8499b700 1 -- 172.21.15.89:0/1566086493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f801010e0 msgr2=0x7f2f8019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.444 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f8499b700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f801010e0 0x7f2f8019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f8499b700 1 -- 172.21.15.89:0/1566086493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f80106730 msgr2=0x7f2f80198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f8499b700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f80106730 0x7f2f80198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.445 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f8499b700 1 -- 172.21.15.89:0/1566086493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f78009c40 con 0x7f2f80100720 2023-10-26T06:54:05.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f8499b700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f80100720 0x7f2f8019ead0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e9a0 tx=0x7f2f7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f7dffb700 1 -- 172.21.15.89:0/1566086493 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74009790 con 0x7f2f80100720 2023-10-26T06:54:05.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f7dffb700 1 -- 172.21.15.89:0/1566086493 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f7400ce80 con 0x7f2f80100720 2023-10-26T06:54:05.446 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f7dffb700 1 -- 172.21.15.89:0/1566086493 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74018680 con 0x7f2f80100720 2023-10-26T06:54:05.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f80199950 con 0x7f2f80100720 2023-10-26T06:54:05.447 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.436+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f801a5a50 con 0x7f2f80100720 2023-10-26T06:54:05.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.439+0000 7f2f7dffb700 1 -- 172.21.15.89:0/1566086493 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2f74010430 con 0x7f2f80100720 2023-10-26T06:54:05.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 Processor -- start 2023-10-26T06:54:05.453 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f7dffb700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f700a1670 0x7f2f700a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 -- start start 2023-10-26T06:54:05.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f7dffb700 1 -- 172.21.15.89:0/1566086493 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2f74014070 con 0x7f2f80100720 2023-10-26T06:54:05.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f8519c700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f700a1670 0x7f2f700a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a27c60 0x565186a28040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a22650 0x565186a22ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186995760 con 0x565186a28580 2023-10-26T06:54:05.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56518697bce0 con 0x565186a22650 2023-10-26T06:54:05.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.441+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186979600 con 0x565186a27c60 2023-10-26T06:54:05.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a22110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44960/0 (socket says 172.21.15.89:44960) 2023-10-26T06:54:05.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 -- 172.21.15.89:0/2044225595 learned_addr learned my addr 172.21.15.89:0/2044225595 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f8499b700 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a27c60 0x565186a28040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f8519c700 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a22650 0x565186a22ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f8519c700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f700a1670 0x7f2f700a3b30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2f8019f760 tx=0x7f2f78007d30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 -- 172.21.15.89:0/2044225595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a27c60 msgr2=0x565186a28040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a27c60 0x565186a28040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 -- 172.21.15.89:0/2044225595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a22650 msgr2=0x565186a22ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a22650 0x565186a22ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 -- 172.21.15.89:0/2044225595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565186932470 con 0x565186a28580 2023-10-26T06:54:05.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a22110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009cf0 tx=0x7f2f6c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6773f37a3ee8ca server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.442+0000 7f2f6affd700 1 -- 172.21.15.89:0/2044225595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011d80 con 0x565186a28580 2023-10-26T06:54:05.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f6affd700 1 -- 172.21.15.89:0/2044225595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f6c004510 con 0x565186a28580 2023-10-26T06:54:05.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f6affd700 1 -- 172.21.15.89:0/2044225595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c005280 con 0x565186a28580 2023-10-26T06:54:05.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/2044225595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 msgr2=0x565186a22110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a22110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009cf0 tx=0x7f2f6c011440 comp rx=0 tx=0).stop 2023-10-26T06:54:05.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/2044225595 shutdown_connections 2023-10-26T06:54:05.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a22650 0x565186a22ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a27c60 0x565186a28040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/2044225595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a22110 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/2044225595 >> 172.21.15.89:0/2044225595 conn(0x5651869967a0 msgr2=0x565186a1cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/2044225595 shutdown_connections 2023-10-26T06:54:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.443+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/2044225595 wait complete. 2023-10-26T06:54:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.444+0000 7f2f9a58cb80 1 -- start start 2023-10-26T06:54:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.444+0000 7f2f9a58cb80 1 Processor -- start 2023-10-26T06:54:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a2e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a27c60 0x565186a2b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a28580 0x565186a2bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186974ef0 con 0x565186a27c60 2023-10-26T06:54:05.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186911790 con 0x565186a28580 2023-10-26T06:54:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186911430 con 0x565186a22650 2023-10-26T06:54:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f9a58cb80 10 client.0 _ll_register_callbacks cb 0x56518685bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a27c60 0x565186a2b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a27c60 0x565186a2b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44968/0 (socket says 172.21.15.89:44968) 2023-10-26T06:54:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 -- 172.21.15.89:0/32703325 learned_addr learned my addr 172.21.15.89:0/32703325 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f8499b700 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a2e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f8519c700 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a28580 0x565186a2bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 -- 172.21.15.89:0/32703325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 msgr2=0x565186a2e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a2e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 -- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a28580 msgr2=0x565186a2bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a28580 0x565186a2bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.445+0000 7f2f7ffff700 1 -- 172.21.15.89:0/32703325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c009920 con 0x565186a27c60 2023-10-26T06:54:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a27c60 0x565186a2b7a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c006010 tx=0x7f2f6c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5037a6e5aab5a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f5ffff700 1 -- 172.21.15.89:0/32703325 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011e90 con 0x565186a27c60 2023-10-26T06:54:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565186979f70 con 0x565186a27c60 2023-10-26T06:54:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f5ffff700 1 -- 172.21.15.89:0/32703325 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2f6c004050 con 0x565186a27c60 2023-10-26T06:54:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f5ffff700 1 -- 172.21.15.89:0/32703325 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f6c004210 con 0x565186a27c60 2023-10-26T06:54:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f9a58cb80 10 client.16287 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56518679fe60 con 0x565186a27c60 2023-10-26T06:54:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f5ffff700 1 -- 172.21.15.89:0/32703325 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565186979f70 con 0x565186a27c60 2023-10-26T06:54:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.446+0000 7f2f5ffff700 1 -- 172.21.15.89:0/32703325 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f6c004ff0 con 0x565186a27c60 2023-10-26T06:54:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f9a58cb80 10 client.16287 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f9a58cb80 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f9a58cb80 20 client.16287 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a3be00 0x565186a3e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f9a58cb80 4 client.16287 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:05.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651869f1930 con 0x565186a3be00 2023-10-26T06:54:05.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f8499b700 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a3be00 0x565186a3e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f5ffff700 10 client.16287 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.447+0000 7f2f8499b700 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a3be00 0x565186a3e220 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c07dbfa1fd93181 server_cookie=acb9a0b3eae74e65 in_seq=0 out_seq=0 2023-10-26T06:54:05.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.448+0000 7f2f5ffff700 1 -- 172.21.15.89:0/32703325 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5651869f1930 con 0x565186a3be00 2023-10-26T06:54:05.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.448+0000 7f2f5ffff700 10 client.16287 handle_command_reply: tid=0 2023-10-26T06:54:05.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.448+0000 7f2f9a58cb80 1 client.16287 shutdown 2023-10-26T06:54:05.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.448+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a27c60 msgr2=0x565186a2b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.448+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a27c60 0x565186a2b7a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c006010 tx=0x7f2f6c005f00 comp rx=0 tx=0).stop 2023-10-26T06:54:05.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 shutdown_connections 2023-10-26T06:54:05.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a3be00 0x565186a3e220 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a28580 0x565186a2bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a2e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f5ffff700 0 client.16287 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/32703325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a27c60 0x565186a2b7a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 >> 172.21.15.89:0/32703325 conn(0x5651869967a0 msgr2=0x565186a21100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 shutdown_connections 2023-10-26T06:54:05.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/32703325 wait complete. 2023-10-26T06:54:05.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.449+0000 7f2f9a58cb80 20 client.16287 trim_cache size 0 max 16384 2023-10-26T06:54:05.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.466+0000 7f2f9a58cb80 1 Processor -- start 2023-10-26T06:54:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.466+0000 7f2f9a58cb80 1 -- start start 2023-10-26T06:54:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a37d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651867f59a0 con 0x565186a28580 2023-10-26T06:54:05.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186979f70 con 0x565186a27c60 2023-10-26T06:54:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186a38970 con 0x565186a22650 2023-10-26T06:54:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a25660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44972/0 (socket says 172.21.15.89:44972) 2023-10-26T06:54:05.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f7ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8499b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a37d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 -- 172.21.15.89:0/3581720478 learned_addr learned my addr 172.21.15.89:0/3581720478 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 -- 172.21.15.89:0/3581720478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 msgr2=0x565186a37d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 --2- 172.21.15.89:0/3581720478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a37d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 -- 172.21.15.89:0/3581720478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 msgr2=0x565186a32f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 --2- 172.21.15.89:0/3581720478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.467+0000 7f2f8519c700 1 -- 172.21.15.89:0/3581720478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565186a42e10 con 0x565186a28580 2023-10-26T06:54:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f8519c700 1 --2- 172.21.15.89:0/3581720478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a25660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2f78015000 tx=0x7f2f78020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68121315d4ff37b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f6b7fe700 1 -- 172.21.15.89:0/3581720478 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7800e040 con 0x565186a28580 2023-10-26T06:54:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f6b7fe700 1 -- 172.21.15.89:0/3581720478 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f78007620 con 0x565186a28580 2023-10-26T06:54:05.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f6b7fe700 1 -- 172.21.15.89:0/3581720478 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7801e6c0 con 0x565186a28580 2023-10-26T06:54:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/3581720478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 msgr2=0x565186a25660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/3581720478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a25660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2f78015000 tx=0x7f2f78020040 comp rx=0 tx=0).stop 2023-10-26T06:54:05.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/3581720478 shutdown_connections 2023-10-26T06:54:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/3581720478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a25660 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/3581720478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a37d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/3581720478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.468+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/3581720478 >> 172.21.15.89:0/3581720478 conn(0x5651869967a0 msgr2=0x565186a37680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.469+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/3581720478 shutdown_connections 2023-10-26T06:54:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.469+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/3581720478 wait complete. 2023-10-26T06:54:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.469+0000 7f2f9a58cb80 1 -- start start 2023-10-26T06:54:05.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.469+0000 7f2f9a58cb80 1 Processor -- start 2023-10-26T06:54:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.469+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a323d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.491 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f9a58cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a89bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186a8a180 con 0x565186a28580 2023-10-26T06:54:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186a8a2f0 con 0x565186a27c60 2023-10-26T06:54:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f9a58cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x565186a8a4b0 con 0x565186a22650 2023-10-26T06:54:05.492 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f9a58cb80 10 client.0 _ll_register_callbacks cb 0x56518685bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8519c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a89bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a323d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:57420/0 (socket says 172.21.15.89:57420) 2023-10-26T06:54:05.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 -- 172.21.15.89:0/457980771 learned_addr learned my addr 172.21.15.89:0/457980771 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 -- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 msgr2=0x565186a32910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 -- 172.21.15.89:0/457980771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 msgr2=0x565186a89bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a89bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 -- 172.21.15.89:0/457980771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8010def0 con 0x565186a22650 2023-10-26T06:54:05.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f8499b700 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a323d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2f740966a0 tx=0x7f2f7409a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9789edcce2910b3e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.470+0000 7f2f5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:54:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f5ffff700 1 -- 172.21.15.89:0/457980771 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740a2530 con 0x565186a22650 2023-10-26T06:54:05.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f5ffff700 1 -- 172.21.15.89:0/457980771 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2f740b8300 con 0x565186a22650 2023-10-26T06:54:05.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565186a230c0 con 0x565186a22650 2023-10-26T06:54:05.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f5ffff700 1 -- 172.21.15.89:0/457980771 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f740b84c0 con 0x565186a22650 2023-10-26T06:54:05.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 10 client.26335 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:05.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x565186788ce0 con 0x565186a22650 2023-10-26T06:54:05.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f5ffff700 1 -- 172.21.15.89:0/457980771 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565186a230c0 con 0x565186a22650 2023-10-26T06:54:05.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f5ffff700 1 -- 172.21.15.89:0/457980771 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f740a20a0 con 0x565186a22650 2023-10-26T06:54:05.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 10 client.26335 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:05.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 10 client.26335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:05.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 20 client.26335 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:05.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a91ed0 0x565186a942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 4 client.26335 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:54:05.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.471+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f2f6c004510 con 0x565186a91ed0 2023-10-26T06:54:05.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.472+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a91ed0 0x565186a942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.472+0000 7f2f5ffff700 10 client.26335 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.472+0000 7f2f7ffff700 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a91ed0 0x565186a942f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fae8d38fab95fa0 server_cookie=875fef20b91b174c in_seq=0 out_seq=0 2023-10-26T06:54:05.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.473+0000 7f2f5ffff700 1 -- 172.21.15.89:0/457980771 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f2f6c004510 con 0x565186a91ed0 2023-10-26T06:54:05.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.473+0000 7f2f5ffff700 10 client.26335 handle_command_reply: tid=0 2023-10-26T06:54:05.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.473+0000 7f2f9a58cb80 1 client.26335 shutdown 2023-10-26T06:54:05.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 msgr2=0x565186a323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a323d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2f740966a0 tx=0x7f2f7409a0a0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 shutdown_connections 2023-10-26T06:54:05.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565186a28580 0x565186a89bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x565186a91ed0 0x565186a942f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565186a22650 0x565186a323d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 --2- 172.21.15.89:0/457980771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565186a27c60 0x565186a32910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f5ffff700 0 client.26335 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 >> 172.21.15.89:0/457980771 conn(0x5651869967a0 msgr2=0x565186a25d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 shutdown_connections 2023-10-26T06:54:05.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 1 -- 172.21.15.89:0/457980771 wait complete. 2023-10-26T06:54:05.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f9a58cb80 20 client.26335 trim_cache size 0 max 16384 2023-10-26T06:54:05.505 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:54:05.505 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:54:05.506 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "fd2be56f-7eac-40b0-8137-0354fd7a6f4b", 2023-10-26T06:54:05.506 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:54:05.506 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:54:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f700a1670 msgr2=0x7f2f700a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.474+0000 7f2f86bff700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f700a1670 0x7f2f700a3b30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2f8019f760 tx=0x7f2f78007d30 comp rx=0 tx=0).stop 2023-10-26T06:54:05.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f80100720 msgr2=0x7f2f8019ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f80100720 0x7f2f8019ead0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e9a0 tx=0x7f2f7400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 shutdown_connections 2023-10-26T06:54:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f700a1670 0x7f2f700a3b30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f80100720 0x7f2f8019ead0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f80106730 0x7f2f80198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 --2- 172.21.15.89:0/1566086493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f801010e0 0x7f2f8019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 >> 172.21.15.89:0/1566086493 conn(0x7f2f800fc1b0 msgr2=0x7f2f800fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 shutdown_connections 2023-10-26T06:54:05.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.475+0000 7f2f86bff700 1 -- 172.21.15.89:0/1566086493 wait complete. 2023-10-26T06:54:05.511 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:54:05.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.710+0000 7f7dd63ff700 1 -- 172.21.15.89:0/3358340887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 msgr2=0x7f7dd010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.710+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/3358340887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd010d5f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800b0a0 tx=0x7f7dc800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.711+0000 7f7dd63ff700 1 -- 172.21.15.89:0/3358340887 shutdown_connections 2023-10-26T06:54:05.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.711+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/3358340887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd01010e0 0x7f7dd010d5f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.711+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/3358340887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0106730 0x7f7dd0106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.711+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/3358340887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd0100720 0x7f7dd0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.711+0000 7f7dd63ff700 1 -- 172.21.15.89:0/3358340887 >> 172.21.15.89:0/3358340887 conn(0x7f7dd00fc1b0 msgr2=0x7f7dd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.711+0000 7f7dd63ff700 1 -- 172.21.15.89:0/3358340887 shutdown_connections 2023-10-26T06:54:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.711+0000 7f7dd63ff700 1 -- 172.21.15.89:0/3358340887 wait complete. 2023-10-26T06:54:05.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 Processor -- start 2023-10-26T06:54:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 -- start start 2023-10-26T06:54:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd01010e0 0x7f7dd019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0106730 0x7f7dd01969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd0197050 con 0x7f7dd0100720 2023-10-26T06:54:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd01971c0 con 0x7f7dd01010e0 2023-10-26T06:54:05.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dd63ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd0197380 con 0x7f7dd0106730 2023-10-26T06:54:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dcffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dcffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45000/0 (socket says 172.21.15.89:45000) 2023-10-26T06:54:05.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.712+0000 7f7dcffff700 1 -- 172.21.15.89:0/1493731023 learned_addr learned my addr 172.21.15.89:0/1493731023 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcf7fe700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd01010e0 0x7f7dd019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dd499c700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0106730 0x7f7dd01969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcffff700 1 -- 172.21.15.89:0/1493731023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0106730 msgr2=0x7f7dd01969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcffff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0106730 0x7f7dd01969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcffff700 1 -- 172.21.15.89:0/1493731023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd01010e0 msgr2=0x7f7dd019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcffff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd01010e0 0x7f7dd019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcffff700 1 -- 172.21.15.89:0/1493731023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8009d00 con 0x7f7dd0100720 2023-10-26T06:54:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcffff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd019c7f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000e9a0 tx=0x7f7dc000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcd7fa700 1 -- 172.21.15.89:0/1493731023 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0009790 con 0x7f7dd0100720 2023-10-26T06:54:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcd7fa700 1 -- 172.21.15.89:0/1493731023 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc0010430 con 0x7f7dd0100720 2023-10-26T06:54:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dcd7fa700 1 -- 172.21.15.89:0/1493731023 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0018680 con 0x7f7dd0100720 2023-10-26T06:54:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.713+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd0197660 con 0x7f7dd0100720 2023-10-26T06:54:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.714+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd01a3890 con 0x7f7dd0100720 2023-10-26T06:54:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.716+0000 7f7dcd7fa700 1 -- 172.21.15.89:0/1493731023 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7dc00105a0 con 0x7f7dd0100720 2023-10-26T06:54:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.717+0000 7f7de9c6eb80 1 Processor -- start 2023-10-26T06:54:05.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7de9c6eb80 1 -- start start 2023-10-26T06:54:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7dcd7fa700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a1670 0x7f7db80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c76be0 0x555dd4c76fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c71a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4be46e0 con 0x555dd4c77500 2023-10-26T06:54:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4bcac60 con 0x555dd4c76be0 2023-10-26T06:54:05.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4bc8580 con 0x555dd4c715d0 2023-10-26T06:54:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7dcd7fa700 1 -- 172.21.15.89:0/1493731023 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f7dc0014070 con 0x7f7dd0100720 2023-10-26T06:54:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7dd499c700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a1670 0x7f7db80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7dcffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c76be0 0x555dd4c76fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c71090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45016/0 (socket says 172.21.15.89:45016) 2023-10-26T06:54:05.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.718+0000 7f7dcf7fe700 1 -- 172.21.15.89:0/4129301894 learned_addr learned my addr 172.21.15.89:0/4129301894 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dd499c700 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c71a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dcffff700 1 -- 172.21.15.89:0/4129301894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 msgr2=0x555dd4c71a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dcffff700 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c71a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dcffff700 1 -- 172.21.15.89:0/4129301894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 msgr2=0x555dd4c71090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dcffff700 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c71090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dcffff700 1 -- 172.21.15.89:0/4129301894 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd4b813f0 con 0x555dd4c76be0 2023-10-26T06:54:05.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dcffff700 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c76be0 0x555dd4c76fc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00ca920 tx=0x7f7dc0099b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950683dd695edaef server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7dd499c700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a1670 0x7f7db80a3b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009cd0 tx=0x7f7dc8009c60 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7db67fc700 1 -- 172.21.15.89:0/4129301894 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0077870 con 0x555dd4c76be0 2023-10-26T06:54:05.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7db67fc700 1 -- 172.21.15.89:0/4129301894 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc0077690 con 0x555dd4c76be0 2023-10-26T06:54:05.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.719+0000 7f7db67fc700 1 -- 172.21.15.89:0/4129301894 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00a25d0 con 0x555dd4c76be0 2023-10-26T06:54:05.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/4129301894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c76be0 msgr2=0x555dd4c76fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c76be0 0x555dd4c76fc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00ca920 tx=0x7f7dc0099b10 comp rx=0 tx=0).stop 2023-10-26T06:54:05.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/4129301894 shutdown_connections 2023-10-26T06:54:05.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c71a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c76be0 0x555dd4c76fc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/4129301894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/4129301894 >> 172.21.15.89:0/4129301894 conn(0x555dd4be5720 msgr2=0x555dd4c6bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/4129301894 shutdown_connections 2023-10-26T06:54:05.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.720+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/4129301894 wait complete. 2023-10-26T06:54:05.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 -- start start 2023-10-26T06:54:05.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 Processor -- start 2023-10-26T06:54:05.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c7d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c7a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4bc3e70 con 0x555dd4c76be0 2023-10-26T06:54:05.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4b60710 con 0x555dd4c77500 2023-10-26T06:54:05.744 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4b603b0 con 0x555dd4c715d0 2023-10-26T06:54:05.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.721+0000 7f7de9c6eb80 10 client.0 _ll_register_callbacks cb 0x555dd4aaaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:05.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c7a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.745 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c7a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45024/0 (socket says 172.21.15.89:45024) 2023-10-26T06:54:05.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcf7fe700 1 -- 172.21.15.89:0/3843168906 learned_addr learned my addr 172.21.15.89:0/3843168906 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcffff700 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c7d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dd499c700 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.746 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcffff700 1 -- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 msgr2=0x555dd4c7af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcffff700 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcffff700 1 -- 172.21.15.89:0/3843168906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 msgr2=0x555dd4c7a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcffff700 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c7a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcffff700 1 -- 172.21.15.89:0/3843168906 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd01a3890 con 0x555dd4c715d0 2023-10-26T06:54:05.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7dcffff700 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c7d520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0077800 tx=0x7f7dc0077b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4c09a140eb8e9c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7daf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:54:05.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7daf7fe700 1 -- 172.21.15.89:0/3843168906 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc009a230 con 0x555dd4c715d0 2023-10-26T06:54:05.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.722+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dd4bc8ef0 con 0x555dd4c715d0 2023-10-26T06:54:05.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7daf7fe700 1 -- 172.21.15.89:0/3843168906 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f7dc0077870 con 0x555dd4c715d0 2023-10-26T06:54:05.749 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7daf7fe700 1 -- 172.21.15.89:0/3843168906 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc009e120 con 0x555dd4c715d0 2023-10-26T06:54:05.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 10 client.26347 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:05.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555dd49eede0 con 0x555dd4c715d0 2023-10-26T06:54:05.750 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7daf7fe700 1 -- 172.21.15.89:0/3843168906 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc00b6080 con 0x555dd4c715d0 2023-10-26T06:54:05.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7daf7fe700 1 -- 172.21.15.89:0/3843168906 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7dc00a20b0 con 0x555dd4c715d0 2023-10-26T06:54:05.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 10 client.26347 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:05.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:05.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 20 client.26347 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:05.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4c8aab0 0x555dd4c8cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 4 client.26347 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:05.752 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dd4c408b0 con 0x555dd4c8aab0 2023-10-26T06:54:05.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.723+0000 7f7dcf7fe700 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4c8aab0 0x555dd4c8cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.724+0000 7f7dcf7fe700 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4c8aab0 0x555dd4c8cea0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2b29a3dacbb34 server_cookie=ce041acfa6eb524a in_seq=0 out_seq=0 2023-10-26T06:54:05.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.724+0000 7f7daf7fe700 10 client.26347 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7daf7fe700 1 -- 172.21.15.89:0/3843168906 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555dd4c408b0 con 0x555dd4c8aab0 2023-10-26T06:54:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7daf7fe700 10 client.26347 handle_command_reply: tid=0 2023-10-26T06:54:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 client.26347 shutdown 2023-10-26T06:54:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 msgr2=0x555dd4c7d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c7d520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0077800 tx=0x7f7dc0077b90 comp rx=0 tx=0).stop 2023-10-26T06:54:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 shutdown_connections 2023-10-26T06:54:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4c8aab0 0x555dd4c8cea0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c7d520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/3843168906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c7a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7daf7fe700 0 client.26347 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 >> 172.21.15.89:0/3843168906 conn(0x555dd4be5720 msgr2=0x555dd4c70080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.757 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 shutdown_connections 2023-10-26T06:54:05.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/3843168906 wait complete. 2023-10-26T06:54:05.758 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.725+0000 7f7de9c6eb80 20 client.26347 trim_cache size 0 max 16384 2023-10-26T06:54:05.763 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 Processor -- start 2023-10-26T06:54:05.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 -- start start 2023-10-26T06:54:05.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c81eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.764 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4a44920 con 0x555dd4c76be0 2023-10-26T06:54:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4c7b550 con 0x555dd4c77500 2023-10-26T06:54:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4c7be10 con 0x555dd4c715d0 2023-10-26T06:54:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7dd499c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7dd499c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:33174/0 (socket says 172.21.15.89:33174) 2023-10-26T06:54:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7dd499c700 1 -- 172.21.15.89:0/2483360456 learned_addr learned my addr 172.21.15.89:0/2483360456 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.743+0000 7f7dcffff700 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c81eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7dd499c700 1 -- 172.21.15.89:0/2483360456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 msgr2=0x555dd4c81eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7dd499c700 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c81eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7dd499c700 1 -- 172.21.15.89:0/2483360456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 msgr2=0x555dd4c6b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7dd499c700 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c6b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7dd499c700 1 -- 172.21.15.89:0/2483360456 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd49d7c60 con 0x555dd4c77500 2023-10-26T06:54:05.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7dcf7fe700 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c6b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:05.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7dd499c700 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7a0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8015000 tx=0x7f7dc800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79c6a88a5bad5cb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7db6ffd700 1 -- 172.21.15.89:0/2483360456 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8026070 con 0x555dd4c77500 2023-10-26T06:54:05.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7db6ffd700 1 -- 172.21.15.89:0/2483360456 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc80079a0 con 0x555dd4c77500 2023-10-26T06:54:05.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.744+0000 7f7db6ffd700 1 -- 172.21.15.89:0/2483360456 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800e040 con 0x555dd4c77500 2023-10-26T06:54:05.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/2483360456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 msgr2=0x555dd4c7a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7a0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8015000 tx=0x7f7dc800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/2483360456 shutdown_connections 2023-10-26T06:54:05.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c77500 0x555dd4c7a0a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c715d0 0x555dd4c81eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/2483360456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c76be0 0x555dd4c6b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/2483360456 >> 172.21.15.89:0/2483360456 conn(0x555dd4be5720 msgr2=0x555dd4c70970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/2483360456 shutdown_connections 2023-10-26T06:54:05.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/2483360456 wait complete. 2023-10-26T06:54:05.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 -- start start 2023-10-26T06:54:05.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.745+0000 7f7de9c6eb80 1 Processor -- start 2023-10-26T06:54:05.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c715d0 0x555dd4c80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c76be0 0x555dd4c80ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7de9c6eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4cde8a0 con 0x555dd4c77500 2023-10-26T06:54:05.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4cde9e0 con 0x555dd4c715d0 2023-10-26T06:54:05.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7de9c6eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd4cdeba0 con 0x555dd4c76be0 2023-10-26T06:54:05.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7de9c6eb80 10 client.0 _ll_register_callbacks cb 0x555dd4aaaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:05.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45048/0 (socket says 172.21.15.89:45048) 2023-10-26T06:54:05.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 -- 172.21.15.89:0/122965956 learned_addr learned my addr 172.21.15.89:0/122965956 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:05.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dcffff700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c715d0 0x555dd4c80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dcf7fe700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c76be0 0x555dd4c80ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 -- 172.21.15.89:0/122965956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c76be0 msgr2=0x555dd4c80ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c76be0 0x555dd4c80ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 -- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c715d0 msgr2=0x555dd4c80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:05.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c715d0 0x555dd4c80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dd499c700 1 -- 172.21.15.89:0/122965956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc4009c00 con 0x555dd4c77500 2023-10-26T06:54:05.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.746+0000 7f7dcffff700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c715d0 0x555dd4c80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:05.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7dd499c700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c811e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8007c20 tx=0x7f7dc8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b3ba78fc612326 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:05.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7daf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:05.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7daf7fe700 1 -- 172.21.15.89:0/122965956 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8026040 con 0x555dd4c77500 2023-10-26T06:54:05.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dc0077690 con 0x555dd4c77500 2023-10-26T06:54:05.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7daf7fe700 1 -- 172.21.15.89:0/122965956 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f7dc80050d0 con 0x555dd4c77500 2023-10-26T06:54:05.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7daf7fe700 1 -- 172.21.15.89:0/122965956 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc8020a60 con 0x555dd4c77500 2023-10-26T06:54:05.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7de9c6eb80 10 client.16311 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:05.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f7dd010def0 con 0x555dd4c77500 2023-10-26T06:54:05.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7daf7fe700 1 -- 172.21.15.89:0/122965956 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc803f080 con 0x555dd4c77500 2023-10-26T06:54:05.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.747+0000 7f7daf7fe700 1 -- 172.21.15.89:0/122965956 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7dc8031030 con 0x555dd4c77500 2023-10-26T06:54:05.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7de9c6eb80 10 client.16311 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:05.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7de9c6eb80 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:05.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7de9c6eb80 20 client.16311 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:05.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4ce0130 0x555dd4ce2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:05.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7de9c6eb80 4 client.16311 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:54:05.783 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555dd4bbce50 con 0x555dd4ce0130 2023-10-26T06:54:05.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7dcffff700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4ce0130 0x555dd4ce2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:05.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7dcffff700 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4ce0130 0x555dd4ce2550 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4d94016d2dd2c server_cookie=dd13b84b7f055b95 in_seq=0 out_seq=0 2023-10-26T06:54:05.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.748+0000 7f7daf7fe700 10 client.16311 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7daf7fe700 1 -- 172.21.15.89:0/122965956 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555dd4bbce50 con 0x555dd4ce0130 2023-10-26T06:54:05.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7daf7fe700 10 client.16311 handle_command_reply: tid=0 2023-10-26T06:54:05.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 client.16311 shutdown 2023-10-26T06:54:05.785 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 msgr2=0x555dd4c811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c811e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8007c20 tx=0x7f7dc8007e10 comp rx=0 tx=0).stop 2023-10-26T06:54:05.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 shutdown_connections 2023-10-26T06:54:05.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555dd4c77500 0x555dd4c811e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555dd4ce0130 0x555dd4ce2550 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555dd4c715d0 0x555dd4c80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 --2- 172.21.15.89:0/122965956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555dd4c76be0 0x555dd4c80ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7daf7fe700 0 client.16311 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:05.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 >> 172.21.15.89:0/122965956 conn(0x555dd4be5720 msgr2=0x555dd4c654b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 shutdown_connections 2023-10-26T06:54:05.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 1 -- 172.21.15.89:0/122965956 wait complete. 2023-10-26T06:54:05.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.749+0000 7f7de9c6eb80 20 client.16311 trim_cache size 0 max 16384 2023-10-26T06:54:05.789 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:54:05.790 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:54:05.790 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:54:05.790 INFO:teuthology.orchestra.run.smithi089.stdout: "fd2be56f-7eac-40b0-8137-0354fd7a6f4b": { 2023-10-26T06:54:05.791 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:54:05.791 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "fd2be56f-7eac-40b0-8137-0354fd7a6f4b", 2023-10-26T06:54:05.791 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:54:05.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:54:05.792 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:54:05.792 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:54:05.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a1670 msgr2=0x7f7db80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a1670 0x7f7db80a3b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009cd0 tx=0x7f7dc8009c60 comp rx=0 tx=0).stop 2023-10-26T06:54:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 msgr2=0x7f7dd019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd019c7f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000e9a0 tx=0x7f7dc000ecb0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 shutdown_connections 2023-10-26T06:54:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7db80a1670 0x7f7db80a3b30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd0100720 0x7f7dd019c7f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dd0106730 0x7f7dd01969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 --2- 172.21.15.89:0/1493731023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dd01010e0 0x7f7dd019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 >> 172.21.15.89:0/1493731023 conn(0x7f7dd00fc1b0 msgr2=0x7f7dd00fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 shutdown_connections 2023-10-26T06:54:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:05.750+0000 7f7dd63ff700 1 -- 172.21.15.89:0/1493731023 wait complete. 2023-10-26T06:54:05.797 INFO:tasks.cephfs.filesystem:scrub status for tag:fd2be56f-7eac-40b0-8137-0354fd7a6f4b - {'path': '/', 'tag': 'fd2be56f-7eac-40b0-8137-0354fd7a6f4b', 'options': 'recursive,force'} 2023-10-26T06:54:05.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:05 smithi192 ceph-mon[131661]: pgmap v527: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:54:05.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:05 smithi135 ceph-mon[131403]: pgmap v527: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:54:06.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:05 smithi089 ceph-mon[95031]: pgmap v527: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:54:06.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:06 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:54:06.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:06 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:06.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:06 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:54:06.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:06 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:06.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:06 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:54:06.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:06 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:06.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:06 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:54:06.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:06 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:07.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:06 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:54:07.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:06 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:07.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:06 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:54:07.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:06 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:07.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:07 smithi192 ceph-mon[131661]: pgmap v528: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:54:07.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:07 smithi135 ceph-mon[131403]: pgmap v528: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:54:08.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:07 smithi089 ceph-mon[95031]: pgmap v528: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:54:09.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:09 smithi192 ceph-mon[131661]: pgmap v529: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:54:09.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:09 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:54:09.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:09 smithi135 ceph-mon[131403]: pgmap v529: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:54:09.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:09 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:54:10.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:09 smithi089 ceph-mon[95031]: pgmap v529: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:54:10.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:09 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:54:10.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:10 smithi192 ceph-mon[131661]: osdmap e89: 12 total, 12 up, 12 in 2023-10-26T06:54:10.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:10 smithi135 ceph-mon[131403]: osdmap e89: 12 total, 12 up, 12 in 2023-10-26T06:54:11.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:10 smithi089 ceph-mon[95031]: osdmap e89: 12 total, 12 up, 12 in 2023-10-26T06:54:11.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:11 smithi192 ceph-mon[131661]: pgmap v531: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:54:11.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:11 smithi135 ceph-mon[131403]: pgmap v531: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:54:12.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:11 smithi089 ceph-mon[95031]: pgmap v531: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:54:13.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:13 smithi192 ceph-mon[131661]: pgmap v532: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-26T06:54:13.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:13 smithi135 ceph-mon[131403]: pgmap v532: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-26T06:54:14.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:13 smithi089 ceph-mon[95031]: pgmap v532: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-26T06:54:15.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:15 smithi192 ceph-mon[131661]: pgmap v533: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:15.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:15 smithi192 ceph-mon[131661]: osdmap e90: 12 total, 12 up, 12 in 2023-10-26T06:54:15.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:15 smithi135 ceph-mon[131403]: pgmap v533: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:15.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:15 smithi135 ceph-mon[131403]: osdmap e90: 12 total, 12 up, 12 in 2023-10-26T06:54:16.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:15 smithi089 ceph-mon[95031]: pgmap v533: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:16.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:15 smithi089 ceph-mon[95031]: osdmap e90: 12 total, 12 up, 12 in 2023-10-26T06:54:17.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:17 smithi192 ceph-mon[131661]: pgmap v535: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2023-10-26T06:54:17.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:17 smithi135 ceph-mon[131403]: pgmap v535: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2023-10-26T06:54:18.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:17 smithi089 ceph-mon[95031]: pgmap v535: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2023-10-26T06:54:19.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:19 smithi192 ceph-mon[131661]: pgmap v536: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 244 B/s wr, 2 op/s 2023-10-26T06:54:19.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:19 smithi135 ceph-mon[131403]: pgmap v536: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 244 B/s wr, 2 op/s 2023-10-26T06:54:20.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:19 smithi089 ceph-mon[95031]: pgmap v536: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 244 B/s wr, 2 op/s 2023-10-26T06:54:21.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:21 smithi192 ceph-mon[131661]: pgmap v537: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:21.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:21 smithi135 ceph-mon[131403]: pgmap v537: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:22.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:21 smithi089 ceph-mon[95031]: pgmap v537: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:23.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:23 smithi192 ceph-mon[131661]: pgmap v538: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:23.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:23 smithi135 ceph-mon[131403]: pgmap v538: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:24.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:23 smithi089 ceph-mon[95031]: pgmap v538: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:54:25.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:25 smithi192 ceph-mon[131661]: pgmap v539: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:54:25.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:25 smithi135 ceph-mon[131403]: pgmap v539: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:54:26.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:25 smithi089 ceph-mon[95031]: pgmap v539: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:54:27.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:27 smithi192 ceph-mon[131661]: pgmap v540: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 95 B/s wr, 2 op/s 2023-10-26T06:54:27.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:27 smithi135 ceph-mon[131403]: pgmap v540: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 95 B/s wr, 2 op/s 2023-10-26T06:54:28.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:27 smithi089 ceph-mon[95031]: pgmap v540: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 95 B/s wr, 2 op/s 2023-10-26T06:54:28.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:28 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:54:28.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:28 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:54:29.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:28 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:54:29.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: pgmap v541: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:54:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:54:29.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:54:29.900 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:29 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:54:29.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: pgmap v541: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:54:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.970 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:29.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:54:29.971 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:54:29.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:29 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:54:30.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: pgmap v541: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2023-10-26T06:54:30.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-26T06:54:30.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:54:30.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:54:30.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:29 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:54:30.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:30 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi135 to 196.2M 2023-10-26T06:54:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:30 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-10-26T06:54:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:30 smithi192 ceph-mon[131661]: Unable to set osd_memory_target on smithi135 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-10-26T06:54:30.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:30 smithi192 ceph-mon[131661]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-10-26T06:54:30.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:30 smithi192 ceph-mon[131661]: Adjusting osd_memory_target on smithi089 to 221.9M 2023-10-26T06:54:30.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:30 smithi192 ceph-mon[131661]: Unable to set osd_memory_target on smithi089 to 232709120: error parsing value: Value '232709120' is below minimum 939524096 2023-10-26T06:54:30.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:30 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi135 to 196.2M 2023-10-26T06:54:30.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:30 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-10-26T06:54:30.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:30 smithi135 ceph-mon[131403]: Unable to set osd_memory_target on smithi135 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-10-26T06:54:30.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:30 smithi135 ceph-mon[131403]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-10-26T06:54:30.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:30 smithi135 ceph-mon[131403]: Adjusting osd_memory_target on smithi089 to 221.9M 2023-10-26T06:54:30.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:30 smithi135 ceph-mon[131403]: Unable to set osd_memory_target on smithi089 to 232709120: error parsing value: Value '232709120' is below minimum 939524096 2023-10-26T06:54:31.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:30 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi135 to 196.2M 2023-10-26T06:54:31.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:30 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi192 to 196.2M 2023-10-26T06:54:31.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:30 smithi089 ceph-mon[95031]: Unable to set osd_memory_target on smithi135 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-10-26T06:54:31.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:30 smithi089 ceph-mon[95031]: Unable to set osd_memory_target on smithi192 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-10-26T06:54:31.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:30 smithi089 ceph-mon[95031]: Adjusting osd_memory_target on smithi089 to 221.9M 2023-10-26T06:54:31.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:30 smithi089 ceph-mon[95031]: Unable to set osd_memory_target on smithi089 to 232709120: error parsing value: Value '232709120' is below minimum 939524096 2023-10-26T06:54:31.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:31 smithi192 ceph-mon[131661]: pgmap v542: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:31.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:31 smithi135 ceph-mon[131403]: pgmap v542: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:32.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:31 smithi089 ceph-mon[95031]: pgmap v542: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:33.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:33 smithi192 ceph-mon[131661]: pgmap v543: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:33.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:33 smithi135 ceph-mon[131403]: pgmap v543: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:34.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:33 smithi089 ceph-mon[95031]: pgmap v543: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:35.798 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:54:35.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:35 smithi192 ceph-mon[131661]: pgmap v544: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:54:35.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:35 smithi135 ceph-mon[131403]: pgmap v544: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:54:36.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 -- 172.21.15.89:0/1714871358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 msgr2=0x7f36bc100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 --2- 172.21.15.89:0/1714871358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc100ba0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009990 tx=0x7f36ac009ca0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 -- 172.21.15.89:0/1714871358 shutdown_connections 2023-10-26T06:54:36.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 --2- 172.21.15.89:0/1714871358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36bc1010e0 0x7f36bc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 --2- 172.21.15.89:0/1714871358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36bc106730 0x7f36bc106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 --2- 172.21.15.89:0/1714871358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc100ba0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 -- 172.21.15.89:0/1714871358 >> 172.21.15.89:0/1714871358 conn(0x7f36bc0fc1b0 msgr2=0x7f36bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 -- 172.21.15.89:0/1714871358 shutdown_connections 2023-10-26T06:54:36.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.008+0000 7f36c3bff700 1 -- 172.21.15.89:0/1714871358 wait complete. 2023-10-26T06:54:36.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 Processor -- start 2023-10-26T06:54:36.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 -- start start 2023-10-26T06:54:36.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36bc1010e0 0x7f36bc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36bc106730 0x7f36bc198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc1991f0 con 0x7f36bc100720 2023-10-26T06:54:36.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc199360 con 0x7f36bc106730 2023-10-26T06:54:36.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.009+0000 7f36c3bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc199520 con 0x7f36bc1010e0 2023-10-26T06:54:36.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46580/0 (socket says 172.21.15.89:46580) 2023-10-26T06:54:36.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 -- 172.21.15.89:0/3019683404 learned_addr learned my addr 172.21.15.89:0/3019683404 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c219c700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36bc106730 0x7f36bc198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c119a700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36bc1010e0 0x7f36bc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 -- 172.21.15.89:0/3019683404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36bc1010e0 msgr2=0x7f36bc19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36bc1010e0 0x7f36bc19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 -- 172.21.15.89:0/3019683404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36bc106730 msgr2=0x7f36bc198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36bc106730 0x7f36bc198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 -- 172.21.15.89:0/3019683404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0095f0 con 0x7f36bc100720 2023-10-26T06:54:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c119a700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36bc1010e0 0x7f36bc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36c199b700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc19ea60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e9a0 tx=0x7f36b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.010+0000 7f36b2ffd700 1 -- 172.21.15.89:0/3019683404 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8009790 con 0x7f36bc100720 2023-10-26T06:54:36.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.011+0000 7f36b2ffd700 1 -- 172.21.15.89:0/3019683404 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b800cbd0 con 0x7f36bc100720 2023-10-26T06:54:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.011+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc199800 con 0x7f36bc100720 2023-10-26T06:54:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.011+0000 7f36b2ffd700 1 -- 172.21.15.89:0/3019683404 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8018680 con 0x7f36bc100720 2023-10-26T06:54:36.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.011+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc1a5a50 con 0x7f36bc100720 2023-10-26T06:54:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.014+0000 7f36b2ffd700 1 -- 172.21.15.89:0/3019683404 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f36b800cd40 con 0x7f36bc100720 2023-10-26T06:54:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.015+0000 7f36b2ffd700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36a80a15f0 0x7f36a80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.015+0000 7f36d7484b80 1 Processor -- start 2023-10-26T06:54:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.015+0000 7f36b2ffd700 1 -- 172.21.15.89:0/3019683404 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f36b8014070 con 0x7f36bc100720 2023-10-26T06:54:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.015+0000 7f36d7484b80 1 -- start start 2023-10-26T06:54:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.015+0000 7f36c119a700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36a80a15f0 0x7f36a80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f7f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36d7484b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f76b6e0 con 0x55965f7fe500 2023-10-26T06:54:36.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36d7484b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f751c60 con 0x55965f7f85d0 2023-10-26T06:54:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36d7484b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f74f580 con 0x55965f7fdbe0 2023-10-26T06:54:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36a80a15f0 0x7f36a80a3ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f36ac006010 tx=0x7f36ac00b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.031 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f7f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f7f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46588/0 (socket says 172.21.15.89:46588) 2023-10-26T06:54:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 -- 172.21.15.89:0/787070876 learned_addr learned my addr 172.21.15.89:0/787070876 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c199b700 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c219c700 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 -- 172.21.15.89:0/787070876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 msgr2=0x55965f7fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 -- 172.21.15.89:0/787070876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 msgr2=0x55965f7f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.016+0000 7f36c119a700 1 -- 172.21.15.89:0/787070876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965f7083f0 con 0x55965f7fe500 2023-10-26T06:54:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.017+0000 7f36c119a700 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f7f8090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f36ac015000 tx=0x7f36ac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899dc61a03fcea38 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.017+0000 7f36a7fff700 1 -- 172.21.15.89:0/787070876 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac02b070 con 0x55965f7fe500 2023-10-26T06:54:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.017+0000 7f36a7fff700 1 -- 172.21.15.89:0/787070876 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ac01e040 con 0x55965f7fe500 2023-10-26T06:54:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.017+0000 7f36a7fff700 1 -- 172.21.15.89:0/787070876 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac004d80 con 0x55965f7fe500 2023-10-26T06:54:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 -- 172.21.15.89:0/787070876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 msgr2=0x55965f7f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f7f8090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f36ac015000 tx=0x7f36ac01f000 comp rx=0 tx=0).stop 2023-10-26T06:54:36.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 -- 172.21.15.89:0/787070876 shutdown_connections 2023-10-26T06:54:36.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 --2- 172.21.15.89:0/787070876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f7f8090 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 -- 172.21.15.89:0/787070876 >> 172.21.15.89:0/787070876 conn(0x55965f76c720 msgr2=0x55965f7f2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 -- 172.21.15.89:0/787070876 shutdown_connections 2023-10-26T06:54:36.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 -- 172.21.15.89:0/787070876 wait complete. 2023-10-26T06:54:36.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.018+0000 7f36d7484b80 1 -- start start 2023-10-26T06:54:36.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 1 Processor -- start 2023-10-26T06:54:36.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f804480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fdbe0 0x55965f801810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fe500 0x55965f801f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f74ae70 con 0x55965f7f85d0 2023-10-26T06:54:36.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f6e7710 con 0x55965f7fdbe0 2023-10-26T06:54:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f6e73b0 con 0x55965f7fe500 2023-10-26T06:54:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36d7484b80 10 client.0 _ll_register_callbacks cb 0x55965f631a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36c199b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f804480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36c119a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fdbe0 0x55965f801810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36c199b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f804480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46594/0 (socket says 172.21.15.89:46594) 2023-10-26T06:54:36.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.019+0000 7f36c199b700 1 -- 172.21.15.89:0/2174867725 learned_addr learned my addr 172.21.15.89:0/2174867725 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c219c700 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fe500 0x55965f801f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c199b700 1 -- 172.21.15.89:0/2174867725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fe500 msgr2=0x55965f801f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c199b700 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fe500 0x55965f801f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c199b700 1 -- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fdbe0 msgr2=0x55965f801810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c199b700 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fdbe0 0x55965f801810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c199b700 1 -- 172.21.15.89:0/2174867725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc10def0 con 0x55965f7f85d0 2023-10-26T06:54:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c119a700 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fdbe0 0x55965f801810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:36.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36c199b700 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f804480 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f36b809a290 tx=0x7f36b8099d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc5a7052b9c836b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36a57fa700 1 -- 172.21.15.89:0/2174867725 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b80999f0 con 0x55965f7f85d0 2023-10-26T06:54:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36a57fa700 1 -- 172.21.15.89:0/2174867725 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f36b809e450 con 0x55965f7f85d0 2023-10-26T06:54:36.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55965f74fef0 con 0x55965f7f85d0 2023-10-26T06:54:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36a57fa700 1 -- 172.21.15.89:0/2174867725 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b80a2690 con 0x55965f7f85d0 2023-10-26T06:54:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36d7484b80 10 client.16323 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:36.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55965f575de0 con 0x55965f7f85d0 2023-10-26T06:54:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.020+0000 7f36a57fa700 1 -- 172.21.15.89:0/2174867725 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55965f74fef0 con 0x55965f7f85d0 2023-10-26T06:54:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36a57fa700 1 -- 172.21.15.89:0/2174867725 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f36b80b83f0 con 0x55965f7f85d0 2023-10-26T06:54:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36d7484b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36d7484b80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:36.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36d7484b80 20 client.16323 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:36.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36d7484b80 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f811ee0 0x55965f8142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36d7484b80 4 client.16323 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:36.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55965f7c78b0 con 0x55965f811ee0 2023-10-26T06:54:36.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.021+0000 7f36c119a700 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f811ee0 0x55965f8142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.022+0000 7f36a57fa700 10 client.16323 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.022+0000 7f36c119a700 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f811ee0 0x55965f8142d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d9411d191ed65c3 server_cookie=7e24d93151cce672 in_seq=0 out_seq=0 2023-10-26T06:54:36.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.022+0000 7f36a57fa700 1 -- 172.21.15.89:0/2174867725 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55965f7c78b0 con 0x55965f811ee0 2023-10-26T06:54:36.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.022+0000 7f36a57fa700 10 client.16323 handle_command_reply: tid=0 2023-10-26T06:54:36.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.022+0000 7f36d7484b80 1 client.16323 shutdown 2023-10-26T06:54:36.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 msgr2=0x55965f804480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f804480 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f36b809a290 tx=0x7f36b8099d50 comp rx=0 tx=0).stop 2023-10-26T06:54:36.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 shutdown_connections 2023-10-26T06:54:36.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f804480 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f811ee0 0x55965f8142d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fe500 0x55965f801f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 --2- 172.21.15.89:0/2174867725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fdbe0 0x55965f801810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 >> 172.21.15.89:0/2174867725 conn(0x55965f76c720 msgr2=0x55965f7f2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36a57fa700 0 client.16323 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 shutdown_connections 2023-10-26T06:54:36.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 1 -- 172.21.15.89:0/2174867725 wait complete. 2023-10-26T06:54:36.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.023+0000 7f36d7484b80 20 client.16323 trim_cache size 0 max 16384 2023-10-26T06:54:36.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:35 smithi089 ceph-mon[95031]: pgmap v544: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:54:36.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.040+0000 7f36d7484b80 1 Processor -- start 2023-10-26T06:54:36.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.040+0000 7f36d7484b80 1 -- start start 2023-10-26T06:54:36.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.040+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f8017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f803fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36d7484b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f5cb920 con 0x55965f7fe500 2023-10-26T06:54:36.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36d7484b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f7fe210 con 0x55965f7f85d0 2023-10-26T06:54:36.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36d7484b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f802dc0 con 0x55965f7fdbe0 2023-10-26T06:54:36.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f8017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c219c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f803fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f8017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46236/0 (socket says 172.21.15.89:46236) 2023-10-26T06:54:36.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 -- 172.21.15.89:0/3846071634 learned_addr learned my addr 172.21.15.89:0/3846071634 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c199b700 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 -- 172.21.15.89:0/3846071634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 msgr2=0x55965f7f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 -- 172.21.15.89:0/3846071634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 msgr2=0x55965f803fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f803fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c119a700 1 -- 172.21.15.89:0/3846071634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965f55ec60 con 0x55965f7fdbe0 2023-10-26T06:54:36.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.041+0000 7f36c219c700 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f803fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:36.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36c119a700 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f8017b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f36ac004950 tx=0x7f36ac004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e0da0080911711 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36b0bff700 1 -- 172.21.15.89:0/3846071634 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac02b040 con 0x55965f7fdbe0 2023-10-26T06:54:36.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36b0bff700 1 -- 172.21.15.89:0/3846071634 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ac028040 con 0x55965f7fdbe0 2023-10-26T06:54:36.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36b0bff700 1 -- 172.21.15.89:0/3846071634 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac004ca0 con 0x55965f7fdbe0 2023-10-26T06:54:36.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36d7484b80 1 -- 172.21.15.89:0/3846071634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 msgr2=0x55965f8017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36d7484b80 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f8017b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f36ac004950 tx=0x7f36ac004980 comp rx=0 tx=0).stop 2023-10-26T06:54:36.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36d7484b80 1 -- 172.21.15.89:0/3846071634 shutdown_connections 2023-10-26T06:54:36.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36d7484b80 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7fe500 0x55965f803fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36d7484b80 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7f85d0 0x55965f7f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36d7484b80 1 --2- 172.21.15.89:0/3846071634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f8017b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.042+0000 7f36d7484b80 1 -- 172.21.15.89:0/3846071634 >> 172.21.15.89:0/3846071634 conn(0x55965f76c720 msgr2=0x55965f8010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.043+0000 7f36d7484b80 1 -- 172.21.15.89:0/3846071634 shutdown_connections 2023-10-26T06:54:36.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.043+0000 7f36d7484b80 1 -- 172.21.15.89:0/3846071634 wait complete. 2023-10-26T06:54:36.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.043+0000 7f36d7484b80 1 -- start start 2023-10-26T06:54:36.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.043+0000 7f36d7484b80 1 Processor -- start 2023-10-26T06:54:36.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f808e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36d7484b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fe500 0x55965f7fa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36d7484b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f7fa610 con 0x55965f7f85d0 2023-10-26T06:54:36.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36d7484b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f7fa780 con 0x55965f7fe500 2023-10-26T06:54:36.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36d7484b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965f7fa940 con 0x55965f7fdbe0 2023-10-26T06:54:36.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36d7484b80 10 client.0 _ll_register_callbacks cb 0x55965f631a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:36.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f808e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f808e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46604/0 (socket says 172.21.15.89:46604) 2023-10-26T06:54:36.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 -- 172.21.15.89:0/4089905503 learned_addr learned my addr 172.21.15.89:0/4089905503 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c119a700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c219c700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fe500 0x55965f7fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 -- 172.21.15.89:0/4089905503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 msgr2=0x55965f7f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7f9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 -- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fe500 msgr2=0x55965f7fa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fe500 0x55965f7fa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 -- 172.21.15.89:0/4089905503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b40074c0 con 0x55965f7f85d0 2023-10-26T06:54:36.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c119a700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c219c700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fe500 0x55965f7fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36c199b700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f808e80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f36b80b8750 tx=0x7f36b809a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc11d50b7d505aa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.044+0000 7f36a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:36.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36a57fa700 1 -- 172.21.15.89:0/4089905503 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b809e450 con 0x55965f7f85d0 2023-10-26T06:54:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36a57fa700 1 -- 172.21.15.89:0/4089905503 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f36b80a2690 con 0x55965f7f85d0 2023-10-26T06:54:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55965f7f9040 con 0x55965f7f85d0 2023-10-26T06:54:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36a57fa700 1 -- 172.21.15.89:0/4089905503 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b80b8500 con 0x55965f7f85d0 2023-10-26T06:54:36.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 10 client.16329 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f36ac034040 con 0x55965f7f85d0 2023-10-26T06:54:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36a57fa700 1 -- 172.21.15.89:0/4089905503 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55965f7f9040 con 0x55965f7f85d0 2023-10-26T06:54:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36a57fa700 1 -- 172.21.15.89:0/4089905503 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f36b809f190 con 0x55965f7f85d0 2023-10-26T06:54:36.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 20 client.16329 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f867010 0x55965f869400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 4 client.16329 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:54:36.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.045+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f36ac01e040 con 0x55965f867010 2023-10-26T06:54:36.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36c119a700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f867010 0x55965f869400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36a57fa700 10 client.16329 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36c119a700 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f867010 0x55965f869400 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c71fbe70cf699f server_cookie=5b855041ac9c5d8e in_seq=0 out_seq=0 2023-10-26T06:54:36.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36a57fa700 1 -- 172.21.15.89:0/4089905503 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f36ac01e040 con 0x55965f867010 2023-10-26T06:54:36.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36a57fa700 10 client.16329 handle_command_reply: tid=0 2023-10-26T06:54:36.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 client.16329 shutdown 2023-10-26T06:54:36.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 msgr2=0x55965f808e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f808e80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f36b80b8750 tx=0x7f36b809a840 comp rx=0 tx=0).stop 2023-10-26T06:54:36.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 shutdown_connections 2023-10-26T06:54:36.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55965f7f85d0 0x55965f808e80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55965f867010 0x55965f869400 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55965f7fe500 0x55965f7fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 --2- 172.21.15.89:0/4089905503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965f7fdbe0 0x55965f7f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36a57fa700 0 client.16329 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.046+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 >> 172.21.15.89:0/4089905503 conn(0x55965f76c720 msgr2=0x55965f7fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 shutdown_connections 2023-10-26T06:54:36.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36d7484b80 1 -- 172.21.15.89:0/4089905503 wait complete. 2023-10-26T06:54:36.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36d7484b80 20 client.16329 trim_cache size 0 max 16384 2023-10-26T06:54:36.084 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:54:36.085 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:54:36.085 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:54:36.085 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:54:36.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36a80a15f0 msgr2=0x7f36a80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36a80a15f0 0x7f36a80a3ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f36ac006010 tx=0x7f36ac00b260 comp rx=0 tx=0).stop 2023-10-26T06:54:36.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 msgr2=0x7f36bc19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc19ea60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e9a0 tx=0x7f36b800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 shutdown_connections 2023-10-26T06:54:36.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f36a80a15f0 0x7f36a80a3ab0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36bc100720 0x7f36bc19ea60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36bc106730 0x7f36bc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 --2- 172.21.15.89:0/3019683404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36bc1010e0 0x7f36bc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.047+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 >> 172.21.15.89:0/3019683404 conn(0x7f36bc0fc1b0 msgr2=0x7f36bc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.048+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 shutdown_connections 2023-10-26T06:54:36.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.048+0000 7f36c3bff700 1 -- 172.21.15.89:0/3019683404 wait complete. 2023-10-26T06:54:36.090 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:54:36.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:54:36.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.289+0000 7f1608dff700 1 -- 172.21.15.89:0/960729428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041006a0 msgr2=0x7f1604100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.289+0000 7f1608dff700 1 --2- 172.21.15.89:0/960729428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041006a0 0x7f1604100b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15f80098f0 tx=0x7f15f8009c00 comp rx=0 tx=0).stop 2023-10-26T06:54:36.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.290+0000 7f1608dff700 1 -- 172.21.15.89:0/960729428 shutdown_connections 2023-10-26T06:54:36.292 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.290+0000 7f1608dff700 1 --2- 172.21.15.89:0/960729428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16041066b0 0x7f1604106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.290+0000 7f1608dff700 1 --2- 172.21.15.89:0/960729428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1604101060 0x7f160410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.290+0000 7f1608dff700 1 --2- 172.21.15.89:0/960729428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041006a0 0x7f1604100b20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.290+0000 7f1608dff700 1 -- 172.21.15.89:0/960729428 >> 172.21.15.89:0/960729428 conn(0x7f16040fc170 msgr2=0x7f16040fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.290+0000 7f1608dff700 1 -- 172.21.15.89:0/960729428 shutdown_connections 2023-10-26T06:54:36.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.290+0000 7f1608dff700 1 -- 172.21.15.89:0/960729428 wait complete. 2023-10-26T06:54:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 Processor -- start 2023-10-26T06:54:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 -- start start 2023-10-26T06:54:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16041006a0 0x7f160419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1604101060 0x7f160419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041066b0 0x7f1604198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16041991f0 con 0x7f1604101060 2023-10-26T06:54:36.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1604199360 con 0x7f16041066b0 2023-10-26T06:54:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1608dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1604199520 con 0x7f16041006a0 2023-10-26T06:54:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1601d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1604101060 0x7f160419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1601d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1604101060 0x7f160419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46628/0 (socket says 172.21.15.89:46628) 2023-10-26T06:54:36.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f1601d9b700 1 -- 172.21.15.89:0/2642660433 learned_addr learned my addr 172.21.15.89:0/2642660433 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.291+0000 7f160259c700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16041006a0 0x7f160419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1602d9d700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041066b0 0x7f1604198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1601d9b700 1 -- 172.21.15.89:0/2642660433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16041006a0 msgr2=0x7f160419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1601d9b700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16041006a0 0x7f160419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1601d9b700 1 -- 172.21.15.89:0/2642660433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041066b0 msgr2=0x7f1604198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1601d9b700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041066b0 0x7f1604198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1601d9b700 1 -- 172.21.15.89:0/2642660433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8009520 con 0x7f1604101060 2023-10-26T06:54:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1601d9b700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1604101060 0x7f160419efa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15f8005950 tx=0x7f15f800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f15f37fe700 1 -- 172.21.15.89:0/2642660433 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f801c070 con 0x7f1604101060 2023-10-26T06:54:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16041997a0 con 0x7f1604101060 2023-10-26T06:54:36.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f15f37fe700 1 -- 172.21.15.89:0/2642660433 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15f800bb20 con 0x7f1604101060 2023-10-26T06:54:36.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.292+0000 7f15f37fe700 1 -- 172.21.15.89:0/2642660433 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80038e0 con 0x7f1604101060 2023-10-26T06:54:36.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.293+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16041a5a50 con 0x7f1604101060 2023-10-26T06:54:36.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.294+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1604101de0 con 0x7f1604101060 2023-10-26T06:54:36.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.295+0000 7f15f37fe700 1 -- 172.21.15.89:0/2642660433 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f15f8020030 con 0x7f1604101060 2023-10-26T06:54:36.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.297+0000 7f15f37fe700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15ec0a15f0 0x7f15ec0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.297+0000 7f15f37fe700 1 -- 172.21.15.89:0/2642660433 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f15f80ce0c0 con 0x7f1604101060 2023-10-26T06:54:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.297+0000 7f160259c700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15ec0a15f0 0x7f15ec0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.298+0000 7f160259c700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15ec0a15f0 0x7f15ec0a3ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005fd0 tx=0x7f15f4009310 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.299+0000 7f15f37fe700 1 -- 172.21.15.89:0/2642660433 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f15f8097420 con 0x7f1604101060 2023-10-26T06:54:36.544 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.543+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1604067530 con 0x7f1604101060 2023-10-26T06:54:36.545 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.543+0000 7f15f37fe700 1 -- 172.21.15.89:0/2642660433 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f15f809aa40 con 0x7f1604101060 2023-10-26T06:54:36.545 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:54:36.546 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:54:36.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15ec0a15f0 msgr2=0x7f15ec0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15ec0a15f0 0x7f15ec0a3ab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005fd0 tx=0x7f15f4009310 comp rx=0 tx=0).stop 2023-10-26T06:54:36.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1604101060 msgr2=0x7f160419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1604101060 0x7f160419efa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15f8005950 tx=0x7f15f800fa00 comp rx=0 tx=0).stop 2023-10-26T06:54:36.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 shutdown_connections 2023-10-26T06:54:36.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f15ec0a15f0 0x7f15ec0a3ab0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16041066b0 0x7f1604198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16041006a0 0x7f160419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 --2- 172.21.15.89:0/2642660433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1604101060 0x7f160419efa0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 >> 172.21.15.89:0/2642660433 conn(0x7f16040fc170 msgr2=0x7f160410be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 shutdown_connections 2023-10-26T06:54:36.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.546+0000 7f1608dff700 1 -- 172.21.15.89:0/2642660433 wait complete. 2023-10-26T06:54:36.551 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:54:36.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:54:36.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 -- 172.21.15.89:0/852483537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 msgr2=0x7fdf8010d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 --2- 172.21.15.89:0/852483537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8010d5f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800b0a0 tx=0x7fdf7800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 -- 172.21.15.89:0/852483537 shutdown_connections 2023-10-26T06:54:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 --2- 172.21.15.89:0/852483537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8010d5f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 --2- 172.21.15.89:0/852483537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf80106730 0x7fdf80106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.770 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 --2- 172.21.15.89:0/852483537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 0x7fdf80100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 -- 172.21.15.89:0/852483537 >> 172.21.15.89:0/852483537 conn(0x7fdf800fc1b0 msgr2=0x7fdf800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 -- 172.21.15.89:0/852483537 shutdown_connections 2023-10-26T06:54:36.771 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.767+0000 7fdf883ff700 1 -- 172.21.15.89:0/852483537 wait complete. 2023-10-26T06:54:36.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.768+0000 7fdf883ff700 1 Processor -- start 2023-10-26T06:54:36.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf883ff700 1 -- start start 2023-10-26T06:54:36.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf883ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 0x7fdf8019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf883ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf883ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf80106730 0x7fdf80198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf883ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80199190 con 0x7fdf801010e0 2023-10-26T06:54:36.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf883ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80199300 con 0x7fdf80100720 2023-10-26T06:54:36.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf883ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801994c0 con 0x7fdf80106730 2023-10-26T06:54:36.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf8599a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf8619b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 0x7fdf8019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.776 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf8619b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 0x7fdf8019ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:50606/0 (socket says 172.21.15.89:50606) 2023-10-26T06:54:36.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf8619b700 1 -- 172.21.15.89:0/2377725249 learned_addr learned my addr 172.21.15.89:0/2377725249 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.769+0000 7fdf8599a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46636/0 (socket says 172.21.15.89:46636) 2023-10-26T06:54:36.777 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8699c700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf80106730 0x7fdf80198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8619b700 1 -- 172.21.15.89:0/2377725249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf80106730 msgr2=0x7fdf80198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.778 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8619b700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf80106730 0x7fdf80198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8619b700 1 -- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 msgr2=0x7fdf8019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8619b700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.779 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8619b700 1 -- 172.21.15.89:0/2377725249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009d00 con 0x7fdf80100720 2023-10-26T06:54:36.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8599a700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:36.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8699c700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf80106730 0x7fdf80198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:36.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf8619b700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 0x7fdf8019ea00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00d950 tx=0x7fdf7c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.780 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.770+0000 7fdf777fe700 1 -- 172.21.15.89:0/2377725249 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c009940 con 0x7fdf80100720 2023-10-26T06:54:36.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.771+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf801997a0 con 0x7fdf80100720 2023-10-26T06:54:36.781 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.771+0000 7fdf777fe700 1 -- 172.21.15.89:0/2377725249 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7c010460 con 0x7fdf80100720 2023-10-26T06:54:36.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.771+0000 7fdf777fe700 1 -- 172.21.15.89:0/2377725249 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00f620 con 0x7fdf80100720 2023-10-26T06:54:36.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.771+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf801a59f0 con 0x7fdf80100720 2023-10-26T06:54:36.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.775+0000 7fdf777fe700 1 -- 172.21.15.89:0/2377725249 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdf7c0108d0 con 0x7fdf80100720 2023-10-26T06:54:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.776+0000 7fdf9bd88b80 1 Processor -- start 2023-10-26T06:54:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.776+0000 7fdf9bd88b80 1 -- start start 2023-10-26T06:54:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.776+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.776+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.776+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e493f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.776+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e493686e0 con 0x558e493f55d0 2023-10-26T06:54:36.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.776+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e4934ec60 con 0x558e493fb500 2023-10-26T06:54:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e4934c580 con 0x558e493fabe0 2023-10-26T06:54:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e493f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8699c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.791 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e493f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46648/0 (socket says 172.21.15.89:46648) 2023-10-26T06:54:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf777fe700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8699c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46308/0 (socket says 172.21.15.89:46308) 2023-10-26T06:54:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 -- 172.21.15.89:0/2596248631 learned_addr learned my addr 172.21.15.89:0/2596248631 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8619b700 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8699c700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 -- 172.21.15.89:0/2596248631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 msgr2=0x558e493fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf777fe700 1 -- 172.21.15.89:0/2377725249 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fdf7c0ccf60 con 0x7fdf80100720 2023-10-26T06:54:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 -- 172.21.15.89:0/2596248631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 msgr2=0x558e493f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.777+0000 7fdf8599a700 1 -- 172.21.15.89:0/2596248631 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e493053f0 con 0x558e493f55d0 2023-10-26T06:54:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.778+0000 7fdf8599a700 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e493f5a50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdf70010450 tx=0x7fdf70010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6fa2cbb695e20e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.778+0000 7fdf74bfe700 1 -- 172.21.15.89:0/2596248631 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70011640 con 0x558e493f55d0 2023-10-26T06:54:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.778+0000 7fdf8699c700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009cd0 tx=0x7fdf7800b3b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.778+0000 7fdf74bfe700 1 -- 172.21.15.89:0/2596248631 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf70015070 con 0x558e493f55d0 2023-10-26T06:54:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.778+0000 7fdf74bfe700 1 -- 172.21.15.89:0/2596248631 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700198d0 con 0x558e493f55d0 2023-10-26T06:54:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.778+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2596248631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 msgr2=0x558e493f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.778+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e493f5a50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdf70010450 tx=0x7fdf70010760 comp rx=0 tx=0).stop 2023-10-26T06:54:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2596248631 shutdown_connections 2023-10-26T06:54:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e493f5a50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2596248631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2596248631 >> 172.21.15.89:0/2596248631 conn(0x558e49369720 msgr2=0x558e493efb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2596248631 shutdown_connections 2023-10-26T06:54:36.798 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2596248631 wait complete. 2023-10-26T06:54:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.779+0000 7fdf9bd88b80 1 -- start start 2023-10-26T06:54:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 1 Processor -- start 2023-10-26T06:54:36.799 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49401410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fabe0 0x558e493fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e493fee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.800 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e49347e70 con 0x558e493f55d0 2023-10-26T06:54:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e492e4710 con 0x558e493fabe0 2023-10-26T06:54:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e492e43b0 con 0x558e493fb500 2023-10-26T06:54:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.780+0000 7fdf9bd88b80 10 client.0 _ll_register_callbacks cb 0x558e4922ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:36.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49401410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49401410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46660/0 (socket says 172.21.15.89:46660) 2023-10-26T06:54:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 -- 172.21.15.89:0/2653181215 learned_addr learned my addr 172.21.15.89:0/2653181215 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.802 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8599a700 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fabe0 0x558e493fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8699c700 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e493fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 -- 172.21.15.89:0/2653181215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 msgr2=0x558e493fee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.803 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e493fee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 -- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fabe0 msgr2=0x558e493fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fabe0 0x558e493fe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 -- 172.21.15.89:0/2653181215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf70007cd0 con 0x558e493f55d0 2023-10-26T06:54:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf8619b700 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49401410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c010fd0 tx=0x7fdf7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a8c2846c52bd03 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:36.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.781+0000 7fdf69ffb700 1 -- 172.21.15.89:0/2653181215 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a1ff0 con 0x558e493f55d0 2023-10-26T06:54:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf69ffb700 1 -- 172.21.15.89:0/2653181215 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fdf7c09e040 con 0x558e493f55d0 2023-10-26T06:54:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e4934cef0 con 0x558e493f55d0 2023-10-26T06:54:36.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf69ffb700 1 -- 172.21.15.89:0/2653181215 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7c0caaf0 con 0x558e493f55d0 2023-10-26T06:54:36.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 10 client.16347 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:36.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558e49172de0 con 0x558e493f55d0 2023-10-26T06:54:36.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf69ffb700 1 -- 172.21.15.89:0/2653181215 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7c0b8030 con 0x558e493f55d0 2023-10-26T06:54:36.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf69ffb700 1 -- 172.21.15.89:0/2653181215 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf7c0993e0 con 0x558e493f55d0 2023-10-26T06:54:36.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:36.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:36.808 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 20 client.16347 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e4940ee60 0x558e49411250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 4 client.16347 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:36.809 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.782+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e493c48b0 con 0x558e4940ee60 2023-10-26T06:54:36.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.783+0000 7fdf8599a700 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e4940ee60 0x558e49411250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.783+0000 7fdf69ffb700 10 client.16347 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.783+0000 7fdf8599a700 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e4940ee60 0x558e49411250 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a1da73f394d626 server_cookie=bb20a05fc984562 in_seq=0 out_seq=0 2023-10-26T06:54:36.810 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf69ffb700 1 -- 172.21.15.89:0/2653181215 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e493c48b0 con 0x558e4940ee60 2023-10-26T06:54:36.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf69ffb700 10 client.16347 handle_command_reply: tid=0 2023-10-26T06:54:36.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 client.16347 shutdown 2023-10-26T06:54:36.811 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 msgr2=0x558e49401410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49401410 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c010fd0 tx=0x7fdf7c000bc0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 shutdown_connections 2023-10-26T06:54:36.812 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49401410 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e4940ee60 0x558e49411250 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e493fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/2653181215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fabe0 0x558e493fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf69ffb700 0 client.16347 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 >> 172.21.15.89:0/2653181215 conn(0x558e49369720 msgr2=0x558e493f4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 shutdown_connections 2023-10-26T06:54:36.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/2653181215 wait complete. 2023-10-26T06:54:36.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.784+0000 7fdf9bd88b80 20 client.16347 trim_cache size 0 max 16384 2023-10-26T06:54:36.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.797+0000 7fdf9bd88b80 1 Processor -- start 2023-10-26T06:54:36.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf9bd88b80 1 -- start start 2023-10-26T06:54:36.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49405eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e491c8920 con 0x558e493f55d0 2023-10-26T06:54:36.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e493ec230 con 0x558e493fb500 2023-10-26T06:54:36.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e494019f0 con 0x558e493fabe0 2023-10-26T06:54:36.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf8619b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49405eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf8619b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49405eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46670/0 (socket says 172.21.15.89:46670) 2023-10-26T06:54:36.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf8619b700 1 -- 172.21.15.89:0/527919141 learned_addr learned my addr 172.21.15.89:0/527919141 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf8699c700 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.798+0000 7fdf8599a700 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf8619b700 1 -- 172.21.15.89:0/527919141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 msgr2=0x558e493fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf8619b700 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf8619b700 1 -- 172.21.15.89:0/527919141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 msgr2=0x558e493fd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf8619b700 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493fd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf8619b700 1 -- 172.21.15.89:0/527919141 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e4915bc60 con 0x558e493f55d0 2023-10-26T06:54:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf8619b700 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49405eb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009520 tx=0x7fdf7c096230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad462ce87f46b3a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf753ff700 1 -- 172.21.15.89:0/527919141 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0caaf0 con 0x558e493f55d0 2023-10-26T06:54:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf753ff700 1 -- 172.21.15.89:0/527919141 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7c0b8040 con 0x558e493f55d0 2023-10-26T06:54:36.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.799+0000 7fdf753ff700 1 -- 172.21.15.89:0/527919141 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c09a150 con 0x558e493f55d0 2023-10-26T06:54:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/527919141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 msgr2=0x558e49405eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49405eb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009520 tx=0x7fdf7c096230 comp rx=0 tx=0).stop 2023-10-26T06:54:36.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/527919141 shutdown_connections 2023-10-26T06:54:36.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493f55d0 0x558e49405eb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493fb500 0x558e493fd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/527919141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fabe0 0x558e493fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.829 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/527919141 >> 172.21.15.89:0/527919141 conn(0x558e49369720 msgr2=0x558e493ef810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/527919141 shutdown_connections 2023-10-26T06:54:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/527919141 wait complete. 2023-10-26T06:54:36.830 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 -- start start 2023-10-26T06:54:36.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.800+0000 7fdf9bd88b80 1 Processor -- start 2023-10-26T06:54:36.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493f55d0 0x558e49405810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.831 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493fabe0 0x558e49405d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf9bd88b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e49403f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e494044e0 con 0x558e493fabe0 2023-10-26T06:54:36.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e49404650 con 0x558e493f55d0 2023-10-26T06:54:36.832 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf9bd88b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e49404810 con 0x558e493fb500 2023-10-26T06:54:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf9bd88b80 10 client.0 _ll_register_callbacks cb 0x558e4922ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493fabe0 0x558e49405d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.833 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493fabe0 0x558e49405d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46686/0 (socket says 172.21.15.89:46686) 2023-10-26T06:54:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 -- 172.21.15.89:0/1777648912 learned_addr learned my addr 172.21.15.89:0/1777648912 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8699c700 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e49403f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.834 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8619b700 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493f55d0 0x558e49405810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 -- 172.21.15.89:0/1777648912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 msgr2=0x558e49403f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e49403f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 -- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493f55d0 msgr2=0x558e49405810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:36.835 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493f55d0 0x558e49405810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.801+0000 7fdf8599a700 1 -- 172.21.15.89:0/1777648912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e49172de0 con 0x558e493fabe0 2023-10-26T06:54:36.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf8599a700 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493fabe0 0x558e49405d50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdf70019440 tx=0x7fdf70005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eab2843d315c6f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:36.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:36.836 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf69ffb700 1 -- 172.21.15.89:0/1777648912 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70028d60 con 0x558e493fabe0 2023-10-26T06:54:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf69ffb700 1 -- 172.21.15.89:0/1777648912 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fdf700052c0 con 0x558e493fabe0 2023-10-26T06:54:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf70015070 con 0x558e493fabe0 2023-10-26T06:54:36.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf69ffb700 1 -- 172.21.15.89:0/1777648912 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf700054a0 con 0x558e493fabe0 2023-10-26T06:54:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf9bd88b80 10 client.16353 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558e49199250 con 0x558e493fabe0 2023-10-26T06:54:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.802+0000 7fdf69ffb700 1 -- 172.21.15.89:0/1777648912 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf70015070 con 0x558e493fabe0 2023-10-26T06:54:36.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf69ffb700 1 -- 172.21.15.89:0/1777648912 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf700037f0 con 0x558e493fabe0 2023-10-26T06:54:36.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf9bd88b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:36.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf9bd88b80 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:36.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf9bd88b80 20 client.16353 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e49463ff0 0x558e49466410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf9bd88b80 4 client.16353 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:54:36.840 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e49340e50 con 0x558e49463ff0 2023-10-26T06:54:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf8619b700 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e49463ff0 0x558e49466410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf8619b700 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e49463ff0 0x558e49466410 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a7c07463fcabd9 server_cookie=ac22ad5780bc9f8d in_seq=0 out_seq=0 2023-10-26T06:54:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.803+0000 7fdf69ffb700 10 client.16353 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.841 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf69ffb700 1 -- 172.21.15.89:0/1777648912 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e49340e50 con 0x558e49463ff0 2023-10-26T06:54:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf69ffb700 10 client.16353 handle_command_reply: tid=0 2023-10-26T06:54:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 client.16353 shutdown 2023-10-26T06:54:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493fabe0 msgr2=0x558e49405d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493fabe0 0x558e49405d50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdf70019440 tx=0x7fdf70005bb0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 shutdown_connections 2023-10-26T06:54:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e49463ff0 0x558e49466410 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e493fb500 0x558e49403f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e493f55d0 0x558e49405810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf69ffb700 0 client.16353 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:36.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 --2- 172.21.15.89:0/1777648912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e493fabe0 0x558e49405d50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 >> 172.21.15.89:0/1777648912 conn(0x558e49369720 msgr2=0x558e493f86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 shutdown_connections 2023-10-26T06:54:36.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 1 -- 172.21.15.89:0/1777648912 wait complete. 2023-10-26T06:54:36.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.804+0000 7fdf9bd88b80 20 client.16353 trim_cache size 0 max 16384 2023-10-26T06:54:36.846 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:54:36.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf6c0a15f0 msgr2=0x7fdf6c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009cd0 tx=0x7fdf7800b3b0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 msgr2=0x7fdf8019ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:36.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 0x7fdf8019ea00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00d950 tx=0x7fdf7c00dc60 comp rx=0 tx=0).stop 2023-10-26T06:54:36.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 shutdown_connections 2023-10-26T06:54:36.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdf6c0a15f0 0x7fdf6c0a3ab0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf80106730 0x7fdf80198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf80100720 0x7fdf8019ea00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 --2- 172.21.15.89:0/2377725249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf801010e0 0x7fdf8019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 >> 172.21.15.89:0/2377725249 conn(0x7fdf800fc1b0 msgr2=0x7fdf801059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 shutdown_connections 2023-10-26T06:54:36.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:36.805+0000 7fdf883ff700 1 -- 172.21.15.89:0/2377725249 wait complete. 2023-10-26T06:54:36.851 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:36 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/2642660433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:54:36.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:36 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/2642660433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:54:36.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:36 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/2642660433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:54:37.852 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:54:37.853 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:54:37.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:37 smithi192 ceph-mon[131661]: pgmap v545: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:37.919 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:37 smithi089 ceph-mon[95031]: pgmap v545: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:37.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:37 smithi135 ceph-mon[131403]: pgmap v545: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 -- 172.21.15.89:0/3263238460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 msgr2=0x7f77d8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 --2- 172.21.15.89:0/3263238460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8100ba0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009990 tx=0x7f77c8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 -- 172.21.15.89:0/3263238460 shutdown_connections 2023-10-26T06:54:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 --2- 172.21.15.89:0/3263238460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d81010e0 0x7f77d810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 --2- 172.21.15.89:0/3263238460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d8106730 0x7f77d8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 --2- 172.21.15.89:0/3263238460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8100ba0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 -- 172.21.15.89:0/3263238460 >> 172.21.15.89:0/3263238460 conn(0x7f77d80fc1b0 msgr2=0x7f77d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 -- 172.21.15.89:0/3263238460 shutdown_connections 2023-10-26T06:54:38.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.064+0000 7f77dffff700 1 -- 172.21.15.89:0/3263238460 wait complete. 2023-10-26T06:54:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.065+0000 7f77dffff700 1 Processor -- start 2023-10-26T06:54:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.065+0000 7f77dffff700 1 -- start start 2023-10-26T06:54:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d81010e0 0x7f77d81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d8106730 0x7f77d819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819dc90 con 0x7f77d8100720 2023-10-26T06:54:38.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dffff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819de00 con 0x7f77d81010e0 2023-10-26T06:54:38.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dffff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819dfc0 con 0x7f77d8106730 2023-10-26T06:54:38.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46704/0 (socket says 172.21.15.89:46704) 2023-10-26T06:54:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 -- 172.21.15.89:0/3079259876 learned_addr learned my addr 172.21.15.89:0/3079259876 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77de59c700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d8106730 0x7f77d819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dd59a700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d81010e0 0x7f77d81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 -- 172.21.15.89:0/3079259876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d8106730 msgr2=0x7f77d819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d8106730 0x7f77d819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 -- 172.21.15.89:0/3079259876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d81010e0 msgr2=0x7f77d81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d81010e0 0x7f77d81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 -- 172.21.15.89:0/3079259876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80095f0 con 0x7f77d8100720 2023-10-26T06:54:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77dd59a700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d81010e0 0x7f77d81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:38.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.066+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8198d70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f77d400e9a0 tx=0x7f77d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.067+0000 7f77ceffd700 1 -- 172.21.15.89:0/3079259876 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4009790 con 0x7f77d8100720 2023-10-26T06:54:38.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.067+0000 7f77ceffd700 1 -- 172.21.15.89:0/3079259876 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d400cbd0 con 0x7f77d8100720 2023-10-26T06:54:38.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.067+0000 7f77ceffd700 1 -- 172.21.15.89:0/3079259876 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4018680 con 0x7f77d8100720 2023-10-26T06:54:38.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.067+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d819e2a0 con 0x7f77d8100720 2023-10-26T06:54:38.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.067+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d81a5ae0 con 0x7f77d8100720 2023-10-26T06:54:38.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.070+0000 7f77ceffd700 1 -- 172.21.15.89:0/3079259876 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f77d400cd40 con 0x7f77d8100720 2023-10-26T06:54:38.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 Processor -- start 2023-10-26T06:54:38.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 -- start start 2023-10-26T06:54:38.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77ceffd700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77c40a1670 0x7f77c40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfcb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77ceffd700 1 -- 172.21.15.89:0/3079259876 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f77d4014070 con 0x7f77d8100720 2023-10-26T06:54:38.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcb580 0x55d35cfc5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfc5650 0x55d35cfc5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cf38760 con 0x55d35cfc5650 2023-10-26T06:54:38.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cf1ece0 con 0x55d35cfcac60 2023-10-26T06:54:38.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77f390cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cf1c600 con 0x55d35cfcb580 2023-10-26T06:54:38.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.072+0000 7f77de59c700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77c40a1670 0x7f77c40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcb580 0x55d35cfc5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcb580 0x55d35cfc5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46368/0 (socket says 172.21.15.89:46368) 2023-10-26T06:54:38.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77ddd9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfcb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.088 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 -- 172.21.15.89:0/1201764411 learned_addr learned my addr 172.21.15.89:0/1201764411 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77de59c700 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfc5650 0x55d35cfc5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 -- 172.21.15.89:0/1201764411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 msgr2=0x55d35cfcb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfcb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 -- 172.21.15.89:0/1201764411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfc5650 msgr2=0x55d35cfc5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfc5650 0x55d35cfc5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 -- 172.21.15.89:0/1201764411 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35ced5470 con 0x55d35cfcb580 2023-10-26T06:54:38.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77dd59a700 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcb580 0x55d35cfc5110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77c800b2e0 tx=0x7f77c8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca565fbd2ff8c3ab server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.073+0000 7f77de59c700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77c40a1670 0x7f77c40a3b30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77d000f870 tx=0x7f77d000fc30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.074+0000 7f77c3fff700 1 -- 172.21.15.89:0/1201764411 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80049e0 con 0x55d35cfcb580 2023-10-26T06:54:38.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.074+0000 7f77c3fff700 1 -- 172.21.15.89:0/1201764411 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c801e040 con 0x55d35cfcb580 2023-10-26T06:54:38.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.074+0000 7f77c3fff700 1 -- 172.21.15.89:0/1201764411 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c800b9b0 con 0x55d35cfcb580 2023-10-26T06:54:38.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.074+0000 7f77f390cb80 1 -- 172.21.15.89:0/1201764411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcb580 msgr2=0x55d35cfc5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.074+0000 7f77f390cb80 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcb580 0x55d35cfc5110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77c800b2e0 tx=0x7f77c8005c00 comp rx=0 tx=0).stop 2023-10-26T06:54:38.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 -- 172.21.15.89:0/1201764411 shutdown_connections 2023-10-26T06:54:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfc5650 0x55d35cfc5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfcb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 --2- 172.21.15.89:0/1201764411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcb580 0x55d35cfc5110 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 -- 172.21.15.89:0/1201764411 >> 172.21.15.89:0/1201764411 conn(0x55d35cf397a0 msgr2=0x55d35cfbfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 -- 172.21.15.89:0/1201764411 shutdown_connections 2023-10-26T06:54:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 -- 172.21.15.89:0/1201764411 wait complete. 2023-10-26T06:54:38.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 -- start start 2023-10-26T06:54:38.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.075+0000 7f77f390cb80 1 Processor -- start 2023-10-26T06:54:38.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcac60 0x55d35cfce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcb580 0x55d35cfcedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77f390cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cf17ef0 con 0x55d35cfcac60 2023-10-26T06:54:38.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77f390cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ceb4790 con 0x55d35cfcb580 2023-10-26T06:54:38.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77f390cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ceb4430 con 0x55d35cfc5650 2023-10-26T06:54:38.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77f390cb80 10 client.0 _ll_register_callbacks cb 0x55d35cdfeb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:38.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77dd59a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcac60 0x55d35cfce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77dd59a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcac60 0x55d35cfce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46726/0 (socket says 172.21.15.89:46726) 2023-10-26T06:54:38.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77dd59a700 1 -- 172.21.15.89:0/950363316 learned_addr learned my addr 172.21.15.89:0/950363316 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77de59c700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcb580 0x55d35cfcedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77ddd9b700 1 -- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcb580 msgr2=0x55d35cfcedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcb580 0x55d35cfcedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77ddd9b700 1 -- 172.21.15.89:0/950363316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcac60 msgr2=0x55d35cfce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcac60 0x55d35cfce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.076+0000 7f77ddd9b700 1 -- 172.21.15.89:0/950363316 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d810def0 con 0x55d35cfc5650 2023-10-26T06:54:38.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77dd59a700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcac60 0x55d35cfce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd1520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f77d40ca9d0 tx=0x7f77d409a960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efe6d77cf9030959 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:54:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77c17fa700 1 -- 172.21.15.89:0/950363316 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40b2070 con 0x55d35cfc5650 2023-10-26T06:54:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d35cf1cf70 con 0x55d35cfc5650 2023-10-26T06:54:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77c17fa700 1 -- 172.21.15.89:0/950363316 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f77d40cd040 con 0x55d35cfc5650 2023-10-26T06:54:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77c17fa700 1 -- 172.21.15.89:0/950363316 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d409e3b0 con 0x55d35cfc5650 2023-10-26T06:54:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77f390cb80 10 client.26404 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d35cd42e60 con 0x55d35cfc5650 2023-10-26T06:54:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.077+0000 7f77c17fa700 1 -- 172.21.15.89:0/950363316 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d35cf1cf70 con 0x55d35cfc5650 2023-10-26T06:54:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77c17fa700 1 -- 172.21.15.89:0/950363316 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77d40b2d20 con 0x55d35cfc5650 2023-10-26T06:54:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77f390cb80 10 client.26404 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77f390cb80 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77f390cb80 20 client.26404 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77f390cb80 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35cfdef10 0x55d35cfe1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77f390cb80 4 client.26404 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:38.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d35cf94930 con 0x55d35cfdef10 2023-10-26T06:54:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.078+0000 7f77dd59a700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35cfdef10 0x55d35cfe1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.079+0000 7f77c17fa700 10 client.26404 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.079+0000 7f77dd59a700 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35cfdef10 0x55d35cfe1300 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dfbe661d1b811c server_cookie=b457f096218b1389 in_seq=0 out_seq=0 2023-10-26T06:54:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.079+0000 7f77c17fa700 1 -- 172.21.15.89:0/950363316 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d35cf94930 con 0x55d35cfdef10 2023-10-26T06:54:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.079+0000 7f77c17fa700 10 client.26404 handle_command_reply: tid=0 2023-10-26T06:54:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.079+0000 7f77f390cb80 1 client.26404 shutdown 2023-10-26T06:54:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 msgr2=0x55d35cfd1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd1520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f77d40ca9d0 tx=0x7f77d409a960 comp rx=0 tx=0).stop 2023-10-26T06:54:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 shutdown_connections 2023-10-26T06:54:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35cfdef10 0x55d35cfe1300 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcb580 0x55d35cfcedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77c17fa700 0 client.26404 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd1520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.109 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 --2- 172.21.15.89:0/950363316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcac60 0x55d35cfce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 >> 172.21.15.89:0/950363316 conn(0x55d35cf397a0 msgr2=0x55d35cfbfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 shutdown_connections 2023-10-26T06:54:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 1 -- 172.21.15.89:0/950363316 wait complete. 2023-10-26T06:54:38.110 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.080+0000 7f77f390cb80 20 client.26404 trim_cache size 0 max 16384 2023-10-26T06:54:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.097+0000 7f77f390cb80 1 Processor -- start 2023-10-26T06:54:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.097+0000 7f77f390cb80 1 -- start start 2023-10-26T06:54:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfc5650 0x55d35cfce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcac60 0x55d35cfd5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfd1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77f390cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cfc01b0 con 0x55d35cfcb580 2023-10-26T06:54:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77f390cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cfbab70 con 0x55d35cfc5650 2023-10-26T06:54:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77f390cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cd989a0 con 0x55d35cfcac60 2023-10-26T06:54:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcac60 0x55d35cfd5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfd1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcac60 0x55d35cfd5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:46388/0 (socket says 172.21.15.89:46388) 2023-10-26T06:54:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 -- 172.21.15.89:0/594555834 learned_addr learned my addr 172.21.15.89:0/594555834 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfc5650 0x55d35cfce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 -- 172.21.15.89:0/594555834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfc5650 msgr2=0x55d35cfce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfc5650 0x55d35cfce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 -- 172.21.15.89:0/594555834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 msgr2=0x55d35cfd1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfd1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 -- 172.21.15.89:0/594555834 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35cfe5f00 con 0x55d35cfcac60 2023-10-26T06:54:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfc5650 0x55d35cfce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.098+0000 7f77dd59a700 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcac60 0x55d35cfd5f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009ce0 tx=0x7f77c800bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c302fd30e68939f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77ccff9700 1 -- 172.21.15.89:0/594555834 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8031d40 con 0x55d35cfcac60 2023-10-26T06:54:38.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77ccff9700 1 -- 172.21.15.89:0/594555834 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c800be60 con 0x55d35cfcac60 2023-10-26T06:54:38.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77ccff9700 1 -- 172.21.15.89:0/594555834 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c800fb00 con 0x55d35cfcac60 2023-10-26T06:54:38.122 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 -- 172.21.15.89:0/594555834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcac60 msgr2=0x55d35cfd5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcac60 0x55d35cfd5f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009ce0 tx=0x7f77c800bb80 comp rx=0 tx=0).stop 2023-10-26T06:54:38.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 -- 172.21.15.89:0/594555834 shutdown_connections 2023-10-26T06:54:38.123 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfd1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfc5650 0x55d35cfce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 --2- 172.21.15.89:0/594555834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfcac60 0x55d35cfd5f60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 -- 172.21.15.89:0/594555834 >> 172.21.15.89:0/594555834 conn(0x55d35cf397a0 msgr2=0x55d35cfce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.124 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.099+0000 7f77f390cb80 1 -- 172.21.15.89:0/594555834 shutdown_connections 2023-10-26T06:54:38.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.100+0000 7f77f390cb80 1 -- 172.21.15.89:0/594555834 wait complete. 2023-10-26T06:54:38.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.100+0000 7f77f390cb80 1 -- start start 2023-10-26T06:54:38.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.100+0000 7f77f390cb80 1 Processor -- start 2023-10-26T06:54:38.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.100+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.100+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfc6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.100+0000 7f77f390cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfc6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77f390cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cfc73c0 con 0x55d35cfcb580 2023-10-26T06:54:38.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77f390cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cfc7530 con 0x55d35cfcac60 2023-10-26T06:54:38.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77f390cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35cfc76f0 con 0x55d35cfc5650 2023-10-26T06:54:38.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77f390cb80 10 client.0 _ll_register_callbacks cb 0x55d35cdfeb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:38.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfc6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77ddd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77dd59a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfc6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77de59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfc6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46738/0 (socket says 172.21.15.89:46738) 2023-10-26T06:54:38.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77de59c700 1 -- 172.21.15.89:0/2084916399 learned_addr learned my addr 172.21.15.89:0/2084916399 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77ddd9b700 1 -- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 msgr2=0x55d35cfc6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfc6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77ddd9b700 1 -- 172.21.15.89:0/2084916399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 msgr2=0x55d35cfc6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfc6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77ddd9b700 1 -- 172.21.15.89:0/2084916399 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d35ced5470 con 0x55d35cfc5650 2023-10-26T06:54:38.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77dd59a700 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfc6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:38.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77ddd9b700 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd5f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77d4096cd0 tx=0x7f77d40cd200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773d6f66e7c900d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.101+0000 7f77c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:54:38.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77c17fa700 1 -- 172.21.15.89:0/2084916399 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40b2040 con 0x55d35cfc5650 2023-10-26T06:54:38.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77c17fa700 1 -- 172.21.15.89:0/2084916399 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f77d40cd040 con 0x55d35cfc5650 2023-10-26T06:54:38.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77c801e040 con 0x55d35cfc5650 2023-10-26T06:54:38.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77c17fa700 1 -- 172.21.15.89:0/2084916399 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d40b8300 con 0x55d35cfc5650 2023-10-26T06:54:38.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 10 client.26410 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:38.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d35cd2bce0 con 0x55d35cfc5650 2023-10-26T06:54:38.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77c17fa700 1 -- 172.21.15.89:0/2084916399 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c801e040 con 0x55d35cfc5650 2023-10-26T06:54:38.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77c17fa700 1 -- 172.21.15.89:0/2084916399 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77d40b2d20 con 0x55d35cfc5650 2023-10-26T06:54:38.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 10 client.26410 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:38.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:38.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 20 client.26410 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:38.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35d034020 0x55d35d036440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 4 client.26410 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:54:38.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.102+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d35cf10ed0 con 0x55d35d034020 2023-10-26T06:54:38.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.103+0000 7f77dd59a700 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35d034020 0x55d35d036440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.103+0000 7f77dd59a700 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35d034020 0x55d35d036440 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7297a796ba9a7470 server_cookie=3a8d283ea8669e92 in_seq=0 out_seq=0 2023-10-26T06:54:38.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.103+0000 7f77c17fa700 10 client.26410 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77c17fa700 1 -- 172.21.15.89:0/2084916399 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d35cf10ed0 con 0x55d35d034020 2023-10-26T06:54:38.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77c17fa700 10 client.26410 handle_command_reply: tid=0 2023-10-26T06:54:38.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77f390cb80 1 client.26410 shutdown 2023-10-26T06:54:38.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 msgr2=0x55d35cfd5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77f390cb80 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd5f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77d4096cd0 tx=0x7f77d40cd200 comp rx=0 tx=0).stop 2023-10-26T06:54:38.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 shutdown_connections 2023-10-26T06:54:38.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77f390cb80 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d35cfcb580 0x55d35cfc6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.104+0000 7f77f390cb80 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d35d034020 0x55d35d036440 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.138 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77f390cb80 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d35cfc5650 0x55d35cfd5f30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77f390cb80 1 --2- 172.21.15.89:0/2084916399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d35cfcac60 0x55d35cfc6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 >> 172.21.15.89:0/2084916399 conn(0x55d35cf397a0 msgr2=0x55d35cfb9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.139 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77c17fa700 0 client.26410 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 shutdown_connections 2023-10-26T06:54:38.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77f390cb80 1 -- 172.21.15.89:0/2084916399 wait complete. 2023-10-26T06:54:38.140 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77f390cb80 20 client.26410 trim_cache size 0 max 16384 2023-10-26T06:54:38.141 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:54:38.141 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:54:38.141 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "0482bb4e-0694-40d3-bf47-11f44008c275", 2023-10-26T06:54:38.142 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:54:38.142 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:54:38.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77c40a1670 msgr2=0x7f77c40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77c40a1670 0x7f77c40a3b30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77d000f870 tx=0x7f77d000fc30 comp rx=0 tx=0).stop 2023-10-26T06:54:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 msgr2=0x7f77d8198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8198d70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f77d400e9a0 tx=0x7f77d400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 shutdown_connections 2023-10-26T06:54:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f77c40a1670 0x7f77c40a3b30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f77d8100720 0x7f77d8198d70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d8106730 0x7f77d819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 --2- 172.21.15.89:0/3079259876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d81010e0 0x7f77d81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.105+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 >> 172.21.15.89:0/3079259876 conn(0x7f77d80fc1b0 msgr2=0x7f77d810bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.106+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 shutdown_connections 2023-10-26T06:54:38.147 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.106+0000 7f77dffff700 1 -- 172.21.15.89:0/3079259876 wait complete. 2023-10-26T06:54:38.147 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:54:38.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.346+0000 7f52411ff700 1 -- 172.21.15.89:0/1198891917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 msgr2=0x7f523c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.346+0000 7f52411ff700 1 --2- 172.21.15.89:0/1198891917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 0x7f523c100ba0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5230009990 tx=0x7f5230009ca0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.347+0000 7f52411ff700 1 -- 172.21.15.89:0/1198891917 shutdown_connections 2023-10-26T06:54:38.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.347+0000 7f52411ff700 1 --2- 172.21.15.89:0/1198891917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c1010e0 0x7f523c10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.347+0000 7f52411ff700 1 --2- 172.21.15.89:0/1198891917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f523c106730 0x7f523c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.347+0000 7f52411ff700 1 --2- 172.21.15.89:0/1198891917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 0x7f523c100ba0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.347+0000 7f52411ff700 1 -- 172.21.15.89:0/1198891917 >> 172.21.15.89:0/1198891917 conn(0x7f523c0fc1b0 msgr2=0x7f523c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.347+0000 7f52411ff700 1 -- 172.21.15.89:0/1198891917 shutdown_connections 2023-10-26T06:54:38.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.347+0000 7f52411ff700 1 -- 172.21.15.89:0/1198891917 wait complete. 2023-10-26T06:54:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.348+0000 7f52411ff700 1 Processor -- start 2023-10-26T06:54:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.348+0000 7f52411ff700 1 -- start start 2023-10-26T06:54:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f52411ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 0x7f523c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f52411ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f523c1010e0 0x7f523c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f52411ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c106730 0x7f523c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f52411ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c19dcd0 con 0x7f523c106730 2023-10-26T06:54:38.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f52411ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c19de40 con 0x7f523c1010e0 2023-10-26T06:54:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f52411ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c19e000 con 0x7f523c100720 2023-10-26T06:54:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c106730 0x7f523c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 0x7f523c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c106730 0x7f523c19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46768/0 (socket says 172.21.15.89:46768) 2023-10-26T06:54:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 -- 172.21.15.89:0/953009062 learned_addr learned my addr 172.21.15.89:0/953009062 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523a59c700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f523c1010e0 0x7f523c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 -- 172.21.15.89:0/953009062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 msgr2=0x7f523c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 0x7f523c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 -- 172.21.15.89:0/953009062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f523c1010e0 msgr2=0x7f523c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f523c1010e0 0x7f523c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.349+0000 7f523b59e700 1 -- 172.21.15.89:0/953009062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234009c40 con 0x7f523c106730 2023-10-26T06:54:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.350+0000 7f523b59e700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c106730 0x7f523c19d690 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f523400c370 tx=0x7f523400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.350+0000 7f5223fff700 1 -- 172.21.15.89:0/953009062 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234017070 con 0x7f523c106730 2023-10-26T06:54:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.350+0000 7f5223fff700 1 -- 172.21.15.89:0/953009062 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523400f040 con 0x7f523c106730 2023-10-26T06:54:38.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.350+0000 7f5223fff700 1 -- 172.21.15.89:0/953009062 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400e050 con 0x7f523c106730 2023-10-26T06:54:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.350+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52300095f0 con 0x7f523c106730 2023-10-26T06:54:38.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.350+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c1a5940 con 0x7f523c106730 2023-10-26T06:54:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.353+0000 7f5223fff700 1 -- 172.21.15.89:0/953009062 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f523400cb60 con 0x7f523c106730 2023-10-26T06:54:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.354+0000 7f5254b0cb80 1 Processor -- start 2023-10-26T06:54:38.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5254b0cb80 1 -- start start 2023-10-26T06:54:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f31fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5223fff700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f52240a15f0 0x7f52240a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f2ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7e9f6e0 con 0x5598f7f31be0 2023-10-26T06:54:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7e85c60 con 0x5598f7f2c5d0 2023-10-26T06:54:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7e83580 con 0x5598f7f32500 2023-10-26T06:54:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f5223fff700 1 -- 172.21.15.89:0/953009062 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f52340ccaf0 con 0x7f523c106730 2023-10-26T06:54:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f523a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f31fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f523ad9d700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f52240a15f0 0x7f52240a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f523a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f31fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46774/0 (socket says 172.21.15.89:46774) 2023-10-26T06:54:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f523a59c700 1 -- 172.21.15.89:0/2752150966 learned_addr learned my addr 172.21.15.89:0/2752150966 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.355+0000 7f523b59e700 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f2ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523a59c700 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523b59e700 1 -- 172.21.15.89:0/2752150966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 msgr2=0x5598f7f2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523b59e700 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f2c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523b59e700 1 -- 172.21.15.89:0/2752150966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 msgr2=0x5598f7f31fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523b59e700 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f31fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523b59e700 1 -- 172.21.15.89:0/2752150966 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598f7e3c3f0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523a59c700 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f31fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:54:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523b59e700 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f2ca50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f52340ca1f0 tx=0x7f52340995c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd303474faffc2b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f523ad9d700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f52240a15f0 0x7f52240a3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f522c009be0 tx=0x7f522c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f52217fa700 1 -- 172.21.15.89:0/2752150966 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340092e0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f52217fa700 1 -- 172.21.15.89:0/2752150966 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523409a180 con 0x5598f7f2c5d0 2023-10-26T06:54:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.356+0000 7f52217fa700 1 -- 172.21.15.89:0/2752150966 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340744f0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 -- 172.21.15.89:0/2752150966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 msgr2=0x5598f7f2ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f2ca50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f52340ca1f0 tx=0x7f52340995c0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 -- 172.21.15.89:0/2752150966 shutdown_connections 2023-10-26T06:54:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f31fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f2ca50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/2752150966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 -- 172.21.15.89:0/2752150966 >> 172.21.15.89:0/2752150966 conn(0x5598f7ea0720 msgr2=0x5598f7f26b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 -- 172.21.15.89:0/2752150966 shutdown_connections 2023-10-26T06:54:38.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.357+0000 7f5254b0cb80 1 -- 172.21.15.89:0/2752150966 wait complete. 2023-10-26T06:54:38.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 -- start start 2023-10-26T06:54:38.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 Processor -- start 2023-10-26T06:54:38.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f2c5d0 0x5598f7f35890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f35dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f32500 0x5598f7f36310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7e7ee70 con 0x5598f7f32500 2023-10-26T06:54:38.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7e1b710 con 0x5598f7f31be0 2023-10-26T06:54:38.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.358+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7e1b3b0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f5254b0cb80 10 client.0 _ll_register_callbacks cb 0x5598f7d65a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:38.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f32500 0x5598f7f36310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f2c5d0 0x5598f7f35890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f32500 0x5598f7f36310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46776/0 (socket says 172.21.15.89:46776) 2023-10-26T06:54:38.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 -- 172.21.15.89:0/272700295 learned_addr learned my addr 172.21.15.89:0/272700295 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523a59c700 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f35dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 -- 172.21.15.89:0/272700295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f2c5d0 msgr2=0x5598f7f35890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f2c5d0 0x5598f7f35890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 -- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 msgr2=0x5598f7f35dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f35dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 -- 172.21.15.89:0/272700295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5230020040 con 0x5598f7f32500 2023-10-26T06:54:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f523b59e700 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f32500 0x5598f7f36310 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5234003d80 tx=0x7f52340996b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ceb4cca8479185 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.359+0000 7f521eff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f521eff5700 1 -- 172.21.15.89:0/272700295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234078070 con 0x5598f7f32500 2023-10-26T06:54:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f521eff5700 1 -- 172.21.15.89:0/272700295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f5234099a80 con 0x5598f7f32500 2023-10-26T06:54:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598f7e83ef0 con 0x5598f7f32500 2023-10-26T06:54:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f521eff5700 1 -- 172.21.15.89:0/272700295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52340a1e10 con 0x5598f7f32500 2023-10-26T06:54:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 10 client.16383 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f523c1a5940 con 0x5598f7f32500 2023-10-26T06:54:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f521eff5700 1 -- 172.21.15.89:0/272700295 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52340b6020 con 0x5598f7f32500 2023-10-26T06:54:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f521eff5700 1 -- 172.21.15.89:0/272700295 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5234004bd0 con 0x5598f7f32500 2023-10-26T06:54:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 10 client.16383 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 20 client.16383 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f45c30 0x5598f7f48020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 4 client.16383 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:54:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.360+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598f7efb8b0 con 0x5598f7f45c30 2023-10-26T06:54:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.361+0000 7f523ad9d700 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f45c30 0x5598f7f48020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.361+0000 7f523ad9d700 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f45c30 0x5598f7f48020 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1629e5a1db9a8e server_cookie=2ab0014f678cbe91 in_seq=0 out_seq=0 2023-10-26T06:54:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.361+0000 7f521eff5700 10 client.16383 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f521eff5700 1 -- 172.21.15.89:0/272700295 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598f7efb8b0 con 0x5598f7f45c30 2023-10-26T06:54:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f521eff5700 10 client.16383 handle_command_reply: tid=0 2023-10-26T06:54:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 client.16383 shutdown 2023-10-26T06:54:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f32500 msgr2=0x5598f7f36310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f32500 0x5598f7f36310 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5234003d80 tx=0x7f52340996b0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 shutdown_connections 2023-10-26T06:54:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f32500 0x5598f7f36310 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f45c30 0x5598f7f48020 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f2c5d0 0x5598f7f35890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/272700295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f35dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f521eff5700 0 client.16383 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 >> 172.21.15.89:0/272700295 conn(0x5598f7ea0720 msgr2=0x5598f7f2b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 shutdown_connections 2023-10-26T06:54:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.362+0000 7f5254b0cb80 1 -- 172.21.15.89:0/272700295 wait complete. 2023-10-26T06:54:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.363+0000 7f5254b0cb80 20 client.16383 trim_cache size 0 max 16384 2023-10-26T06:54:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.379+0000 7f5254b0cb80 1 Processor -- start 2023-10-26T06:54:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f5254b0cb80 1 -- start start 2023-10-26T06:54:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f36130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7f27130 con 0x5598f7f2c5d0 2023-10-26T06:54:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7f21af0 con 0x5598f7f31be0 2023-10-26T06:54:38.399 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7cff920 con 0x5598f7f32500 2023-10-26T06:54:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46786/0 (socket says 172.21.15.89:46786) 2023-10-26T06:54:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 -- 172.21.15.89:0/3243882041 learned_addr learned my addr 172.21.15.89:0/3243882041 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523b59e700 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523a59c700 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f36130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 -- 172.21.15.89:0/3243882041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 msgr2=0x5598f7f309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 -- 172.21.15.89:0/3243882041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 msgr2=0x5598f7f36130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f36130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523ad9d700 1 -- 172.21.15.89:0/3243882041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598f7ca9de0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.380+0000 7f523a59c700 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.381+0000 7f523ad9d700 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3ceb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x5598f7f48860 tx=0x7f522c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821cae7fc998005c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.381+0000 7f5221ffb700 1 -- 172.21.15.89:0/3243882041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c025070 con 0x5598f7f2c5d0 2023-10-26T06:54:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.381+0000 7f5221ffb700 1 -- 172.21.15.89:0/3243882041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f522c011be0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.381+0000 7f5221ffb700 1 -- 172.21.15.89:0/3243882041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c026ca0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3243882041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f2c5d0 msgr2=0x5598f7f3ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3ceb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x5598f7f48860 tx=0x7f522c014000 comp rx=0 tx=0).stop 2023-10-26T06:54:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3243882041 shutdown_connections 2023-10-26T06:54:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3ceb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3243882041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f31be0 0x5598f7f36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3243882041 >> 172.21.15.89:0/3243882041 conn(0x5598f7ea0720 msgr2=0x5598f7f2ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3243882041 shutdown_connections 2023-10-26T06:54:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3243882041 wait complete. 2023-10-26T06:54:38.406 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 -- start start 2023-10-26T06:54:38.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.382+0000 7f5254b0cb80 1 Processor -- start 2023-10-26T06:54:38.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.407 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f5254b0cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7f3c160 con 0x5598f7f31be0 2023-10-26T06:54:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7f3c2d0 con 0x5598f7f2c5d0 2023-10-26T06:54:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f5254b0cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598f7f999a0 con 0x5598f7f32500 2023-10-26T06:54:38.408 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f5254b0cb80 10 client.0 _ll_register_callbacks cb 0x5598f7d65a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:54:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f3b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46796/0 (socket says 172.21.15.89:46796) 2023-10-26T06:54:38.409 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 -- 172.21.15.89:0/3190454232 learned_addr learned my addr 172.21.15.89:0/3190454232 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:54:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523ad9d700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523b59e700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 -- 172.21.15.89:0/3190454232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 msgr2=0x5598f7f3ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.410 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f3ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 -- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 msgr2=0x5598f7f3cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:54:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523a59c700 1 -- 172.21.15.89:0/3190454232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c017070 con 0x5598f7f31be0 2023-10-26T06:54:38.411 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.383+0000 7f523ad9d700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:54:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f523a59c700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f3b4b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f5230005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a55787ff9740af server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:54:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f521eff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:54:38.412 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f521eff5700 1 -- 172.21.15.89:0/3190454232 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523001d040 con 0x5598f7f31be0 2023-10-26T06:54:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f523409a180 con 0x5598f7f31be0 2023-10-26T06:54:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f521eff5700 1 -- 172.21.15.89:0/3190454232 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f523000b9e0 con 0x5598f7f31be0 2023-10-26T06:54:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f521eff5700 1 -- 172.21.15.89:0/3190454232 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f523000bba0 con 0x5598f7f31be0 2023-10-26T06:54:38.413 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f5254b0cb80 10 client.16389 fetch_fsmap learned FSMap version 20 2023-10-26T06:54:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5598f7c92c60 con 0x5598f7f31be0 2023-10-26T06:54:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.384+0000 7f521eff5700 1 -- 172.21.15.89:0/3190454232 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523409a180 con 0x5598f7f31be0 2023-10-26T06:54:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f521eff5700 1 -- 172.21.15.89:0/3190454232 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5230028460 con 0x5598f7f31be0 2023-10-26T06:54:38.414 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f5254b0cb80 10 client.16389 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:54:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f5254b0cb80 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:54:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f5254b0cb80 20 client.16389 populate_metadata read hostname 'smithi089' 2023-10-26T06:54:38.415 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f9b270 0x5598f7f9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:54:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f5254b0cb80 4 client.16389 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:54:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598f7e77e50 con 0x5598f7f9b270 2023-10-26T06:54:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.385+0000 7f523ad9d700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f9b270 0x5598f7f9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:54:38.416 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f523ad9d700 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f9b270 0x5598f7f9d690 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9185289e3448f87b server_cookie=90d05d7a7c1314ff in_seq=0 out_seq=0 2023-10-26T06:54:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f521eff5700 10 client.16389 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f521eff5700 1 -- 172.21.15.89:0/3190454232 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5598f7e77e50 con 0x5598f7f9b270 2023-10-26T06:54:38.417 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f521eff5700 10 client.16389 handle_command_reply: tid=0 2023-10-26T06:54:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 client.16389 shutdown 2023-10-26T06:54:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 msgr2=0x5598f7f3b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f3b4b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5230000c00 tx=0x7f5230005460 comp rx=0 tx=0).stop 2023-10-26T06:54:38.418 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 shutdown_connections 2023-10-26T06:54:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5598f7f9b270 0x5598f7f9d690 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5598f7f32500 0x5598f7f3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598f7f2c5d0 0x5598f7f3cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.419 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f521eff5700 0 client.16389 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:54:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 --2- 172.21.15.89:0/3190454232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5598f7f31be0 0x5598f7f3b4b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 >> 172.21.15.89:0/3190454232 conn(0x5598f7ea0720 msgr2=0x5598f7f309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.420 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 shutdown_connections 2023-10-26T06:54:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 1 -- 172.21.15.89:0/3190454232 wait complete. 2023-10-26T06:54:38.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.386+0000 7f5254b0cb80 20 client.16389 trim_cache size 0 max 16384 2023-10-26T06:54:38.421 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:54:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:54:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:54:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "0482bb4e-0694-40d3-bf47-11f44008c275": { 2023-10-26T06:54:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:54:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "0482bb4e-0694-40d3-bf47-11f44008c275", 2023-10-26T06:54:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:54:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:54:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:54:38.425 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:54:38.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f52240a15f0 msgr2=0x7f52240a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f52240a15f0 0x7f52240a3ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f522c009be0 tx=0x7f522c009260 comp rx=0 tx=0).stop 2023-10-26T06:54:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c106730 msgr2=0x7f523c19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:54:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c106730 0x7f523c19d690 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f523400c370 tx=0x7f523400c680 comp rx=0 tx=0).stop 2023-10-26T06:54:38.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 shutdown_connections 2023-10-26T06:54:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f52240a15f0 0x7f52240a3ab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f523c106730 0x7f523c19d690 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f523c100720 0x7f523c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 --2- 172.21.15.89:0/953009062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f523c1010e0 0x7f523c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:54:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 >> 172.21.15.89:0/953009062 conn(0x7f523c0fc1b0 msgr2=0x7f523c10be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:54:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 shutdown_connections 2023-10-26T06:54:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:54:38.387+0000 7f52411ff700 1 -- 172.21.15.89:0/953009062 wait complete. 2023-10-26T06:54:38.430 INFO:tasks.cephfs.filesystem:scrub status for tag:0482bb4e-0694-40d3-bf47-11f44008c275 - {'path': '/', 'tag': '0482bb4e-0694-40d3-bf47-11f44008c275', 'options': 'recursive,force'} 2023-10-26T06:54:38.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:38 smithi135 ceph-mon[131403]: pgmap v546: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:39.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:38 smithi089 ceph-mon[95031]: pgmap v546: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:39.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:38 smithi192 ceph-mon[131661]: pgmap v546: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:39.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:39 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:54:39.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:39 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:39.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:39 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:54:39.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:39 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:39.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:39 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:54:40.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:39 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:54:40.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:39 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:40.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:39 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:54:40.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:39 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:40.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:39 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:54:40.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:39 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:54:40.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:39 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:40.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:39 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:54:40.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:39 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:54:40.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:39 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:54:41.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:41 smithi089 ceph-mon[95031]: pgmap v547: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:54:41.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:41 smithi192 ceph-mon[131661]: pgmap v547: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:54:41.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:41 smithi135 ceph-mon[131403]: pgmap v547: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:54:43.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:43 smithi089 ceph-mon[95031]: pgmap v548: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:54:43.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:43 smithi192 ceph-mon[131661]: pgmap v548: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:54:43.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:43 smithi135 ceph-mon[131403]: pgmap v548: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:54:45.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:45 smithi089 ceph-mon[95031]: pgmap v549: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:54:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:45 smithi192 ceph-mon[131661]: pgmap v549: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:54:45.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:45 smithi135 ceph-mon[131403]: pgmap v549: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:54:47.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:47 smithi089 ceph-mon[95031]: pgmap v550: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:54:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:47 smithi192 ceph-mon[131661]: pgmap v550: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:54:47.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:47 smithi135 ceph-mon[131403]: pgmap v550: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:54:49.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:49 smithi089 ceph-mon[95031]: pgmap v551: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:54:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:49 smithi192 ceph-mon[131661]: pgmap v551: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:54:49.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:49 smithi135 ceph-mon[131403]: pgmap v551: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:54:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:51 smithi192 ceph-mon[131661]: pgmap v552: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:54:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:51 smithi135 ceph-mon[131403]: pgmap v552: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:54:51.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:51 smithi089 ceph-mon[95031]: pgmap v552: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:54:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:53 smithi192 ceph-mon[131661]: pgmap v553: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:54:53.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:53 smithi135 ceph-mon[131403]: pgmap v553: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:54:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:53 smithi089 ceph-mon[95031]: pgmap v553: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:54:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:55 smithi192 ceph-mon[131661]: pgmap v554: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:54:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:55 smithi135 ceph-mon[131403]: pgmap v554: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:54:55.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:55 smithi089 ceph-mon[95031]: pgmap v554: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:54:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:57 smithi192 ceph-mon[131661]: pgmap v555: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:57 smithi135 ceph-mon[131403]: pgmap v555: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:57.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:57 smithi089 ceph-mon[95031]: pgmap v555: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:54:59.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:54:59 smithi192 ceph-mon[131661]: pgmap v556: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:54:59 smithi135 ceph-mon[131403]: pgmap v556: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:54:59.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:54:59 smithi089 ceph-mon[95031]: pgmap v556: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:55:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:01 smithi192 ceph-mon[131661]: pgmap v557: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:01 smithi135 ceph-mon[131403]: pgmap v557: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:01 smithi089 ceph-mon[95031]: pgmap v557: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:03 smithi192 ceph-mon[131661]: pgmap v558: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 22 KiB/s wr, 4 op/s 2023-10-26T06:55:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:03 smithi135 ceph-mon[131403]: pgmap v558: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 22 KiB/s wr, 4 op/s 2023-10-26T06:55:03.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:03 smithi089 ceph-mon[95031]: pgmap v558: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 22 KiB/s wr, 4 op/s 2023-10-26T06:55:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:05 smithi192 ceph-mon[131661]: pgmap v559: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:55:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:05 smithi135 ceph-mon[131403]: pgmap v559: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:55:05.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:05 smithi089 ceph-mon[95031]: pgmap v559: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:55:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:07 smithi192 ceph-mon[131661]: pgmap v560: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:55:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:07 smithi135 ceph-mon[131403]: pgmap v560: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:55:07.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:07 smithi089 ceph-mon[95031]: pgmap v560: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-10-26T06:55:08.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:55:08.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 -- 172.21.15.89:0/2608357622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1681032f0 msgr2=0x7fe16810f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/2608357622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1681032f0 0x7fe16810f800 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b0a0 tx=0x7fe16000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 -- 172.21.15.89:0/2608357622 shutdown_connections 2023-10-26T06:55:08.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/2608357622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe1681032f0 0x7fe16810f800 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/2608357622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168108940 0x7fe168108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/2608357622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe168102930 0x7fe168102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 -- 172.21.15.89:0/2608357622 >> 172.21.15.89:0/2608357622 conn(0x7fe1680fe3e0 msgr2=0x7fe168100800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:08.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 -- 172.21.15.89:0/2608357622 shutdown_connections 2023-10-26T06:55:08.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.639+0000 7fe16d7ff700 1 -- 172.21.15.89:0/2608357622 wait complete. 2023-10-26T06:55:08.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 Processor -- start 2023-10-26T06:55:08.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 -- start start 2023-10-26T06:55:08.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168102930 0x7fe168198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1681032f0 0x7fe168199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe168108940 0x7fe16819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819dcc0 con 0x7fe168108940 2023-10-26T06:55:08.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819de30 con 0x7fe1681032f0 2023-10-26T06:55:08.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.640+0000 7fe16d7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16819dff0 con 0x7fe168102930 2023-10-26T06:55:08.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe168108940 0x7fe16819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1667fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1681032f0 0x7fe168199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe168108940 0x7fe16819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54008/0 (socket says 172.21.15.89:54008) 2023-10-26T06:55:08.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1667fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1681032f0 0x7fe168199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:55922/0 (socket says 172.21.15.89:55922) 2023-10-26T06:55:08.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 -- 172.21.15.89:0/4288898118 learned_addr learned my addr 172.21.15.89:0/4288898118 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:08.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe166ffd700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168102930 0x7fe168198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 -- 172.21.15.89:0/4288898118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168102930 msgr2=0x7fe168198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168102930 0x7fe168198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 -- 172.21.15.89:0/4288898118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1681032f0 msgr2=0x7fe168199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1681032f0 0x7fe168199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 -- 172.21.15.89:0/4288898118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160009d00 con 0x7fe168108940 2023-10-26T06:55:08.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1667fc700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1681032f0 0x7fe168199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:08.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe1677fe700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe168108940 0x7fe16819d680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b070 tx=0x7fe1600077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.641+0000 7fe14ffff700 1 -- 172.21.15.89:0/4288898118 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16000e040 con 0x7fe168108940 2023-10-26T06:55:08.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.642+0000 7fe14ffff700 1 -- 172.21.15.89:0/4288898118 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1600094c0 con 0x7fe168108940 2023-10-26T06:55:08.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.642+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16819e270 con 0x7fe168108940 2023-10-26T06:55:08.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.642+0000 7fe14ffff700 1 -- 172.21.15.89:0/4288898118 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1600043a0 con 0x7fe168108940 2023-10-26T06:55:08.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.642+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1681a5b10 con 0x7fe168108940 2023-10-26T06:55:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.645+0000 7fe14ffff700 1 -- 172.21.15.89:0/4288898118 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe160004500 con 0x7fe168108940 2023-10-26T06:55:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 Processor -- start 2023-10-26T06:55:08.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 -- start start 2023-10-26T06:55:08.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe14ffff700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1500a1670 0x7fe1500a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca7efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe14ffff700 1 -- 172.21.15.89:0/4288898118 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fe1600cdf30 con 0x7fe168108940 2023-10-26T06:55:08.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca79a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcc9ec6e0 con 0x55edcca7f500 2023-10-26T06:55:08.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcc9d2c60 con 0x55edcca795d0 2023-10-26T06:55:08.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.646+0000 7fe1810aab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcc9d0580 con 0x55edcca7ebe0 2023-10-26T06:55:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe1667fc700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1500a1670 0x7fe1500a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca7efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca7efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:35504/0 (socket says 172.21.15.89:35504) 2023-10-26T06:55:08.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 -- 172.21.15.89:0/1790289252 learned_addr learned my addr 172.21.15.89:0/1790289252 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe1667fc700 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe1677fe700 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca79a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 -- 172.21.15.89:0/1790289252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 msgr2=0x55edcca79a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca79a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 -- 172.21.15.89:0/1790289252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 msgr2=0x55edcca79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 -- 172.21.15.89:0/1790289252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edcc9893f0 con 0x55edcca7ebe0 2023-10-26T06:55:08.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe1667fc700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1500a1670 0x7fe1500a3b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097c0 tx=0x7fe15c009f70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.647+0000 7fe166ffd700 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca7efc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe158009d00 tx=0x7fe158011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89f83b4568180ba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.648+0000 7fe14d7fa700 1 -- 172.21.15.89:0/1790289252 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158004230 con 0x55edcca7ebe0 2023-10-26T06:55:08.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.648+0000 7fe14d7fa700 1 -- 172.21.15.89:0/1790289252 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe158004390 con 0x55edcca7ebe0 2023-10-26T06:55:08.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.648+0000 7fe14d7fa700 1 -- 172.21.15.89:0/1790289252 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158019750 con 0x55edcca7ebe0 2023-10-26T06:55:08.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.648+0000 7fe1810aab80 1 -- 172.21.15.89:0/1790289252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 msgr2=0x55edcca7efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.648+0000 7fe1810aab80 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca7efc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe158009d00 tx=0x7fe158011720 comp rx=0 tx=0).stop 2023-10-26T06:55:08.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 -- 172.21.15.89:0/1790289252 shutdown_connections 2023-10-26T06:55:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca79a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca7efc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 --2- 172.21.15.89:0/1790289252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 -- 172.21.15.89:0/1790289252 >> 172.21.15.89:0/1790289252 conn(0x55edcc9ed720 msgr2=0x55edcca73b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 -- 172.21.15.89:0/1790289252 shutdown_connections 2023-10-26T06:55:08.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 -- 172.21.15.89:0/1790289252 wait complete. 2023-10-26T06:55:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 -- start start 2023-10-26T06:55:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.649+0000 7fe1810aab80 1 Processor -- start 2023-10-26T06:55:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca795d0 0x55edcca85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca7f500 0x55edcca82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1810aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcc9cbe70 con 0x55edcca795d0 2023-10-26T06:55:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1810aab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcc968710 con 0x55edcca7f500 2023-10-26T06:55:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1810aab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcc9683b0 con 0x55edcca7ebe0 2023-10-26T06:55:08.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1810aab80 10 client.0 _ll_register_callbacks cb 0x55edcc8b2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:08.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca795d0 0x55edcca85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca795d0 0x55edcca85440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54032/0 (socket says 172.21.15.89:54032) 2023-10-26T06:55:08.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 -- 172.21.15.89:0/647978335 learned_addr learned my addr 172.21.15.89:0/647978335 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:08.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1667fc700 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe1677fe700 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca7f500 0x55edcca82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 -- 172.21.15.89:0/647978335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 msgr2=0x55edcca826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca826c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 -- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca7f500 msgr2=0x55edcca82cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca7f500 0x55edcca82cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.650+0000 7fe166ffd700 1 -- 172.21.15.89:0/647978335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe158009960 con 0x55edcca795d0 2023-10-26T06:55:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe166ffd700 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca795d0 0x55edcca85440 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe158003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a562b7455259a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe14aff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:55:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe14aff5700 1 -- 172.21.15.89:0/647978335 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158005450 con 0x55edcca795d0 2023-10-26T06:55:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edcc9d0ef0 con 0x55edcca795d0 2023-10-26T06:55:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe14aff5700 1 -- 172.21.15.89:0/647978335 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fe158005ce0 con 0x55edcca795d0 2023-10-26T06:55:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe14aff5700 1 -- 172.21.15.89:0/647978335 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe158004500 con 0x55edcca795d0 2023-10-26T06:55:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe1810aab80 10 client.16401 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.651+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55edcc7f6de0 con 0x55edcca795d0 2023-10-26T06:55:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe14aff5700 1 -- 172.21.15.89:0/647978335 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55edcc9d0ef0 con 0x55edcca795d0 2023-10-26T06:55:08.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe14aff5700 1 -- 172.21.15.89:0/647978335 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe158028020 con 0x55edcca795d0 2023-10-26T06:55:08.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe1810aab80 10 client.16401 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:08.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe1810aab80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:08.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe1810aab80 20 client.16401 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:08.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe1810aab80 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edcca92d30 0x55edcca95150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe1810aab80 4 client.16401 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:08.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edcca488b0 con 0x55edcca92d30 2023-10-26T06:55:08.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.652+0000 7fe1667fc700 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edcca92d30 0x55edcca95150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.653+0000 7fe1667fc700 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edcca92d30 0x55edcca95150 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90cca60a9421be9b server_cookie=81edfb5c566d0d69 in_seq=0 out_seq=0 2023-10-26T06:55:08.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.653+0000 7fe14aff5700 10 client.16401 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:08.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.653+0000 7fe14aff5700 1 -- 172.21.15.89:0/647978335 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55edcca488b0 con 0x55edcca92d30 2023-10-26T06:55:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.653+0000 7fe14aff5700 10 client.16401 handle_command_reply: tid=0 2023-10-26T06:55:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.653+0000 7fe1810aab80 1 client.16401 shutdown 2023-10-26T06:55:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca795d0 msgr2=0x55edcca85440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca795d0 0x55edcca85440 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe158003960 comp rx=0 tx=0).stop 2023-10-26T06:55:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 shutdown_connections 2023-10-26T06:55:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca795d0 0x55edcca85440 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edcca92d30 0x55edcca95150 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca7f500 0x55edcca82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 --2- 172.21.15.89:0/647978335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca826c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 >> 172.21.15.89:0/647978335 conn(0x55edcc9ed720 msgr2=0x55edcca78080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe14aff5700 0 client.16401 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 shutdown_connections 2023-10-26T06:55:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 1 -- 172.21.15.89:0/647978335 wait complete. 2023-10-26T06:55:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.654+0000 7fe1810aab80 20 client.16401 trim_cache size 0 max 16384 2023-10-26T06:55:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 Processor -- start 2023-10-26T06:55:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 -- start start 2023-10-26T06:55:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca7cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca6e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca89eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcc84c920 con 0x55edcca7f500 2023-10-26T06:55:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcca740c0 con 0x55edcca795d0 2023-10-26T06:55:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.671+0000 7fe1810aab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcca748a0 con 0x55edcca7ebe0 2023-10-26T06:55:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca89eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe166ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca7cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca89eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54036/0 (socket says 172.21.15.89:54036) 2023-10-26T06:55:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 -- 172.21.15.89:0/3897789270 learned_addr learned my addr 172.21.15.89:0/3897789270 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1667fc700 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca6e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 -- 172.21.15.89:0/3897789270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 msgr2=0x55edcca6e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca6e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 -- 172.21.15.89:0/3897789270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 msgr2=0x55edcca7cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca7cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 -- 172.21.15.89:0/3897789270 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edcc7dfc60 con 0x55edcca7f500 2023-10-26T06:55:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe166ffd700 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca7cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe1677fe700 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca89eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1600cb5d0 tx=0x7fe1600099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de15366c0cfeb528 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe14dffb700 1 -- 172.21.15.89:0/3897789270 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16009a8b0 con 0x55edcca7f500 2023-10-26T06:55:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe14dffb700 1 -- 172.21.15.89:0/3897789270 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe16000be20 con 0x55edcca7f500 2023-10-26T06:55:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.672+0000 7fe14dffb700 1 -- 172.21.15.89:0/3897789270 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1600a10b0 con 0x55edcca7f500 2023-10-26T06:55:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 -- 172.21.15.89:0/3897789270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 msgr2=0x55edcca89eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca89eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1600cb5d0 tx=0x7fe1600099e0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 -- 172.21.15.89:0/3897789270 shutdown_connections 2023-10-26T06:55:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7f500 0x55edcca89eb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca7cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 --2- 172.21.15.89:0/3897789270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7ebe0 0x55edcca6e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 -- 172.21.15.89:0/3897789270 >> 172.21.15.89:0/3897789270 conn(0x55edcc9ed720 msgr2=0x55edcca7c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 -- 172.21.15.89:0/3897789270 shutdown_connections 2023-10-26T06:55:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 -- 172.21.15.89:0/3897789270 wait complete. 2023-10-26T06:55:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.673+0000 7fe1810aab80 1 -- start start 2023-10-26T06:55:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 1 Processor -- start 2023-10-26T06:55:08.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca89640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7ebe0 0x55edcca89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7f500 0x55edcca87e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcca885a0 con 0x55edcca7ebe0 2023-10-26T06:55:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcca88710 con 0x55edcca795d0 2023-10-26T06:55:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcca888d0 con 0x55edcca7f500 2023-10-26T06:55:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1810aab80 10 client.0 _ll_register_callbacks cb 0x55edcc8b2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:08.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1667fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7ebe0 0x55edcca89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1667fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7ebe0 0x55edcca89b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54044/0 (socket says 172.21.15.89:54044) 2023-10-26T06:55:08.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe1667fc700 1 -- 172.21.15.89:0/4069556281 learned_addr learned my addr 172.21.15.89:0/4069556281 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:08.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.674+0000 7fe166ffd700 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca89640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1677fe700 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7f500 0x55edcca87e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1667fc700 1 -- 172.21.15.89:0/4069556281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7f500 msgr2=0x55edcca87e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1667fc700 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7f500 0x55edcca87e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1667fc700 1 -- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 msgr2=0x55edcca89640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1667fc700 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca89640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1667fc700 1 -- 172.21.15.89:0/4069556281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edcc7f6de0 con 0x55edcca7ebe0 2023-10-26T06:55:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1667fc700 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7ebe0 0x55edcca89b80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55edcca757d0 tx=0x7fe15c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1522c544e72d101 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe14aff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:55:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe14aff5700 1 -- 172.21.15.89:0/4069556281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c016040 con 0x55edcca7ebe0 2023-10-26T06:55:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe14aff5700 1 -- 172.21.15.89:0/4069556281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fe15c011040 con 0x55edcca7ebe0 2023-10-26T06:55:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe158004390 con 0x55edcca7ebe0 2023-10-26T06:55:08.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe14aff5700 1 -- 172.21.15.89:0/4069556281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe15c011200 con 0x55edcca7ebe0 2023-10-26T06:55:08.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1810aab80 10 client.16407 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:08.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.675+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fe1681a5b10 con 0x55edcca7ebe0 2023-10-26T06:55:08.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe14aff5700 1 -- 172.21.15.89:0/4069556281 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe15c041020 con 0x55edcca7ebe0 2023-10-26T06:55:08.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe14aff5700 1 -- 172.21.15.89:0/4069556281 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe15c010a00 con 0x55edcca7ebe0 2023-10-26T06:55:08.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe1810aab80 10 client.16407 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe1810aab80 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe1810aab80 20 client.16407 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:08.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe1810aab80 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edccae7e10 0x55edccaea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe1810aab80 4 client.16407 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:55:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55edcc9c4e50 con 0x55edccae7e10 2023-10-26T06:55:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.676+0000 7fe166ffd700 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edccae7e10 0x55edccaea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe14aff5700 10 client.16407 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:08.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe166ffd700 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edccae7e10 0x55edccaea230 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51592f07a6d9c8b5 server_cookie=eb0fac25f7a2f83c in_seq=0 out_seq=0 2023-10-26T06:55:08.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe14aff5700 1 -- 172.21.15.89:0/4069556281 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55edcc9c4e50 con 0x55edccae7e10 2023-10-26T06:55:08.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe14aff5700 10 client.16407 handle_command_reply: tid=0 2023-10-26T06:55:08.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 client.16407 shutdown 2023-10-26T06:55:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7ebe0 msgr2=0x55edcca89b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7ebe0 0x55edcca89b80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55edcca757d0 tx=0x7fe15c009260 comp rx=0 tx=0).stop 2023-10-26T06:55:08.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 shutdown_connections 2023-10-26T06:55:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55edccae7e10 0x55edccaea230 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edcca7f500 0x55edcca87e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe14aff5700 0 client.16407 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:08.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edcca795d0 0x55edcca89640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 --2- 172.21.15.89:0/4069556281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55edcca7ebe0 0x55edcca89b80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 >> 172.21.15.89:0/4069556281 conn(0x55edcc9ed720 msgr2=0x55edcca81f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:08.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 shutdown_connections 2023-10-26T06:55:08.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.677+0000 7fe1810aab80 1 -- 172.21.15.89:0/4069556281 wait complete. 2023-10-26T06:55:08.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe1810aab80 20 client.16407 trim_cache size 0 max 16384 2023-10-26T06:55:08.714 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:55:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:55:08.714 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:55:08.715 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:55:08.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1500a1670 msgr2=0x7fe1500a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1500a1670 0x7fe1500a3b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097c0 tx=0x7fe15c009f70 comp rx=0 tx=0).stop 2023-10-26T06:55:08.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe168108940 msgr2=0x7fe16819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe168108940 0x7fe16819d680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b070 tx=0x7fe1600077f0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 shutdown_connections 2023-10-26T06:55:08.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe1500a1670 0x7fe1500a3b30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe168108940 0x7fe16819d680 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe168102930 0x7fe168198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 --2- 172.21.15.89:0/4288898118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1681032f0 0x7fe168199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 >> 172.21.15.89:0/4288898118 conn(0x7fe1680fe3e0 msgr2=0x7fe1680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:08.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 shutdown_connections 2023-10-26T06:55:08.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.678+0000 7fe16d7ff700 1 -- 172.21.15.89:0/4288898118 wait complete. 2023-10-26T06:55:08.720 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:55:08.720 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:55:08.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 -- 172.21.15.89:0/304361486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c1066b0 msgr2=0x7fdd3c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 --2- 172.21.15.89:0/304361486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c1066b0 0x7fdd3c106a90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009990 tx=0x7fdd30009ca0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 -- 172.21.15.89:0/304361486 shutdown_connections 2023-10-26T06:55:08.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 --2- 172.21.15.89:0/304361486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c1066b0 0x7fdd3c106a90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 --2- 172.21.15.89:0/304361486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c101060 0x7fdd3c10d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 --2- 172.21.15.89:0/304361486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd3c1006a0 0x7fdd3c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 -- 172.21.15.89:0/304361486 >> 172.21.15.89:0/304361486 conn(0x7fdd3c0fc1d0 msgr2=0x7fdd3c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:08.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 -- 172.21.15.89:0/304361486 shutdown_connections 2023-10-26T06:55:08.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.918+0000 7fdd431ff700 1 -- 172.21.15.89:0/304361486 wait complete. 2023-10-26T06:55:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 Processor -- start 2023-10-26T06:55:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 -- start start 2023-10-26T06:55:08.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd3c1006a0 0x7fdd3c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c101060 0x7fdd3c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c1066b0 0x7fdd3c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c19cb40 con 0x7fdd3c101060 2023-10-26T06:55:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c19ccb0 con 0x7fdd3c1066b0 2023-10-26T06:55:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.919+0000 7fdd431ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c19ce70 con 0x7fdd3c1006a0 2023-10-26T06:55:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c101060 0x7fdd3c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c101060 0x7fdd3c199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54064/0 (socket says 172.21.15.89:54064) 2023-10-26T06:55:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 -- 172.21.15.89:0/1877500286 learned_addr learned my addr 172.21.15.89:0/1877500286 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd40f9b700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd3c1006a0 0x7fdd3c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd4179c700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c1066b0 0x7fdd3c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 -- 172.21.15.89:0/1877500286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd3c1006a0 msgr2=0x7fdd3c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd3c1006a0 0x7fdd3c198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 -- 172.21.15.89:0/1877500286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c1066b0 msgr2=0x7fdd3c19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c1066b0 0x7fdd3c19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 -- 172.21.15.89:0/1877500286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd300095f0 con 0x7fdd3c101060 2023-10-26T06:55:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.920+0000 7fdd3bfff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c101060 0x7fdd3c199250 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800e9a0 tx=0x7fdd2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.921+0000 7fdd39ffb700 1 -- 172.21.15.89:0/1877500286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28009790 con 0x7fdd3c101060 2023-10-26T06:55:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.921+0000 7fdd39ffb700 1 -- 172.21.15.89:0/1877500286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd28010430 con 0x7fdd3c101060 2023-10-26T06:55:08.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.921+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c19d150 con 0x7fdd3c101060 2023-10-26T06:55:08.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.921+0000 7fdd39ffb700 1 -- 172.21.15.89:0/1877500286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28018680 con 0x7fdd3c101060 2023-10-26T06:55:08.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.921+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c1a5ab0 con 0x7fdd3c101060 2023-10-26T06:55:08.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.922+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3c101de0 con 0x7fdd3c101060 2023-10-26T06:55:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.924+0000 7fdd39ffb700 1 -- 172.21.15.89:0/1877500286 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdd280105a0 con 0x7fdd3c101060 2023-10-26T06:55:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.925+0000 7fdd39ffb700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd2c0a15f0 0x7fdd2c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.925+0000 7fdd39ffb700 1 -- 172.21.15.89:0/1877500286 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fdd28014070 con 0x7fdd3c101060 2023-10-26T06:55:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.925+0000 7fdd40f9b700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd2c0a15f0 0x7fdd2c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.926+0000 7fdd40f9b700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd2c0a15f0 0x7fdd2c0a3ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b2e0 tx=0x7fdd30005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:08.927+0000 7fdd39ffb700 1 -- 172.21.15.89:0/1877500286 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd280967d0 con 0x7fdd3c101060 2023-10-26T06:55:09.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.173+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd3c067530 con 0x7fdd3c101060 2023-10-26T06:55:09.175 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.174+0000 7fdd39ffb700 1 -- 172.21.15.89:0/1877500286 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fdd28026090 con 0x7fdd3c101060 2023-10-26T06:55:09.176 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:55:09.176 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:55:09.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.176+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd2c0a15f0 msgr2=0x7fdd2c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.176+0000 7fdd431ff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd2c0a15f0 0x7fdd2c0a3ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b2e0 tx=0x7fdd30005fd0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.176+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c101060 msgr2=0x7fdd3c199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.176+0000 7fdd431ff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c101060 0x7fdd3c199250 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdd2800e9a0 tx=0x7fdd2800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 shutdown_connections 2023-10-26T06:55:09.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fdd2c0a15f0 0x7fdd2c0a3ab0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd3c1066b0 0x7fdd3c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd3c1006a0 0x7fdd3c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 --2- 172.21.15.89:0/1877500286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd3c101060 0x7fdd3c199250 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 >> 172.21.15.89:0/1877500286 conn(0x7fdd3c0fc1d0 msgr2=0x7fdd3c10be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:09.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 shutdown_connections 2023-10-26T06:55:09.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.177+0000 7fdd431ff700 1 -- 172.21.15.89:0/1877500286 wait complete. 2023-10-26T06:55:09.182 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:55:09.196 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:55:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.453+0000 7f4474f9d700 1 -- 172.21.15.89:0/955557895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 msgr2=0x7f44680a9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.453+0000 7f4474f9d700 1 --2- 172.21.15.89:0/955557895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 0x7f44680a9250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4470063b90 tx=0x7f4470068f00 comp rx=0 tx=0).stop 2023-10-26T06:55:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.454+0000 7f4474f9d700 1 -- 172.21.15.89:0/955557895 shutdown_connections 2023-10-26T06:55:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.454+0000 7f4474f9d700 1 --2- 172.21.15.89:0/955557895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f446809c5b0 0x7f446809c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.454+0000 7f4474f9d700 1 --2- 172.21.15.89:0/955557895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 0x7f44680a9250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.454+0000 7f4474f9d700 1 --2- 172.21.15.89:0/955557895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f446809cf60 0x7f44680a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.454+0000 7f4474f9d700 1 -- 172.21.15.89:0/955557895 >> 172.21.15.89:0/955557895 conn(0x7f446800c870 msgr2=0x7f446809a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.454+0000 7f4474f9d700 1 -- 172.21.15.89:0/955557895 shutdown_connections 2023-10-26T06:55:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.454+0000 7f4474f9d700 1 -- 172.21.15.89:0/955557895 wait complete. 2023-10-26T06:55:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.456+0000 7f4474f9d700 1 Processor -- start 2023-10-26T06:55:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.457+0000 7f4474f9d700 1 -- start start 2023-10-26T06:55:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.458+0000 7f4474f9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f446809c5b0 0x7f4468138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.458+0000 7f4474f9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 0x7f4468138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.458+0000 7f446f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f446809c5b0 0x7f4468138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f446f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f446809c5b0 0x7f4468138660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54080/0 (socket says 172.21.15.89:54080) 2023-10-26T06:55:09.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f446f7fe700 1 -- 172.21.15.89:0/3947602103 learned_addr learned my addr 172.21.15.89:0/3947602103 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:09.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f446effd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 0x7f4468138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f4474f9d700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4468132790 0x7f4468132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4468133210 con 0x7f446809c5b0 2023-10-26T06:55:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4468133350 con 0x7f4468132790 2023-10-26T06:55:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4468133510 con 0x7f44680a16d0 2023-10-26T06:55:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.460+0000 7f446ffff700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4468132790 0x7f4468132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.461+0000 7f446ffff700 1 -- 172.21.15.89:0/3947602103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 msgr2=0x7f4468138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.461+0000 7f446ffff700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 0x7f4468138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.461+0000 7f446ffff700 1 -- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f446809c5b0 msgr2=0x7f4468138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.461+0000 7f446ffff700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f446809c5b0 0x7f4468138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.461+0000 7f446ffff700 1 -- 172.21.15.89:0/3947602103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447004ed50 con 0x7f4468132790 2023-10-26T06:55:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.462+0000 7f446ffff700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4468132790 0x7f4468132c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f447004fe60 tx=0x7f4470073fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.462+0000 7f446cff9700 1 -- 172.21.15.89:0/3947602103 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470074690 con 0x7f4468132790 2023-10-26T06:55:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.462+0000 7f446cff9700 1 -- 172.21.15.89:0/3947602103 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44700747f0 con 0x7f4468132790 2023-10-26T06:55:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.464+0000 7f446cff9700 1 -- 172.21.15.89:0/3947602103 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470074690 con 0x7f4468132790 2023-10-26T06:55:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.464+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446813f470 con 0x7f4468132790 2023-10-26T06:55:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.465+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446813f8a0 con 0x7f4468132790 2023-10-26T06:55:09.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.472+0000 7f446cff9700 1 -- 172.21.15.89:0/3947602103 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f44700680d0 con 0x7f4468132790 2023-10-26T06:55:09.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.475+0000 7f4489c03b80 1 Processor -- start 2023-10-26T06:55:09.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.475+0000 7f4489c03b80 1 -- start start 2023-10-26T06:55:09.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.476+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4ffd60 0x55d49b500140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.476+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.476+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4fa820 0x55d49b4fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.476+0000 7f4489c03b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b46d6e0 con 0x55d49b500680 2023-10-26T06:55:09.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.476+0000 7f4489c03b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b453c60 con 0x55d49b4ffd60 2023-10-26T06:55:09.479 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.476+0000 7f4489c03b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b451580 con 0x55d49b4fa820 2023-10-26T06:55:09.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.477+0000 7f446f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.480 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.477+0000 7f446f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54094/0 (socket says 172.21.15.89:54094) 2023-10-26T06:55:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.477+0000 7f446f7fe700 1 -- 172.21.15.89:0/1345782260 learned_addr learned my addr 172.21.15.89:0/1345782260 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.477+0000 7f446effd700 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4ffd60 0x55d49b500140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.477+0000 7f446cff9700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44600a1510 0x7f44600a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.481 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.477+0000 7f446cff9700 1 -- 172.21.15.89:0/3947602103 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f44701373b0 con 0x7f4468132790 2023-10-26T06:55:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.478+0000 7f446effd700 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4fa820 0x55d49b4fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.482 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.478+0000 7f446effd700 1 -- 172.21.15.89:0/1345782260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4fa820 msgr2=0x55d49b4fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:09.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.478+0000 7f446effd700 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4fa820 0x55d49b4fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.478+0000 7f446effd700 1 -- 172.21.15.89:0/1345782260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 msgr2=0x55d49b4fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:09.483 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.478+0000 7f446effd700 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.478+0000 7f446effd700 1 -- 172.21.15.89:0/1345782260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d49b40a3f0 con 0x55d49b4ffd60 2023-10-26T06:55:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.479+0000 7f446effd700 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4ffd60 0x55d49b500140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4468133bb0 tx=0x7f445c005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516c2fe565c9a2f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.480+0000 7f4459ffb700 1 -- 172.21.15.89:0/1345782260 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c004230 con 0x55d49b4ffd60 2023-10-26T06:55:09.484 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.480+0000 7f4459ffb700 1 -- 172.21.15.89:0/1345782260 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c004cf0 con 0x55d49b4ffd60 2023-10-26T06:55:09.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.480+0000 7f4459ffb700 1 -- 172.21.15.89:0/1345782260 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0054c0 con 0x55d49b4ffd60 2023-10-26T06:55:09.485 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.481+0000 7f446ffff700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44600a1510 0x7f44600a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.486 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.482+0000 7f4489c03b80 1 -- 172.21.15.89:0/1345782260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4ffd60 msgr2=0x55d49b500140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.482+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4ffd60 0x55d49b500140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4468133bb0 tx=0x7f445c005d20 comp rx=0 tx=0).stop 2023-10-26T06:55:09.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.483+0000 7f4489c03b80 1 -- 172.21.15.89:0/1345782260 shutdown_connections 2023-10-26T06:55:09.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.483+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4fa820 0x55d49b4fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.487 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.483+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4ffd60 0x55d49b500140 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.483+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1345782260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.483+0000 7f4489c03b80 1 -- 172.21.15.89:0/1345782260 >> 172.21.15.89:0/1345782260 conn(0x55d49b46e720 msgr2=0x55d49b4f4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:09.488 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.484+0000 7f446ffff700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44600a1510 0x7f44600a39d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4470134ba0 tx=0x7f4470066970 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:09.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.484+0000 7f4489c03b80 1 -- 172.21.15.89:0/1345782260 shutdown_connections 2023-10-26T06:55:09.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.485+0000 7f4489c03b80 1 -- 172.21.15.89:0/1345782260 wait complete. 2023-10-26T06:55:09.489 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.487+0000 7f4489c03b80 1 -- start start 2023-10-26T06:55:09.490 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.489+0000 7f4489c03b80 1 Processor -- start 2023-10-26T06:55:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.492+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b4fa820 0x55d49b503c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.494 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.492+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b504190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.492+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b500680 0x55d49b5046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.492+0000 7f4489c03b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b44ce70 con 0x55d49b4fa820 2023-10-26T06:55:09.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.492+0000 7f4489c03b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b3e9710 con 0x55d49b500680 2023-10-26T06:55:09.495 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.492+0000 7f4489c03b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b3e93b0 con 0x55d49b4ffd60 2023-10-26T06:55:09.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.492+0000 7f4489c03b80 10 client.0 _ll_register_callbacks cb 0x55d49b333a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:09.496 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.494+0000 7f446effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b4fa820 0x55d49b503c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.494+0000 7f446effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b4fa820 0x55d49b503c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54098/0 (socket says 172.21.15.89:54098) 2023-10-26T06:55:09.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.494+0000 7f446effd700 1 -- 172.21.15.89:0/1846839024 learned_addr learned my addr 172.21.15.89:0/1846839024 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:09.497 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.495+0000 7f446effd700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b500680 0x55d49b5046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.495+0000 7f446f7fe700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b504190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.498 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.497+0000 7f446effd700 1 -- 172.21.15.89:0/1846839024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 msgr2=0x55d49b504190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:09.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.497+0000 7f446effd700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b504190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.497+0000 7f446effd700 1 -- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b500680 msgr2=0x55d49b5046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:09.499 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.497+0000 7f446effd700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b500680 0x55d49b5046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.497+0000 7f446effd700 1 -- 172.21.15.89:0/1846839024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c00bd80 con 0x55d49b4fa820 2023-10-26T06:55:09.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.498+0000 7f446f7fe700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b504190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:09.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.498+0000 7f446effd700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b4fa820 0x55d49b503c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f445c004b00 tx=0x7f445c005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d628d468cc9a253 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:09.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.498+0000 7f444effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:55:09.501 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.500+0000 7f444effd700 1 -- 172.21.15.89:0/1846839024 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c004450 con 0x55d49b4fa820 2023-10-26T06:55:09.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.500+0000 7f444effd700 1 -- 172.21.15.89:0/1846839024 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f445c005160 con 0x55d49b4fa820 2023-10-26T06:55:09.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.501+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d49b451ef0 con 0x55d49b4fa820 2023-10-26T06:55:09.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.501+0000 7f444effd700 1 -- 172.21.15.89:0/1846839024 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f445c004450 con 0x55d49b4fa820 2023-10-26T06:55:09.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.502+0000 7f4489c03b80 10 client.16422 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.502+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d49b277de0 con 0x55d49b4fa820 2023-10-26T06:55:09.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f444effd700 1 -- 172.21.15.89:0/1846839024 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d49b451ef0 con 0x55d49b4fa820 2023-10-26T06:55:09.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f444effd700 1 -- 172.21.15.89:0/1846839024 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f445c003c60 con 0x55d49b4fa820 2023-10-26T06:55:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f4489c03b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:09.506 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f4489c03b80 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f4489c03b80 20 client.16422 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b525060 0x55d49b527480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f4489c03b80 4 client.16422 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:09.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.503+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d49b4c98b0 con 0x55d49b525060 2023-10-26T06:55:09.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.505+0000 7f446f7fe700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b525060 0x55d49b527480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.506+0000 7f446f7fe700 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b525060 0x55d49b527480 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d86a26bce1079e server_cookie=9cabb9d029bbd405 in_seq=0 out_seq=0 2023-10-26T06:55:09.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.507+0000 7f444effd700 10 client.16422 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:09.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.508+0000 7f444effd700 1 -- 172.21.15.89:0/1846839024 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d49b4c98b0 con 0x55d49b525060 2023-10-26T06:55:09.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.509+0000 7f444effd700 10 client.16422 handle_command_reply: tid=0 2023-10-26T06:55:09.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.509+0000 7f4489c03b80 1 client.16422 shutdown 2023-10-26T06:55:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.509+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b4fa820 msgr2=0x55d49b503c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.509+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b4fa820 0x55d49b503c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f445c004b00 tx=0x7f445c005950 comp rx=0 tx=0).stop 2023-10-26T06:55:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 shutdown_connections 2023-10-26T06:55:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b4fa820 0x55d49b503c50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b525060 0x55d49b527480 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b500680 0x55d49b5046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1846839024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b504190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 >> 172.21.15.89:0/1846839024 conn(0x55d49b46e720 msgr2=0x55d49b4f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:09.513 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f444effd700 0 client.16422 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.510+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 shutdown_connections 2023-10-26T06:55:09.514 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.513+0000 7f4489c03b80 1 -- 172.21.15.89:0/1846839024 wait complete. 2023-10-26T06:55:09.515 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.513+0000 7f4489c03b80 20 client.16422 trim_cache size 0 max 16384 2023-10-26T06:55:09.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:09 smithi089 ceph-mon[95031]: pgmap v561: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:55:09.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:09 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1877500286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:55:09.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.546+0000 7f4489c03b80 1 Processor -- start 2023-10-26T06:55:09.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.546+0000 7f4489c03b80 1 -- start start 2023-10-26T06:55:09.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.548+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b4fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.548+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b51c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.548+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.548+0000 7f4489c03b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b2cd920 con 0x55d49b500680 2023-10-26T06:55:09.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.548+0000 7f4489c03b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b521ca0 con 0x55d49b4fa820 2023-10-26T06:55:09.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.548+0000 7f4489c03b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b517e20 con 0x55d49b4ffd60 2023-10-26T06:55:09.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.548+0000 7f446effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.549+0000 7f446effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54104/0 (socket says 172.21.15.89:54104) 2023-10-26T06:55:09.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.549+0000 7f446effd700 1 -- 172.21.15.89:0/2610969643 learned_addr learned my addr 172.21.15.89:0/2610969643 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:09.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.549+0000 7f446effd700 1 -- 172.21.15.89:0/2610969643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 msgr2=0x55d49b51c0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:55:09.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.549+0000 7f446effd700 1 --2- 172.21.15.89:0/2610969643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b51c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.549+0000 7f446effd700 1 -- 172.21.15.89:0/2610969643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 msgr2=0x55d49b4fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:09.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.549+0000 7f446effd700 1 --2- 172.21.15.89:0/2610969643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b4fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.549+0000 7f446effd700 1 -- 172.21.15.89:0/2610969643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d49b260c60 con 0x55d49b500680 2023-10-26T06:55:09.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.550+0000 7f446effd700 1 --2- 172.21.15.89:0/2610969643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fed00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f445c00bfa0 tx=0x7f445c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f7f57f52504e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:09.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.556+0000 7f445a7fc700 1 -- 172.21.15.89:0/2610969643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0207a0 con 0x55d49b500680 2023-10-26T06:55:09.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.556+0000 7f445a7fc700 1 -- 172.21.15.89:0/2610969643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c020e20 con 0x55d49b500680 2023-10-26T06:55:09.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.556+0000 7f445a7fc700 1 -- 172.21.15.89:0/2610969643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0038e0 con 0x55d49b500680 2023-10-26T06:55:09.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.557+0000 7f4489c03b80 1 -- 172.21.15.89:0/2610969643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 msgr2=0x55d49b4fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.557+0000 7f4489c03b80 1 --2- 172.21.15.89:0/2610969643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fed00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f445c00bfa0 tx=0x7f445c00bfd0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.558+0000 7f4489c03b80 1 -- 172.21.15.89:0/2610969643 shutdown_connections 2023-10-26T06:55:09.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.558+0000 7f4489c03b80 1 --2- 172.21.15.89:0/2610969643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b4fed00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.558+0000 7f4489c03b80 1 --2- 172.21.15.89:0/2610969643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b4fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.558+0000 7f4489c03b80 1 --2- 172.21.15.89:0/2610969643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b51c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.558+0000 7f4489c03b80 1 -- 172.21.15.89:0/2610969643 >> 172.21.15.89:0/2610969643 conn(0x55d49b46e720 msgr2=0x55d49b4fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:09.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.559+0000 7f4489c03b80 1 -- 172.21.15.89:0/2610969643 shutdown_connections 2023-10-26T06:55:09.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.559+0000 7f4489c03b80 1 -- 172.21.15.89:0/2610969643 wait complete. 2023-10-26T06:55:09.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.560+0000 7f4489c03b80 1 -- start start 2023-10-26T06:55:09.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.561+0000 7f4489c03b80 1 Processor -- start 2023-10-26T06:55:09.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.564+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b51a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b51a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f4489c03b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b571e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f4489c03b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b5723a0 con 0x55d49b500680 2023-10-26T06:55:09.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f4489c03b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b572510 con 0x55d49b4fa820 2023-10-26T06:55:09.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f4489c03b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d49b5726d0 con 0x55d49b4ffd60 2023-10-26T06:55:09.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f446effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b51a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f4489c03b80 10 client.0 _ll_register_callbacks cb 0x55d49b333a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:09.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f446effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b51a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:56030/0 (socket says 172.21.15.89:56030) 2023-10-26T06:55:09.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.565+0000 7f446effd700 1 -- 172.21.15.89:0/1684513366 learned_addr learned my addr 172.21.15.89:0/1684513366 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:09.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.566+0000 7f446effd700 1 -- 172.21.15.89:0/1684513366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 msgr2=0x55d49b51a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:09.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.566+0000 7f446effd700 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b51a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.566+0000 7f446effd700 1 -- 172.21.15.89:0/1684513366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 msgr2=0x55d49b571e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:09.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.566+0000 7f446effd700 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b571e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.566+0000 7f446effd700 1 -- 172.21.15.89:0/1684513366 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d49b277de0 con 0x55d49b4fa820 2023-10-26T06:55:09.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.566+0000 7f446effd700 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b51a290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f445c0059c0 tx=0x7f445c004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e27248e470597851 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:09.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.566+0000 7f444effd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:09.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.567+0000 7f444effd700 1 -- 172.21.15.89:0/1684513366 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0207a0 con 0x55d49b4fa820 2023-10-26T06:55:09.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.567+0000 7f444effd700 1 -- 172.21.15.89:0/1684513366 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f445c000e40 con 0x55d49b4fa820 2023-10-26T06:55:09.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.567+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f445c004cf0 con 0x55d49b4fa820 2023-10-26T06:55:09.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.568+0000 7f444effd700 1 -- 172.21.15.89:0/1684513366 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f445c004250 con 0x55d49b4fa820 2023-10-26T06:55:09.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.568+0000 7f4489c03b80 10 client.26465 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:09.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.568+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55d49b29e250 con 0x55d49b4fa820 2023-10-26T06:55:09.574 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.569+0000 7f444effd700 1 -- 172.21.15.89:0/1684513366 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c004cf0 con 0x55d49b4fa820 2023-10-26T06:55:09.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.570+0000 7f444effd700 1 -- 172.21.15.89:0/1684513366 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f445c004f70 con 0x55d49b4fa820 2023-10-26T06:55:09.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.570+0000 7f4489c03b80 10 client.26465 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:09.575 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.570+0000 7f4489c03b80 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:09.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.570+0000 7f4489c03b80 20 client.26465 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:09.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.570+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b579ab0 0x55d49b57bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:09.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.571+0000 7f4489c03b80 4 client.26465 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:55:09.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.571+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d49b445e50 con 0x55d49b579ab0 2023-10-26T06:55:09.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.572+0000 7f446f7fe700 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b579ab0 0x55d49b57bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:09.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.576+0000 7f446f7fe700 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b579ab0 0x55d49b57bea0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d276d509a8b814 server_cookie=7b58912479e168d1 in_seq=0 out_seq=0 2023-10-26T06:55:09.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.576+0000 7f444effd700 10 client.26465 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:09.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.576+0000 7f444effd700 1 -- 172.21.15.89:0/1684513366 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d49b445e50 con 0x55d49b579ab0 2023-10-26T06:55:09.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.576+0000 7f444effd700 10 client.26465 handle_command_reply: tid=0 2023-10-26T06:55:09.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 client.26465 shutdown 2023-10-26T06:55:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 msgr2=0x55d49b51a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b51a290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f445c0059c0 tx=0x7f445c004be0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 shutdown_connections 2023-10-26T06:55:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d49b500680 0x55d49b571e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55d49b579ab0 0x55d49b57bea0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d49b4fa820 0x55d49b51a290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 --2- 172.21.15.89:0/1684513366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d49b4ffd60 0x55d49b51a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f444effd700 0 client.26465 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:09.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.577+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 >> 172.21.15.89:0/1684513366 conn(0x55d49b46e720 msgr2=0x55d49b4fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:09.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.578+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 shutdown_connections 2023-10-26T06:55:09.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.578+0000 7f4489c03b80 1 -- 172.21.15.89:0/1684513366 wait complete. 2023-10-26T06:55:09.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.578+0000 7f4489c03b80 20 client.26465 trim_cache size 0 max 16384 2023-10-26T06:55:09.583 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:55:09.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.579+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44600a1510 msgr2=0x7f44600a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.579+0000 7f4474f9d700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44600a1510 0x7f44600a39d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4470134ba0 tx=0x7f4470066970 comp rx=0 tx=0).stop 2023-10-26T06:55:09.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.579+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4468132790 msgr2=0x7f4468132c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:09.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.579+0000 7f4474f9d700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4468132790 0x7f4468132c40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f447004fe60 tx=0x7f4470073fd0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 shutdown_connections 2023-10-26T06:55:09.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f44600a1510 0x7f44600a39d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f446809c5b0 0x7f4468138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4468132790 0x7f4468132c40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 --2- 172.21.15.89:0/3947602103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44680a16d0 0x7f4468138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:09.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 >> 172.21.15.89:0/3947602103 conn(0x7f446800c870 msgr2=0x7f44680095b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:09.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 shutdown_connections 2023-10-26T06:55:09.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:09.580+0000 7f4474f9d700 1 -- 172.21.15.89:0/3947602103 wait complete. 2023-10-26T06:55:09.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:09 smithi192 ceph-mon[131661]: pgmap v561: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:55:09.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:09 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1877500286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:55:09.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:09 smithi135 ceph-mon[131403]: pgmap v561: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 10 KiB/s rd, 45 KiB/s wr, 17 op/s 2023-10-26T06:55:09.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:09 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1877500286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:55:10.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:55:10.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:55:10.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:10 smithi192 ceph-mon[131661]: osdmap e91: 12 total, 12 up, 12 in 2023-10-26T06:55:10.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:10 smithi135 ceph-mon[131403]: osdmap e91: 12 total, 12 up, 12 in 2023-10-26T06:55:10.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:10 smithi089 ceph-mon[95031]: osdmap e91: 12 total, 12 up, 12 in 2023-10-26T06:55:10.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.981+0000 7f649bbff700 1 -- 172.21.15.89:0/2792720147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494105b30 msgr2=0x7f649410d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:10.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.981+0000 7f649bbff700 1 --2- 172.21.15.89:0/2792720147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494105b30 0x7f649410d540 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f648c0099c0 tx=0x7f648c009cd0 comp rx=0 tx=0).stop 2023-10-26T06:55:10.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.983+0000 7f649bbff700 1 -- 172.21.15.89:0/2792720147 shutdown_connections 2023-10-26T06:55:10.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.983+0000 7f649bbff700 1 --2- 172.21.15.89:0/2792720147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494105b30 0x7f649410d540 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:10.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.983+0000 7f649bbff700 1 --2- 172.21.15.89:0/2792720147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6494067d40 0x7f6494068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:10.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.983+0000 7f649bbff700 1 --2- 172.21.15.89:0/2792720147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6494105170 0x7f64941055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.983+0000 7f649bbff700 1 -- 172.21.15.89:0/2792720147 >> 172.21.15.89:0/2792720147 conn(0x7f64940767f0 msgr2=0x7f6494076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.983+0000 7f649bbff700 1 -- 172.21.15.89:0/2792720147 shutdown_connections 2023-10-26T06:55:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.984+0000 7f649bbff700 1 -- 172.21.15.89:0/2792720147 wait complete. 2023-10-26T06:55:10.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.985+0000 7f649bbff700 1 Processor -- start 2023-10-26T06:55:10.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.986+0000 7f649bbff700 1 -- start start 2023-10-26T06:55:10.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.986+0000 7f649bbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494067d40 0x7f64940ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:10.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.987+0000 7f649bbff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6494105170 0x7f64940ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:10.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.987+0000 7f649bbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6494105b30 0x7f6494100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:10.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.987+0000 7f649bbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6494101630 con 0x7f6494067d40 2023-10-26T06:55:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.987+0000 7f649bbff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a9620 con 0x7f6494105170 2023-10-26T06:55:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.987+0000 7f649bbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941a9790 con 0x7f6494105b30 2023-10-26T06:55:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.987+0000 7f649919a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6494105170 0x7f64940ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:10.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.988+0000 7f649919a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6494105170 0x7f64940ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:44428/0 (socket says 172.21.15.89:44428) 2023-10-26T06:55:10.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.988+0000 7f649919a700 1 -- 172.21.15.89:0/3118305751 learned_addr learned my addr 172.21.15.89:0/3118305751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:10.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.988+0000 7f649999b700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494067d40 0x7f64940ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:10.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.989+0000 7f649999b700 1 -- 172.21.15.89:0/3118305751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6494105b30 msgr2=0x7f6494100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:10.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.989+0000 7f649999b700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6494105b30 0x7f6494100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:10.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.989+0000 7f649999b700 1 -- 172.21.15.89:0/3118305751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6494105170 msgr2=0x7f64940ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:10.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.989+0000 7f649999b700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6494105170 0x7f64940ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.989+0000 7f649999b700 1 -- 172.21.15.89:0/3118305751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c0095f0 con 0x7f6494067d40 2023-10-26T06:55:10.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.990+0000 7f649999b700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494067d40 0x7f64940ff6b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f648400b450 tx=0x7f648400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:10.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.991+0000 7f648affd700 1 -- 172.21.15.89:0/3118305751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484011830 con 0x7f6494067d40 2023-10-26T06:55:10.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.991+0000 7f648affd700 1 -- 172.21.15.89:0/3118305751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6484011e50 con 0x7f6494067d40 2023-10-26T06:55:10.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.991+0000 7f648affd700 1 -- 172.21.15.89:0/3118305751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484010d60 con 0x7f6494067d40 2023-10-26T06:55:10.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.991+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64941a9a70 con 0x7f6494067d40 2023-10-26T06:55:10.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:10.991+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64941a9f30 con 0x7f6494067d40 2023-10-26T06:55:11.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.001+0000 7f648affd700 1 -- 172.21.15.89:0/3118305751 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6484010380 con 0x7f6494067d40 2023-10-26T06:55:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.002+0000 7f64af5a1b80 1 Processor -- start 2023-10-26T06:55:11.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.003+0000 7f64af5a1b80 1 -- start start 2023-10-26T06:55:11.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.005+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d753c60 0x560f9d754040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.005+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d754580 0x560f9d74e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.005+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d74ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.005+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d6c1760 con 0x560f9d74e650 2023-10-26T06:55:11.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.005+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d6a7ce0 con 0x560f9d753c60 2023-10-26T06:55:11.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.005+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d6a5600 con 0x560f9d754580 2023-10-26T06:55:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.006+0000 7f649a19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d753c60 0x560f9d754040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.006+0000 7f649a19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d753c60 0x560f9d754040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:44434/0 (socket says 172.21.15.89:44434) 2023-10-26T06:55:11.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.006+0000 7f649a19c700 1 -- 172.21.15.89:0/234304616 learned_addr learned my addr 172.21.15.89:0/234304616 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.006+0000 7f648affd700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64800a1730 0x7f64800a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.006+0000 7f648affd700 1 -- 172.21.15.89:0/3118305751 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f64840cca30 con 0x7f6494067d40 2023-10-26T06:55:11.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.007+0000 7f649a19c700 1 -- 172.21.15.89:0/234304616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d754580 msgr2=0x560f9d74e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.007+0000 7f649a19c700 1 --2- 172.21.15.89:0/234304616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d754580 0x560f9d74e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.007+0000 7f649a19c700 1 -- 172.21.15.89:0/234304616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 msgr2=0x560f9d74ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.007+0000 7f649a19c700 1 --2- 172.21.15.89:0/234304616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d74ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.007+0000 7f649a19c700 1 -- 172.21.15.89:0/234304616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9d65e470 con 0x560f9d753c60 2023-10-26T06:55:11.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.007+0000 7f649a19c700 1 --2- 172.21.15.89:0/234304616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d753c60 0x560f9d754040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f648c003740 tx=0x7f648c004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fb1bbc57c96fa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.010+0000 7f649a19c700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64800a1730 0x7f64800a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.012+0000 7f647ffff700 1 -- 172.21.15.89:0/234304616 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c01d070 con 0x560f9d753c60 2023-10-26T06:55:11.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.012+0000 7f647ffff700 1 -- 172.21.15.89:0/234304616 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648c0053b0 con 0x560f9d753c60 2023-10-26T06:55:11.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.012+0000 7f647ffff700 1 -- 172.21.15.89:0/234304616 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00f750 con 0x560f9d753c60 2023-10-26T06:55:11.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.012+0000 7f649a19c700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64800a1730 0x7f64800a3bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f648c023000 tx=0x7f648c01e000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.013+0000 7f64af5a1b80 1 -- 172.21.15.89:0/234304616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d753c60 msgr2=0x560f9d754040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.013+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/234304616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d753c60 0x560f9d754040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f648c003740 tx=0x7f648c004500 comp rx=0 tx=0).stop 2023-10-26T06:55:11.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.013+0000 7f64af5a1b80 1 -- 172.21.15.89:0/234304616 shutdown_connections 2023-10-26T06:55:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.013+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/234304616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d74ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.014+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/234304616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d753c60 0x560f9d754040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.014+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/234304616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d754580 0x560f9d74e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.014+0000 7f64af5a1b80 1 -- 172.21.15.89:0/234304616 >> 172.21.15.89:0/234304616 conn(0x560f9d6c27a0 msgr2=0x560f9d748be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.014+0000 7f64af5a1b80 1 -- 172.21.15.89:0/234304616 shutdown_connections 2023-10-26T06:55:11.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.014+0000 7f64af5a1b80 1 -- 172.21.15.89:0/234304616 wait complete. 2023-10-26T06:55:11.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.016+0000 7f64af5a1b80 1 -- start start 2023-10-26T06:55:11.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.018+0000 7f64af5a1b80 1 Processor -- start 2023-10-26T06:55:11.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.019+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d75a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.020+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.020+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d757fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.020+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d6a0ef0 con 0x560f9d74e650 2023-10-26T06:55:11.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.020+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d63d790 con 0x560f9d754580 2023-10-26T06:55:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.020+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d63d430 con 0x560f9d753c60 2023-10-26T06:55:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.020+0000 7f64af5a1b80 10 client.0 _ll_register_callbacks cb 0x560f9d587b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:11.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:40008/0 (socket says 172.21.15.89:40008) 2023-10-26T06:55:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 -- 172.21.15.89:0/2719909133 learned_addr learned my addr 172.21.15.89:0/2719909133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649a19c700 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d757fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 -- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 msgr2=0x560f9d757fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d757fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 -- 172.21.15.89:0/2719909133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 msgr2=0x560f9d75a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d75a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649919a700 1 -- 172.21.15.89:0/2719909133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649410de70 con 0x560f9d753c60 2023-10-26T06:55:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.021+0000 7f649a19c700 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d757fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:55:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.022+0000 7f649919a700 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7578a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f649000e8e0 tx=0x7f649000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc917412308b4a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.022+0000 7f647d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:55:11.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.026+0000 7f647d7fa700 1 -- 172.21.15.89:0/2719909133 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490009b80 con 0x560f9d753c60 2023-10-26T06:55:11.028 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.026+0000 7f647d7fa700 1 -- 172.21.15.89:0/2719909133 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f6490006e40 con 0x560f9d753c60 2023-10-26T06:55:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.026+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f9d6a5f70 con 0x560f9d753c60 2023-10-26T06:55:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.027+0000 7f647d7fa700 1 -- 172.21.15.89:0/2719909133 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f649000eee0 con 0x560f9d753c60 2023-10-26T06:55:11.029 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.027+0000 7f64af5a1b80 10 client.26467 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:11.030 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.027+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560f9d4cbe60 con 0x560f9d753c60 2023-10-26T06:55:11.032 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.031+0000 7f647d7fa700 1 -- 172.21.15.89:0/2719909133 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f9d6a5f70 con 0x560f9d753c60 2023-10-26T06:55:11.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.031+0000 7f647d7fa700 1 -- 172.21.15.89:0/2719909133 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f649001d3f0 con 0x560f9d753c60 2023-10-26T06:55:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.033+0000 7f64af5a1b80 10 client.26467 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:11.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.033+0000 7f64af5a1b80 10 client.26467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.033+0000 7f64af5a1b80 20 client.26467 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.033+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d77b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.033+0000 7f64af5a1b80 4 client.26467 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:11.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.033+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f9d71d930 con 0x560f9d778d30 2023-10-26T06:55:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.033+0000 7f649999b700 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d77b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.034+0000 7f649999b700 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d77b150 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77e92662f833c7b server_cookie=fadaa1be63fd1d4 in_seq=0 out_seq=0 2023-10-26T06:55:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.034+0000 7f647d7fa700 10 client.26467 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.035+0000 7f647d7fa700 1 -- 172.21.15.89:0/2719909133 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f9d71d930 con 0x560f9d778d30 2023-10-26T06:55:11.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.035+0000 7f647d7fa700 10 client.26467 handle_command_reply: tid=0 2023-10-26T06:55:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.035+0000 7f64af5a1b80 1 client.26467 shutdown 2023-10-26T06:55:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 msgr2=0x560f9d7578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7578a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f649000e8e0 tx=0x7f649000eca0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 shutdown_connections 2023-10-26T06:55:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d75a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d77b150 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d757fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f647d7fa700 0 client.26467 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/2719909133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7578a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.036+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 >> 172.21.15.89:0/2719909133 conn(0x560f9d6c27a0 msgr2=0x560f9d74d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.037+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 shutdown_connections 2023-10-26T06:55:11.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.037+0000 7f64af5a1b80 1 -- 172.21.15.89:0/2719909133 wait complete. 2023-10-26T06:55:11.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.037+0000 7f64af5a1b80 20 client.26467 trim_cache size 0 max 16384 2023-10-26T06:55:11.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.069+0000 7f64af5a1b80 1 Processor -- start 2023-10-26T06:55:11.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.069+0000 7f64af5a1b80 1 -- start start 2023-10-26T06:55:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.070+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d751f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d5219a0 con 0x560f9d74e650 2023-10-26T06:55:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d7499d0 con 0x560f9d754580 2023-10-26T06:55:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d7452b0 con 0x560f9d753c60 2023-10-26T06:55:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f649919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f649a19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f649999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d751f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f649a19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:44464/0 (socket says 172.21.15.89:44464) 2023-10-26T06:55:11.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.071+0000 7f649a19c700 1 -- 172.21.15.89:0/3681677358 learned_addr learned my addr 172.21.15.89:0/3681677358 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.076 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.072+0000 7f649a19c700 1 -- 172.21.15.89:0/3681677358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 msgr2=0x560f9d7577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.072+0000 7f649a19c700 1 --2- 172.21.15.89:0/3681677358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.072+0000 7f649a19c700 1 -- 172.21.15.89:0/3681677358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 msgr2=0x560f9d751f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.077 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.072+0000 7f649a19c700 1 --2- 172.21.15.89:0/3681677358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d751f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.072+0000 7f649a19c700 1 -- 172.21.15.89:0/3681677358 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9d4b4ce0 con 0x560f9d754580 2023-10-26T06:55:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.073+0000 7f649a19c700 1 --2- 172.21.15.89:0/3681677358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76fee0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f648c009f50 tx=0x7f648c02e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d444f418a826f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.073+0000 7f6488bff700 1 -- 172.21.15.89:0/3681677358 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c01d070 con 0x560f9d754580 2023-10-26T06:55:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.074+0000 7f6488bff700 1 -- 172.21.15.89:0/3681677358 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648c00be80 con 0x560f9d754580 2023-10-26T06:55:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.074+0000 7f6488bff700 1 -- 172.21.15.89:0/3681677358 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c004b00 con 0x560f9d754580 2023-10-26T06:55:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 -- 172.21.15.89:0/3681677358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 msgr2=0x560f9d76fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/3681677358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76fee0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f648c009f50 tx=0x7f648c02e000 comp rx=0 tx=0).stop 2023-10-26T06:55:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 -- 172.21.15.89:0/3681677358 shutdown_connections 2023-10-26T06:55:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/3681677358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d74e650 0x560f9d751f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.080 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/3681677358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76fee0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/3681677358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d753c60 0x560f9d7577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.081 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 -- 172.21.15.89:0/3681677358 >> 172.21.15.89:0/3681677358 conn(0x560f9d6c27a0 msgr2=0x560f9d7529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.075+0000 7f64af5a1b80 1 -- 172.21.15.89:0/3681677358 shutdown_connections 2023-10-26T06:55:11.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.076+0000 7f64af5a1b80 1 -- 172.21.15.89:0/3681677358 wait complete. 2023-10-26T06:55:11.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.076+0000 7f64af5a1b80 1 -- start start 2023-10-26T06:55:11.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.076+0000 7f64af5a1b80 1 Processor -- start 2023-10-26T06:55:11.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.079+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d74e650 0x560f9d76fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.079+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d753c60 0x560f9d76e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.079+0000 7f64af5a1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.081+0000 7f649999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d74e650 0x560f9d76fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.081+0000 7f64af5a1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d76ed10 con 0x560f9d753c60 2023-10-26T06:55:11.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.081+0000 7f649a19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.081+0000 7f649a19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:44472/0 (socket says 172.21.15.89:44472) 2023-10-26T06:55:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.081+0000 7f649a19c700 1 -- 172.21.15.89:0/44047978 learned_addr learned my addr 172.21.15.89:0/44047978 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.082+0000 7f649919a700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d753c60 0x560f9d76e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.086 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.082+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d76ee80 con 0x560f9d754580 2023-10-26T06:55:11.087 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.082+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9d76f040 con 0x560f9d74e650 2023-10-26T06:55:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.083+0000 7f649a19c700 1 -- 172.21.15.89:0/44047978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d74e650 msgr2=0x560f9d76fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.089 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.083+0000 7f64af5a1b80 10 client.0 _ll_register_callbacks cb 0x560f9d587b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.083+0000 7f649a19c700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d74e650 0x560f9d76fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.084+0000 7f649a19c700 1 -- 172.21.15.89:0/44047978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d753c60 msgr2=0x560f9d76e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.090 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.084+0000 7f649a19c700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d753c60 0x560f9d76e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.084+0000 7f649a19c700 1 -- 172.21.15.89:0/44047978 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648c027050 con 0x560f9d754580 2023-10-26T06:55:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.084+0000 7f649999b700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d74e650 0x560f9d76fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.084+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f648c0053b0 con 0x560f9d754580 2023-10-26T06:55:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.085+0000 7f649919a700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d753c60 0x560f9d76e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.085+0000 7f649a19c700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76e7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c010 tx=0x7f648c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c748c1a6562348e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.085+0000 7f647d7fa700 10 client.26480 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:11.092 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.086+0000 7f647d7fa700 1 -- 172.21.15.89:0/44047978 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c01d040 con 0x560f9d754580 2023-10-26T06:55:11.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.086+0000 7f647d7fa700 1 -- 172.21.15.89:0/44047978 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f648c0047a0 con 0x560f9d754580 2023-10-26T06:55:11.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.086+0000 7f647d7fa700 1 -- 172.21.15.89:0/44047978 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f648c004960 con 0x560f9d754580 2023-10-26T06:55:11.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.086+0000 7f64af5a1b80 10 client.26480 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:11.093 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.086+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=20}) v3 -- 0x560f9d4f22d0 con 0x560f9d754580 2023-10-26T06:55:11.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f647d7fa700 1 -- 172.21.15.89:0/44047978 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f648c004ac0 con 0x560f9d754580 2023-10-26T06:55:11.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f64af5a1b80 10 client.26480 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:11.094 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f64af5a1b80 10 client.26480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:11.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f64af5a1b80 20 client.26480 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:11.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d7cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f64af5a1b80 4 client.26480 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:55:11.095 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560f9d699ed0 con 0x560f9d778d30 2023-10-26T06:55:11.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.087+0000 7f649999b700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d7cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.088+0000 7f649999b700 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d7cfa20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d1fdc9cb93d1a8 server_cookie=4eb89730b6badcdd in_seq=0 out_seq=0 2023-10-26T06:55:11.096 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.088+0000 7f647d7fa700 10 client.26480 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.097 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.089+0000 7f647d7fa700 1 -- 172.21.15.89:0/44047978 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560f9d699ed0 con 0x560f9d778d30 2023-10-26T06:55:11.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.090+0000 7f647d7fa700 10 client.26480 handle_command_reply: tid=0 2023-10-26T06:55:11.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.090+0000 7f64af5a1b80 1 client.26480 shutdown 2023-10-26T06:55:11.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.090+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 msgr2=0x560f9d76e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.098 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.090+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76e7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c010 tx=0x7f648c009510 comp rx=0 tx=0).stop 2023-10-26T06:55:11.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.090+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 shutdown_connections 2023-10-26T06:55:11.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.090+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f9d778d30 0x560f9d7cfa20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.099 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.091+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f9d754580 0x560f9d76e7d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.091+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f9d74e650 0x560f9d76fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.091+0000 7f64af5a1b80 1 --2- 172.21.15.89:0/44047978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f9d753c60 0x560f9d76e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.091+0000 7f647d7fa700 0 client.26480 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.100 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.091+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 >> 172.21.15.89:0/44047978 conn(0x560f9d6c27a0 msgr2=0x560f9d751dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.091+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 shutdown_connections 2023-10-26T06:55:11.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.091+0000 7f64af5a1b80 1 -- 172.21.15.89:0/44047978 wait complete. 2023-10-26T06:55:11.101 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.092+0000 7f64af5a1b80 20 client.26480 trim_cache size 0 max 16384 2023-10-26T06:55:11.102 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:55:11.102 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:55:11.103 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "9a7701cd-980c-4ff8-b8c1-0014546e3e45", 2023-10-26T06:55:11.103 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:55:11.103 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:55:11.104 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64800a1730 msgr2=0x7f64800a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64800a1730 0x7f64800a3bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f648c023000 tx=0x7f648c01e000 comp rx=0 tx=0).stop 2023-10-26T06:55:11.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494067d40 msgr2=0x7f64940ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.105 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494067d40 0x7f64940ff6b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f648400b450 tx=0x7f648400b760 comp rx=0 tx=0).stop 2023-10-26T06:55:11.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 shutdown_connections 2023-10-26T06:55:11.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f64800a1730 0x7f64800a3bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.106 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6494067d40 0x7f64940ff6b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6494105b30 0x7f6494100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 --2- 172.21.15.89:0/3118305751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6494105170 0x7f64940ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.107 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 >> 172.21.15.89:0/3118305751 conn(0x7f64940767f0 msgr2=0x7f64940fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 shutdown_connections 2023-10-26T06:55:11.108 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.093+0000 7f649bbff700 1 -- 172.21.15.89:0/3118305751 wait complete. 2023-10-26T06:55:11.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:55:11.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.499+0000 7fa90edff700 1 -- 172.21.15.89:0/1968288386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa908100000 msgr2=0x7fa90810c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.500 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.499+0000 7fa90edff700 1 --2- 172.21.15.89:0/1968288386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa908100000 0x7fa90810c4a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa90000b0a0 tx=0x7fa90000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.499+0000 7fa90edff700 1 -- 172.21.15.89:0/1968288386 shutdown_connections 2023-10-26T06:55:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.499+0000 7fa90edff700 1 --2- 172.21.15.89:0/1968288386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9081056e0 0x7fa908105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.502 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.499+0000 7fa90edff700 1 --2- 172.21.15.89:0/1968288386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa908100000 0x7fa90810c4a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.499+0000 7fa90edff700 1 --2- 172.21.15.89:0/1968288386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9080ff640 0x7fa9080ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.499+0000 7fa90edff700 1 -- 172.21.15.89:0/1968288386 >> 172.21.15.89:0/1968288386 conn(0x7fa9080767f0 msgr2=0x7fa908076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.500+0000 7fa90edff700 1 -- 172.21.15.89:0/1968288386 shutdown_connections 2023-10-26T06:55:11.503 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.500+0000 7fa90edff700 1 -- 172.21.15.89:0/1968288386 wait complete. 2023-10-26T06:55:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.502+0000 7fa90edff700 1 Processor -- start 2023-10-26T06:55:11.504 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.503+0000 7fa90edff700 1 -- start start 2023-10-26T06:55:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.503+0000 7fa90edff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9080ff640 0x7fa90810e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.503+0000 7fa90edff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa908100000 0x7fa90810e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.505 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.504+0000 7fa90edff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9081056e0 0x7fa90810eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.504+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9080ff640 0x7fa90810e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.504+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9080ff640 0x7fa90810e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:40062/0 (socket says 172.21.15.89:40062) 2023-10-26T06:55:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90d39c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9081056e0 0x7fa90810eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.507 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90d39c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9081056e0 0x7fa90810eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:44500/0 (socket says 172.21.15.89:44500) 2023-10-26T06:55:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90edff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908110040 con 0x7fa908100000 2023-10-26T06:55:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90edff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081a94c0 con 0x7fa9081056e0 2023-10-26T06:55:11.508 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90edff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081a9680 con 0x7fa9080ff640 2023-10-26T06:55:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90cb9b700 1 -- 172.21.15.89:0/1165310521 learned_addr learned my addr 172.21.15.89:0/1165310521 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.509 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90d39c700 1 -- 172.21.15.89:0/1165310521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9080ff640 msgr2=0x7fa90810e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90d39c700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9080ff640 0x7fa90810e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90d39c700 1 -- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa908100000 msgr2=0x7fa90810e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.510 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90d39c700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa908100000 0x7fa90810e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.505+0000 7fa90d39c700 1 -- 172.21.15.89:0/1165310521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900009d00 con 0x7fa9081056e0 2023-10-26T06:55:11.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.506+0000 7fa90d39c700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9081056e0 0x7fa90810eaf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa900006070 tx=0x7fa9000077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.506+0000 7fa905ffb700 1 -- 172.21.15.89:0/1165310521 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000e040 con 0x7fa9081056e0 2023-10-26T06:55:11.511 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.506+0000 7fa905ffb700 1 -- 172.21.15.89:0/1165310521 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa900004e60 con 0x7fa9081056e0 2023-10-26T06:55:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.506+0000 7fa905ffb700 1 -- 172.21.15.89:0/1165310521 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90001c640 con 0x7fa9081056e0 2023-10-26T06:55:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.506+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9081a9900 con 0x7fa9081056e0 2023-10-26T06:55:11.512 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.507+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9081a9d60 con 0x7fa9081056e0 2023-10-26T06:55:11.516 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.514+0000 7fa905ffb700 1 -- 172.21.15.89:0/1165310521 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa90001c7a0 con 0x7fa9081056e0 2023-10-26T06:55:11.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.517+0000 7fa905ffb700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa8f80a1730 0x7fa8f80a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.517+0000 7fa905ffb700 1 -- 172.21.15.89:0/1165310521 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7fa9000cdd00 con 0x7fa9081056e0 2023-10-26T06:55:11.521 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.519+0000 7fa922708b80 1 Processor -- start 2023-10-26T06:55:11.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.519+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa8f80a1730 0x7fa8f80a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.522 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.520+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa8f80a1730 0x7fa8f80a3bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00aca0 tx=0x7fa8fc005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.521+0000 7fa922708b80 1 -- start start 2023-10-26T06:55:11.523 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.521+0000 7fa922708b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f39fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.521+0000 7fa922708b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.521+0000 7fa922708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f34a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.521+0000 7fa922708b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58ea76e0 con 0x55cf58f3a500 2023-10-26T06:55:11.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.521+0000 7fa922708b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58e8dc60 con 0x55cf58f345d0 2023-10-26T06:55:11.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.521+0000 7fa922708b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58e8b580 con 0x55cf58f39be0 2023-10-26T06:55:11.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.522+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f39fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.522+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f39fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:40076/0 (socket says 172.21.15.89:40076) 2023-10-26T06:55:11.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.522+0000 7fa90cb9b700 1 -- 172.21.15.89:0/1097712254 learned_addr learned my addr 172.21.15.89:0/1097712254 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.522+0000 7fa90d39c700 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f34a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.523+0000 7fa90cb9b700 1 -- 172.21.15.89:0/1097712254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 msgr2=0x55cf58f34a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.523+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f34a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.527 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.523+0000 7fa90cb9b700 1 -- 172.21.15.89:0/1097712254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 msgr2=0x55cf58f34090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.523+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.528 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.523+0000 7fa90cb9b700 1 -- 172.21.15.89:0/1097712254 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf58e443f0 con 0x55cf58f39be0 2023-10-26T06:55:11.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.523+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f39fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00feb0 tx=0x7fa8fc012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f04aaeadaf36df server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.525+0000 7fa8f2ffd700 1 -- 172.21.15.89:0/1097712254 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00fee0 con 0x55cf58f39be0 2023-10-26T06:55:11.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.525+0000 7fa8f2ffd700 1 -- 172.21.15.89:0/1097712254 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8fc020070 con 0x55cf58f39be0 2023-10-26T06:55:11.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.526+0000 7fa922708b80 1 -- 172.21.15.89:0/1097712254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 msgr2=0x55cf58f39fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.526+0000 7fa8f2ffd700 1 -- 172.21.15.89:0/1097712254 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00fee0 con 0x55cf58f39be0 2023-10-26T06:55:11.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.527+0000 7fa922708b80 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f39fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00feb0 tx=0x7fa8fc012000 comp rx=0 tx=0).stop 2023-10-26T06:55:11.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.527+0000 7fa922708b80 1 -- 172.21.15.89:0/1097712254 shutdown_connections 2023-10-26T06:55:11.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.527+0000 7fa922708b80 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f34a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.527+0000 7fa922708b80 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f39fc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.527+0000 7fa922708b80 1 --2- 172.21.15.89:0/1097712254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f34090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.527+0000 7fa922708b80 1 -- 172.21.15.89:0/1097712254 >> 172.21.15.89:0/1097712254 conn(0x55cf58ea8720 msgr2=0x55cf58f2eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.527+0000 7fa922708b80 1 -- 172.21.15.89:0/1097712254 shutdown_connections 2023-10-26T06:55:11.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.528+0000 7fa922708b80 1 -- 172.21.15.89:0/1097712254 wait complete. 2023-10-26T06:55:11.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.529+0000 7fa922708b80 1 -- start start 2023-10-26T06:55:11.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.532+0000 7fa922708b80 1 Processor -- start 2023-10-26T06:55:11.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.534+0000 7fa922708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.537+0000 7fa922708b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f3d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.537+0000 7fa922708b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f3dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.537+0000 7fa922708b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58e86e70 con 0x55cf58f3a500 2023-10-26T06:55:11.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.537+0000 7fa922708b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58e23710 con 0x55cf58f345d0 2023-10-26T06:55:11.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.537+0000 7fa922708b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58e233b0 con 0x55cf58f39be0 2023-10-26T06:55:11.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f40360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:44524/0 (socket says 172.21.15.89:44524) 2023-10-26T06:55:11.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa90cb9b700 1 -- 172.21.15.89:0/2086346789 learned_addr learned my addr 172.21.15.89:0/2086346789 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa922708b80 10 client.0 _ll_register_callbacks cb 0x55cf58d6da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:11.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa907fff700 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.542 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa90cb9b700 1 -- 172.21.15.89:0/2086346789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 msgr2=0x55cf58f3d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f3d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.538+0000 7fa90cb9b700 1 -- 172.21.15.89:0/2086346789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 msgr2=0x55cf58f3dbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-26T06:55:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.539+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f3dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.543 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.539+0000 7fa90cb9b700 1 -- 172.21.15.89:0/2086346789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc00fc40 con 0x55cf58f345d0 2023-10-26T06:55:11.544 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:11 smithi089 ceph-mon[95031]: pgmap v563: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:55:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.539+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f40360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc005f30 tx=0x7fa8fc00a9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d5ab948492febd server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.539+0000 7fa8e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:11.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.540+0000 7fa8e7fff700 1 -- 172.21.15.89:0/2086346789 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc005040 con 0x55cf58f345d0 2023-10-26T06:55:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.540+0000 7fa8e7fff700 1 -- 172.21.15.89:0/2086346789 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7fa8fc0051a0 con 0x55cf58f345d0 2023-10-26T06:55:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.540+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf58e8bef0 con 0x55cf58f345d0 2023-10-26T06:55:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.541+0000 7fa8e7fff700 1 -- 172.21.15.89:0/2086346789 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8fc005360 con 0x55cf58f345d0 2023-10-26T06:55:11.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.541+0000 7fa922708b80 10 client.26492 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.542+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cf58cb1de0 con 0x55cf58f345d0 2023-10-26T06:55:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.543+0000 7fa8e7fff700 1 -- 172.21.15.89:0/2086346789 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf58e8bef0 con 0x55cf58f345d0 2023-10-26T06:55:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.544+0000 7fa8e7fff700 1 -- 172.21.15.89:0/2086346789 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa8fc036070 con 0x55cf58f345d0 2023-10-26T06:55:11.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.544+0000 7fa922708b80 10 client.26492 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.544+0000 7fa922708b80 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.544+0000 7fa922708b80 20 client.26492 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:11.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.544+0000 7fa922708b80 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58f4ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.545+0000 7fa922708b80 4 client.26492 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:11.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.546+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf58f038b0 con 0x55cf58f4dae0 2023-10-26T06:55:11.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.552+0000 7fa907fff700 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58f4ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.555+0000 7fa8e7fff700 10 client.26492 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.555+0000 7fa907fff700 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58f4ff00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9d28aafd3f9c28 server_cookie=a2ff6a17d154f2 in_seq=0 out_seq=0 2023-10-26T06:55:11.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.561+0000 7fa8e7fff700 1 -- 172.21.15.89:0/2086346789 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cf58f038b0 con 0x55cf58f4dae0 2023-10-26T06:55:11.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.561+0000 7fa8e7fff700 10 client.26492 handle_command_reply: tid=0 2023-10-26T06:55:11.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.561+0000 7fa922708b80 1 client.26492 shutdown 2023-10-26T06:55:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.562+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 msgr2=0x55cf58f40360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.562+0000 7fa922708b80 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f40360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc005f30 tx=0x7fa8fc00a9a0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.562+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 shutdown_connections 2023-10-26T06:55:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.562+0000 7fa922708b80 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f3dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.562+0000 7fa922708b80 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58f4ff00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.562+0000 7fa922708b80 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f40360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.563+0000 7fa8e7fff700 0 client.26492 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.563+0000 7fa922708b80 1 --2- 172.21.15.89:0/2086346789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f3d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.563+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 >> 172.21.15.89:0/2086346789 conn(0x55cf58ea8720 msgr2=0x55cf58f33080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.563+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 shutdown_connections 2023-10-26T06:55:11.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.563+0000 7fa922708b80 1 -- 172.21.15.89:0/2086346789 wait complete. 2023-10-26T06:55:11.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.564+0000 7fa922708b80 20 client.26492 trim_cache size 0 max 16384 2023-10-26T06:55:11.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.595+0000 7fa922708b80 1 Processor -- start 2023-10-26T06:55:11.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.596+0000 7fa922708b80 1 -- start start 2023-10-26T06:55:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.597+0000 7fa922708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.597+0000 7fa922708b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f44f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.597+0000 7fa922708b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.597+0000 7fa922708b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58d07920 con 0x55cf58f3a500 2023-10-26T06:55:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.597+0000 7fa922708b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58f26fd0 con 0x55cf58f345d0 2023-10-26T06:55:11.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.597+0000 7fa922708b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58f454c0 con 0x55cf58f39be0 2023-10-26T06:55:11.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.598+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.598+0000 7fa907fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f44f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.598+0000 7fa907fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f44f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:40096/0 (socket says 172.21.15.89:40096) 2023-10-26T06:55:11.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.598+0000 7fa907fff700 1 -- 172.21.15.89:0/2484348220 learned_addr learned my addr 172.21.15.89:0/2484348220 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.599+0000 7fa907fff700 1 -- 172.21.15.89:0/2484348220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 msgr2=0x55cf58f349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.599+0000 7fa907fff700 1 --2- 172.21.15.89:0/2484348220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.599+0000 7fa907fff700 1 -- 172.21.15.89:0/2484348220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 msgr2=0x55cf58f376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.599+0000 7fa907fff700 1 --2- 172.21.15.89:0/2484348220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.599+0000 7fa907fff700 1 -- 172.21.15.89:0/2484348220 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf58c9ac60 con 0x55cf58f39be0 2023-10-26T06:55:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.600+0000 7fa907fff700 1 --2- 172.21.15.89:0/2484348220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f44f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa90810f7a0 tx=0x7fa8f4005d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7dd2e7df4bd47ec server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.600+0000 7fa8f37fe700 1 -- 172.21.15.89:0/2484348220 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4017070 con 0x55cf58f39be0 2023-10-26T06:55:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.600+0000 7fa8f37fe700 1 -- 172.21.15.89:0/2484348220 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8f4009dd0 con 0x55cf58f39be0 2023-10-26T06:55:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.601+0000 7fa922708b80 1 -- 172.21.15.89:0/2484348220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 msgr2=0x55cf58f44f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.601+0000 7fa922708b80 1 --2- 172.21.15.89:0/2484348220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f44f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa90810f7a0 tx=0x7fa8f4005d00 comp rx=0 tx=0).stop 2023-10-26T06:55:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.601+0000 7fa922708b80 1 -- 172.21.15.89:0/2484348220 shutdown_connections 2023-10-26T06:55:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.601+0000 7fa922708b80 1 --2- 172.21.15.89:0/2484348220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.602+0000 7fa922708b80 1 --2- 172.21.15.89:0/2484348220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.602+0000 7fa922708b80 1 --2- 172.21.15.89:0/2484348220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f44f80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.602+0000 7fa922708b80 1 -- 172.21.15.89:0/2484348220 >> 172.21.15.89:0/2484348220 conn(0x55cf58ea8720 msgr2=0x55cf58f2eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.602+0000 7fa922708b80 1 -- 172.21.15.89:0/2484348220 shutdown_connections 2023-10-26T06:55:11.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.602+0000 7fa922708b80 1 -- 172.21.15.89:0/2484348220 wait complete. 2023-10-26T06:55:11.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.602+0000 7fa922708b80 1 -- start start 2023-10-26T06:55:11.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.603+0000 7fa922708b80 1 Processor -- start 2023-10-26T06:55:11.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa922708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f44df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa922708b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f43330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa922708b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f44df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa907fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f43330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa907fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f43330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:40098/0 (socket says 172.21.15.89:40098) 2023-10-26T06:55:11.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa907fff700 1 -- 172.21.15.89:0/275032827 learned_addr learned my addr 172.21.15.89:0/275032827 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:11.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58f43fa0 con 0x55cf58f3a500 2023-10-26T06:55:11.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58f44110 con 0x55cf58f345d0 2023-10-26T06:55:11.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf58fa18c0 con 0x55cf58f39be0 2023-10-26T06:55:11.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.604+0000 7fa90cb9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f44df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:44544/0 (socket says 172.21.15.89:44544) 2023-10-26T06:55:11.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa90d39c700 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa922708b80 10 client.0 _ll_register_callbacks cb 0x55cf58d6da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:11.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa907fff700 1 -- 172.21.15.89:0/275032827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 msgr2=0x55cf58f44df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa907fff700 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f44df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa907fff700 1 -- 172.21.15.89:0/275032827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 msgr2=0x55cf58f43890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:11.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa907fff700 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f43890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa907fff700 1 -- 172.21.15.89:0/275032827 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f4009ab0 con 0x55cf58f39be0 2023-10-26T06:55:11.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.605+0000 7fa907fff700 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f43330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4017770 tx=0x7fa8f4009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f3fd86d1452256 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:11.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.606+0000 7fa8e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:55:11.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.608+0000 7fa8e7fff700 1 -- 172.21.15.89:0/275032827 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4017050 con 0x55cf58f39be0 2023-10-26T06:55:11.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.608+0000 7fa8e7fff700 1 -- 172.21.15.89:0/275032827 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8543+0+0 (secure 0 0 0) 0x7fa8f402a470 con 0x55cf58f39be0 2023-10-26T06:55:11.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.608+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8fc020070 con 0x55cf58f39be0 2023-10-26T06:55:11.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.608+0000 7fa8e7fff700 1 -- 172.21.15.89:0/275032827 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8f4004830 con 0x55cf58f39be0 2023-10-26T06:55:11.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.608+0000 7fa922708b80 10 client.26488 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:11.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.608+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55cf58cd8250 con 0x55cf58f39be0 2023-10-26T06:55:11.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.609+0000 7fa8e7fff700 1 -- 172.21.15.89:0/275032827 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8fc020070 con 0x55cf58f39be0 2023-10-26T06:55:11.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.610+0000 7fa8e7fff700 1 -- 172.21.15.89:0/275032827 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa8f402a630 con 0x55cf58f39be0 2023-10-26T06:55:11.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.613+0000 7fa922708b80 10 client.26488 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:11.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.613+0000 7fa922708b80 10 client.26488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:11.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.614+0000 7fa922708b80 20 client.26488 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:11.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.614+0000 7fa922708b80 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58fa4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:11.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.614+0000 7fa922708b80 4 client.26488 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:55:11.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.614+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cf58e7fe50 con 0x55cf58f4dae0 2023-10-26T06:55:11.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.616+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58fa4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:11.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.617+0000 7fa8e7fff700 10 client.26488 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.617+0000 7fa90cb9b700 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58fa4ee0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9386be20983ff9c3 server_cookie=d362b8b51f116ca3 in_seq=0 out_seq=0 2023-10-26T06:55:11.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.618+0000 7fa8e7fff700 1 -- 172.21.15.89:0/275032827 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cf58e7fe50 con 0x55cf58f4dae0 2023-10-26T06:55:11.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.618+0000 7fa8e7fff700 10 client.26488 handle_command_reply: tid=0 2023-10-26T06:55:11.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.618+0000 7fa922708b80 1 client.26488 shutdown 2023-10-26T06:55:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.618+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 msgr2=0x55cf58f43330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.619+0000 7fa922708b80 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f43330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4017770 tx=0x7fa8f4009750 comp rx=0 tx=0).stop 2023-10-26T06:55:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.620+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 shutdown_connections 2023-10-26T06:55:11.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa922708b80 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55cf58f3a500 0x55cf58f43890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa922708b80 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55cf58f4dae0 0x55cf58fa4ee0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa922708b80 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cf58f345d0 0x55cf58f44df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa922708b80 1 --2- 172.21.15.89:0/275032827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf58f39be0 0x55cf58f43330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 >> 172.21.15.89:0/275032827 conn(0x55cf58ea8720 msgr2=0x55cf58f3bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa8e7fff700 0 client.26488 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:11.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 shutdown_connections 2023-10-26T06:55:11.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.621+0000 7fa922708b80 1 -- 172.21.15.89:0/275032827 wait complete. 2023-10-26T06:55:11.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.622+0000 7fa922708b80 20 client.26488 trim_cache size 0 max 16384 2023-10-26T06:55:11.628 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:55:11.628 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:55:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:55:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "9a7701cd-980c-4ff8-b8c1-0014546e3e45": { 2023-10-26T06:55:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:55:11.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "9a7701cd-980c-4ff8-b8c1-0014546e3e45", 2023-10-26T06:55:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:55:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:55:11.630 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:55:11.631 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:55:11.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa8f80a1730 msgr2=0x7fa8f80a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa8f80a1730 0x7fa8f80a3bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00aca0 tx=0x7fa8fc005f90 comp rx=0 tx=0).stop 2023-10-26T06:55:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9081056e0 msgr2=0x7fa90810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:11.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9081056e0 0x7fa90810eaf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa900006070 tx=0x7fa9000077f0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 shutdown_connections 2023-10-26T06:55:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fa8f80a1730 0x7fa8f80a3bf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9081056e0 0x7fa90810eaf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9080ff640 0x7fa90810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 --2- 172.21.15.89:0/1165310521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa908100000 0x7fa90810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.623+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 >> 172.21.15.89:0/1165310521 conn(0x7fa9080767f0 msgr2=0x7fa9080fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.624+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 shutdown_connections 2023-10-26T06:55:11.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:11.624+0000 7fa90edff700 1 -- 172.21.15.89:0/1165310521 wait complete. 2023-10-26T06:55:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:11 smithi192 ceph-mon[131661]: pgmap v563: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:55:11.659 INFO:tasks.cephfs.filesystem:scrub status for tag:9a7701cd-980c-4ff8-b8c1-0014546e3e45 - {'path': '/', 'tag': '9a7701cd-980c-4ff8-b8c1-0014546e3e45', 'options': 'recursive,force'} 2023-10-26T06:55:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:11 smithi135 ceph-mon[131403]: pgmap v563: 97 pgs: 97 active+clean; 24 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 19 op/s 2023-10-26T06:55:12.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:12 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:55:12.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:12 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:12.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:12 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:55:12.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:12 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:12.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:12 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:55:12.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:12 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:12.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:12 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:55:12.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:12 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:12.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:12 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:55:12.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:12 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:12.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:12 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:55:12.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:12 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:13.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:13 smithi192 ceph-mon[131661]: pgmap v564: 97 pgs: 13 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 58 active+clean; 20 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-26T06:55:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:13 smithi135 ceph-mon[131403]: pgmap v564: 97 pgs: 13 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 58 active+clean; 20 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-26T06:55:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:13 smithi089 ceph-mon[95031]: pgmap v564: 97 pgs: 13 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 58 active+clean; 20 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 17 op/s 2023-10-26T06:55:14.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:14 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:55:14.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:14 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:55:14.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:14 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:55:15.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:15 smithi192 ceph-mon[131661]: pgmap v565: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 17 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:15.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:15 smithi135 ceph-mon[131403]: pgmap v565: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 17 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:15 smithi089 ceph-mon[95031]: pgmap v565: 97 pgs: 15 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 50 active+clean; 17 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:17.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:17 smithi192 ceph-mon[131661]: pgmap v566: 97 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 55 active+clean; 17 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:17 smithi135 ceph-mon[131403]: pgmap v566: 97 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 55 active+clean; 17 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:17 smithi089 ceph-mon[95031]: pgmap v566: 97 pgs: 13 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 55 active+clean; 17 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:19.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:19 smithi135 ceph-mon[131403]: pgmap v567: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:19.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:19 smithi089 ceph-mon[95031]: pgmap v567: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:19.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:19 smithi192 ceph-mon[131661]: pgmap v567: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:21.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:21 smithi089 ceph-mon[95031]: pgmap v568: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 192 B/s wr, 2 op/s 2023-10-26T06:55:21.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:21 smithi089 ceph-mon[95031]: osdmap e92: 12 total, 12 up, 12 in 2023-10-26T06:55:21.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:21 smithi192 ceph-mon[131661]: pgmap v568: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 192 B/s wr, 2 op/s 2023-10-26T06:55:21.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:21 smithi192 ceph-mon[131661]: osdmap e92: 12 total, 12 up, 12 in 2023-10-26T06:55:21.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:21 smithi135 ceph-mon[131403]: pgmap v568: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 192 B/s wr, 2 op/s 2023-10-26T06:55:21.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:21 smithi135 ceph-mon[131403]: osdmap e92: 12 total, 12 up, 12 in 2023-10-26T06:55:23.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:23 smithi089 ceph-mon[95031]: pgmap v570: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:23.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:23 smithi192 ceph-mon[131661]: pgmap v570: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:23 smithi135 ceph-mon[131403]: pgmap v570: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:55:25.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:25 smithi192 ceph-mon[131661]: pgmap v571: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:55:25.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:25 smithi135 ceph-mon[131403]: pgmap v571: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:55:25.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:25 smithi089 ceph-mon[95031]: pgmap v571: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:55:27.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:27 smithi192 ceph-mon[131661]: pgmap v572: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:55:27.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:27 smithi135 ceph-mon[131403]: pgmap v572: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:55:27.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:27 smithi089 ceph-mon[95031]: pgmap v572: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:55:29.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:29 smithi192 ceph-mon[131661]: pgmap v573: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:29 smithi135 ceph-mon[131403]: pgmap v573: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:29 smithi089 ceph-mon[95031]: pgmap v573: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:30.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:30 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:55:30.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:30 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:55:30.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:30 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:55:31.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:31 smithi192 ceph-mon[131661]: pgmap v574: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:31.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:55:31.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:55:31.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:55:31.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:31 smithi135 ceph-mon[131403]: pgmap v574: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:31.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:55:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:55:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:55:31.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:31 smithi089 ceph-mon[95031]: pgmap v574: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:31.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:55:31.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:55:31.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:55:33.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:33 smithi192 ceph-mon[131661]: pgmap v575: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:55:33.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:33 smithi135 ceph-mon[131403]: pgmap v575: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:55:33.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:33 smithi089 ceph-mon[95031]: pgmap v575: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:55:35.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:35 smithi192 ceph-mon[131661]: pgmap v576: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:55:35.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:35 smithi135 ceph-mon[131403]: pgmap v576: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:55:35.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:35 smithi089 ceph-mon[95031]: pgmap v576: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:55:37.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:37 smithi192 ceph-mon[131661]: pgmap v577: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:37.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:37 smithi135 ceph-mon[131403]: pgmap v577: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:37.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:37 smithi089 ceph-mon[95031]: pgmap v577: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:39.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:39 smithi192 ceph-mon[131661]: pgmap v578: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:55:39.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:39 smithi135 ceph-mon[131403]: pgmap v578: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:55:39.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:39 smithi089 ceph-mon[95031]: pgmap v578: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:55:41.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:41 smithi192 ceph-mon[131661]: pgmap v579: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:41.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:55:41.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:41 smithi135 ceph-mon[131403]: pgmap v579: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:41.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:41 smithi089 ceph-mon[95031]: pgmap v579: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:41.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 -- 172.21.15.89:0/4029521477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 msgr2=0x7f63c810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.864 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/4029521477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 0x7f63c810d5f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b0a0 tx=0x7f63c000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 -- 172.21.15.89:0/4029521477 shutdown_connections 2023-10-26T06:55:41.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/4029521477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 0x7f63c810d5f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.865 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/4029521477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8106730 0x7f63c8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/4029521477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63c8100720 0x7f63c8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 -- 172.21.15.89:0/4029521477 >> 172.21.15.89:0/4029521477 conn(0x7f63c80fc1b0 msgr2=0x7f63c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:41.866 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 -- 172.21.15.89:0/4029521477 shutdown_connections 2023-10-26T06:55:41.867 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.861+0000 7f63ce1ff700 1 -- 172.21.15.89:0/4029521477 wait complete. 2023-10-26T06:55:41.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.863+0000 7f63ce1ff700 1 Processor -- start 2023-10-26T06:55:41.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.863+0000 7f63ce1ff700 1 -- start start 2023-10-26T06:55:41.869 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.863+0000 7f63ce1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8100720 0x7f63c8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.863+0000 7f63ce1ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 0x7f63c8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63ce1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63c8106730 0x7f63c8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63ce1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c81a4f50 con 0x7f63c8106730 2023-10-26T06:55:41.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63ce1ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c81a50c0 con 0x7f63c81010e0 2023-10-26T06:55:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63ce1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c81a5280 con 0x7f63c8100720 2023-10-26T06:55:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8100720 0x7f63c8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8100720 0x7f63c8072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60430/0 (socket says 172.21.15.89:60430) 2023-10-26T06:55:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 -- 172.21.15.89:0/1928375650 learned_addr learned my addr 172.21.15.89:0/1928375650 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 0x7f63c8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c7fff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63c8106730 0x7f63c8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 -- 172.21.15.89:0/1928375650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 msgr2=0x7f63c8073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 0x7f63c8073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 -- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63c8106730 msgr2=0x7f63c8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63c8106730 0x7f63c8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 -- 172.21.15.89:0/1928375650 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc009520 con 0x7f63c8100720 2023-10-26T06:55:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c7fff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63c8106730 0x7f63c8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:55:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.864+0000 7f63c77fe700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8100720 0x7f63c8072af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f63b800e9a0 tx=0x7f63b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:41.874 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.865+0000 7f63c4ff9700 1 -- 172.21.15.89:0/1928375650 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8009790 con 0x7f63c8100720 2023-10-26T06:55:41.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.865+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c0009d00 con 0x7f63c8100720 2023-10-26T06:55:41.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.865+0000 7f63c4ff9700 1 -- 172.21.15.89:0/1928375650 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b800cbd0 con 0x7f63c8100720 2023-10-26T06:55:41.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.865+0000 7f63c4ff9700 1 -- 172.21.15.89:0/1928375650 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8018680 con 0x7f63c8100720 2023-10-26T06:55:41.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.865+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c81a5930 con 0x7f63c8100720 2023-10-26T06:55:41.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.868+0000 7f63c4ff9700 1 -- 172.21.15.89:0/1928375650 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f63b800cd40 con 0x7f63c8100720 2023-10-26T06:55:41.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.870+0000 7f63e1b96b80 1 Processor -- start 2023-10-26T06:55:41.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.870+0000 7f63c4ff9700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f63b00a15f0 0x7f63b00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.870+0000 7f63e1b96b80 1 -- start start 2023-10-26T06:55:41.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.870+0000 7f63c4ff9700 1 -- 172.21.15.89:0/1928375650 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f63b8014070 con 0x7f63c8100720 2023-10-26T06:55:41.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.870+0000 7f63c7fff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f63b00a15f0 0x7f63b00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.870+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.870+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad67a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3acda6e0 con 0x558e3ad6cbe0 2023-10-26T06:55:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3acc0c60 con 0x558e3ad675d0 2023-10-26T06:55:41.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3acbe580 con 0x558e3ad6d500 2023-10-26T06:55:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad67090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60434/0 (socket says 172.21.15.89:60434) 2023-10-26T06:55:41.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 -- 172.21.15.89:0/2167025267 learned_addr learned my addr 172.21.15.89:0/2167025267 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59898/0 (socket says 172.21.15.89:59898) 2023-10-26T06:55:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c7fff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f63b00a15f0 0x7f63b00a3ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009610 tx=0x7f63c0006230 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 -- 172.21.15.89:0/2167025267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 msgr2=0x558e3ad67a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/2167025267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad67a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 -- 172.21.15.89:0/2167025267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 msgr2=0x558e3ad6cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/2167025267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.888 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.871+0000 7f63c6ffd700 1 -- 172.21.15.89:0/2167025267 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e3ac773f0 con 0x558e3ad6d500 2023-10-26T06:55:41.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.872+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/2167025267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad67090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009eb0 tx=0x7f63bc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca709d84252b744c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:41.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.872+0000 7f63adffb700 1 -- 172.21.15.89:0/2167025267 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00e670 con 0x558e3ad6d500 2023-10-26T06:55:41.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.872+0000 7f63adffb700 1 -- 172.21.15.89:0/2167025267 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63bc00ec90 con 0x558e3ad6d500 2023-10-26T06:55:41.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.872+0000 7f63adffb700 1 -- 172.21.15.89:0/2167025267 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc019d30 con 0x558e3ad6d500 2023-10-26T06:55:41.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.872+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2167025267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 msgr2=0x558e3ad67090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.872+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2167025267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad67090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009eb0 tx=0x7f63bc011720 comp rx=0 tx=0).stop 2023-10-26T06:55:41.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2167025267 shutdown_connections 2023-10-26T06:55:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2167025267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2167025267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad67a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2167025267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad67090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2167025267 >> 172.21.15.89:0/2167025267 conn(0x558e3acdb720 msgr2=0x558e3ad61b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:41.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2167025267 shutdown_connections 2023-10-26T06:55:41.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2167025267 wait complete. 2023-10-26T06:55:41.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 -- start start 2023-10-26T06:55:41.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.873+0000 7f63e1b96b80 1 Processor -- start 2023-10-26T06:55:41.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad70820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad70d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3acb9e70 con 0x558e3ad6cbe0 2023-10-26T06:55:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3ac56710 con 0x558e3ad675d0 2023-10-26T06:55:41.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3ac563b0 con 0x558e3ad6d500 2023-10-26T06:55:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63e1b96b80 10 client.0 _ll_register_callbacks cb 0x558e3aba0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63c6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad70d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad70820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad70820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:59866/0 (socket says 172.21.15.89:59866) 2023-10-26T06:55:41.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63c6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad70d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59910/0 (socket says 172.21.15.89:59910) 2023-10-26T06:55:41.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63c77fe700 1 -- 172.21.15.89:0/1213547878 learned_addr learned my addr 172.21.15.89:0/1213547878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:41.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.874+0000 7f63c7fff700 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63c77fe700 1 -- 172.21.15.89:0/1213547878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 msgr2=0x558e3ad712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63c77fe700 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63c77fe700 1 -- 172.21.15.89:0/1213547878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 msgr2=0x558e3ad70d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63c77fe700 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad70d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63c77fe700 1 -- 172.21.15.89:0/1213547878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc009b10 con 0x558e3ad675d0 2023-10-26T06:55:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad70d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63c77fe700 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad70820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f63b809a290 tx=0x7f63b809aa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826a010e7130c95c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:41.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63a6ffd700 1 -- 172.21.15.89:0/1213547878 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b80999f0 con 0x558e3ad675d0 2023-10-26T06:55:41.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63a6ffd700 1 -- 172.21.15.89:0/1213547878 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f63b809e450 con 0x558e3ad675d0 2023-10-26T06:55:41.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e3acbeef0 con 0x558e3ad675d0 2023-10-26T06:55:41.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63a6ffd700 1 -- 172.21.15.89:0/1213547878 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b80a2690 con 0x558e3ad675d0 2023-10-26T06:55:41.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63e1b96b80 10 client.26507 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:41.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558e3aae4de0 con 0x558e3ad675d0 2023-10-26T06:55:41.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.875+0000 7f63a6ffd700 1 -- 172.21.15.89:0/1213547878 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e3acbeef0 con 0x558e3ad675d0 2023-10-26T06:55:41.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63a6ffd700 1 -- 172.21.15.89:0/1213547878 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f63b80b83f0 con 0x558e3ad675d0 2023-10-26T06:55:41.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63e1b96b80 10 client.26507 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:41.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63e1b96b80 10 client.26507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:41.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63e1b96b80 20 client.26507 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:41.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3ad80ff0 0x558e3ad83410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63e1b96b80 4 client.26507 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:41.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e3ad368b0 con 0x558e3ad80ff0 2023-10-26T06:55:41.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3ad80ff0 0x558e3ad83410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63a6ffd700 10 client.26507 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:41.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.876+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3ad80ff0 0x558e3ad83410 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ab36c894871b72 server_cookie=2f7aeccca0b5cc9d in_seq=0 out_seq=0 2023-10-26T06:55:41.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63a6ffd700 1 -- 172.21.15.89:0/1213547878 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e3ad368b0 con 0x558e3ad80ff0 2023-10-26T06:55:41.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63a6ffd700 10 client.26507 handle_command_reply: tid=0 2023-10-26T06:55:41.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 client.26507 shutdown 2023-10-26T06:55:41.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 msgr2=0x558e3ad70820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad70820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f63b809a290 tx=0x7f63b809aa80 comp rx=0 tx=0).stop 2023-10-26T06:55:41.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 shutdown_connections 2023-10-26T06:55:41.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3ad80ff0 0x558e3ad83410 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63a6ffd700 0 client.26507 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:41.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad70820 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/1213547878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad70d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 >> 172.21.15.89:0/1213547878 conn(0x558e3acdb720 msgr2=0x558e3ad66080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:41.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.877+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 shutdown_connections 2023-10-26T06:55:41.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.878+0000 7f63e1b96b80 1 -- 172.21.15.89:0/1213547878 wait complete. 2023-10-26T06:55:41.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.878+0000 7f63e1b96b80 20 client.26507 trim_cache size 0 max 16384 2023-10-26T06:55:41.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 Processor -- start 2023-10-26T06:55:41.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 -- start start 2023-10-26T06:55:41.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad675d0 0x558e3ad77ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6d500 0x558e3ad710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3ab3a920 con 0x558e3ad6d500 2023-10-26T06:55:41.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3ad71e20 con 0x558e3ad6cbe0 2023-10-26T06:55:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3ad5e230 con 0x558e3ad675d0 2023-10-26T06:55:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad675d0 0x558e3ad77ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63c7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6d500 0x558e3ad710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad675d0 0x558e3ad77ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60454/0 (socket says 172.21.15.89:60454) 2023-10-26T06:55:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63c6ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.895+0000 7f63c77fe700 1 -- 172.21.15.89:0/2084602852 learned_addr learned my addr 172.21.15.89:0/2084602852 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:41.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63c77fe700 1 -- 172.21.15.89:0/2084602852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad6cbe0 msgr2=0x558e3ad6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63c77fe700 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63c77fe700 1 -- 172.21.15.89:0/2084602852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6d500 msgr2=0x558e3ad710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63c77fe700 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6d500 0x558e3ad710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63c77fe700 1 -- 172.21.15.89:0/2084602852 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e3aacdc60 con 0x558e3ad675d0 2023-10-26T06:55:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63c7fff700 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6d500 0x558e3ad710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63c77fe700 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad675d0 0x558e3ad77ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000c00 tx=0x7f63b809a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b0e8c2c241d7e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:41.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63ae7fc700 1 -- 172.21.15.89:0/2084602852 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b809e450 con 0x558e3ad675d0 2023-10-26T06:55:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63ae7fc700 1 -- 172.21.15.89:0/2084602852 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b80b0070 con 0x558e3ad675d0 2023-10-26T06:55:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.896+0000 7f63ae7fc700 1 -- 172.21.15.89:0/2084602852 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8099a50 con 0x558e3ad675d0 2023-10-26T06:55:41.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2084602852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad675d0 msgr2=0x558e3ad77ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad675d0 0x558e3ad77ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000c00 tx=0x7f63b809a480 comp rx=0 tx=0).stop 2023-10-26T06:55:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2084602852 shutdown_connections 2023-10-26T06:55:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6d500 0x558e3ad710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad675d0 0x558e3ad77ee0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/2084602852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad6cbe0 0x558e3ad6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2084602852 >> 172.21.15.89:0/2084602852 conn(0x558e3acdb720 msgr2=0x558e3ad6b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:41.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2084602852 shutdown_connections 2023-10-26T06:55:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 -- 172.21.15.89:0/2084602852 wait complete. 2023-10-26T06:55:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.897+0000 7f63e1b96b80 1 -- start start 2023-10-26T06:55:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 1 Processor -- start 2023-10-26T06:55:41.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad77e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad76810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3ad76ee0 con 0x558e3ad6cbe0 2023-10-26T06:55:41.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3ad77050 con 0x558e3ad675d0 2023-10-26T06:55:41.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e3add4e30 con 0x558e3ad6d500 2023-10-26T06:55:41.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63e1b96b80 10 client.0 _ll_register_callbacks cb 0x558e3aba0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad77e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63c77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad77e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:59890/0 (socket says 172.21.15.89:59890) 2023-10-26T06:55:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63c77fe700 1 -- 172.21.15.89:0/215850533 learned_addr learned my addr 172.21.15.89:0/215850533 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.898+0000 7f63c7fff700 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad76810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63c77fe700 1 -- 172.21.15.89:0/215850533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 msgr2=0x558e3ad76810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63c77fe700 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad76810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63c77fe700 1 -- 172.21.15.89:0/215850533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 msgr2=0x558e3ad762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63c77fe700 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63c77fe700 1 -- 172.21.15.89:0/215850533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c810def0 con 0x558e3ad675d0 2023-10-26T06:55:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63c77fe700 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad77e90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63b8099280 tx=0x7f63b80092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a9b184e6e514a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63a6ffd700 1 -- 172.21.15.89:0/215850533 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b80a2690 con 0x558e3ad675d0 2023-10-26T06:55:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63a6ffd700 1 -- 172.21.15.89:0/215850533 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f63b809e450 con 0x558e3ad675d0 2023-10-26T06:55:41.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63bc00ec90 con 0x558e3ad675d0 2023-10-26T06:55:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63a6ffd700 1 -- 172.21.15.89:0/215850533 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b80997f0 con 0x558e3ad675d0 2023-10-26T06:55:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63e1b96b80 10 client.26513 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558e3aae4de0 con 0x558e3ad675d0 2023-10-26T06:55:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.899+0000 7f63a6ffd700 1 -- 172.21.15.89:0/215850533 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63bc00ec90 con 0x558e3ad675d0 2023-10-26T06:55:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63a6ffd700 1 -- 172.21.15.89:0/215850533 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f63b80a8020 con 0x558e3ad675d0 2023-10-26T06:55:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63e1b96b80 10 client.26513 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63e1b96b80 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63e1b96b80 20 client.26513 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3add5fe0 0x558e3add83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63e1b96b80 4 client.26513 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:55:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e3acb2e50 con 0x558e3add5fe0 2023-10-26T06:55:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3add5fe0 0x558e3add83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63a6ffd700 10 client.26513 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63c6ffd700 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3add5fe0 0x558e3add83d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de153bb48fb4559d server_cookie=91139fa37a5f301e in_seq=0 out_seq=0 2023-10-26T06:55:41.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63a6ffd700 1 -- 172.21.15.89:0/215850533 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558e3acb2e50 con 0x558e3add5fe0 2023-10-26T06:55:41.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63a6ffd700 10 client.26513 handle_command_reply: tid=0 2023-10-26T06:55:41.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.900+0000 7f63e1b96b80 1 client.26513 shutdown 2023-10-26T06:55:41.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 msgr2=0x558e3ad77e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad77e90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63b8099280 tx=0x7f63b80092d0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 shutdown_connections 2023-10-26T06:55:41.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x558e3add5fe0 0x558e3add83d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e3ad6d500 0x558e3ad76810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63a6ffd700 0 client.26513 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:41.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e3ad675d0 0x558e3ad77e90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 --2- 172.21.15.89:0/215850533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558e3ad6cbe0 0x558e3ad762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 >> 172.21.15.89:0/215850533 conn(0x558e3acdb720 msgr2=0x558e3ad5b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 shutdown_connections 2023-10-26T06:55:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 1 -- 172.21.15.89:0/215850533 wait complete. 2023-10-26T06:55:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.901+0000 7f63e1b96b80 20 client.26513 trim_cache size 0 max 16384 2023-10-26T06:55:41.936 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:55:41.936 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:55:41.937 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:55:41.937 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:55:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f63b00a15f0 msgr2=0x7f63b00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f63b00a15f0 0x7f63b00a3ab0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009610 tx=0x7f63c0006230 comp rx=0 tx=0).stop 2023-10-26T06:55:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8100720 msgr2=0x7f63c8072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8100720 0x7f63c8072af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f63b800e9a0 tx=0x7f63b800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 shutdown_connections 2023-10-26T06:55:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f63b00a15f0 0x7f63b00a3ab0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63c8106730 0x7f63c8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63c8100720 0x7f63c8072af0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 --2- 172.21.15.89:0/1928375650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81010e0 0x7f63c8073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 >> 172.21.15.89:0/1928375650 conn(0x7f63c80fc1b0 msgr2=0x7f63c80fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 shutdown_connections 2023-10-26T06:55:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:41.902+0000 7f63ce1ff700 1 -- 172.21.15.89:0/1928375650 wait complete. 2023-10-26T06:55:41.942 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:55:41.942 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:55:42.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 -- 172.21.15.89:0/667113867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 msgr2=0x7f8690106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.148 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 --2- 172.21.15.89:0/667113867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 0x7f8690106a90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f868c009990 tx=0x7f868c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 -- 172.21.15.89:0/667113867 shutdown_connections 2023-10-26T06:55:42.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 --2- 172.21.15.89:0/667113867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8690101060 0x7f869010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 --2- 172.21.15.89:0/667113867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 0x7f8690106a90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 --2- 172.21.15.89:0/667113867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f86901006a0 0x7f8690100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 -- 172.21.15.89:0/667113867 >> 172.21.15.89:0/667113867 conn(0x7f86900fc170 msgr2=0x7f86900fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 -- 172.21.15.89:0/667113867 shutdown_connections 2023-10-26T06:55:42.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.145+0000 7f8697bff700 1 -- 172.21.15.89:0/667113867 wait complete. 2023-10-26T06:55:42.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.146+0000 7f8697bff700 1 Processor -- start 2023-10-26T06:55:42.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f8697bff700 1 -- start start 2023-10-26T06:55:42.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f8697bff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86901006a0 0x7f869019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f8697bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8690101060 0x7f869019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f8697bff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 0x7f8690198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f8697bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8690199160 con 0x7f8690101060 2023-10-26T06:55:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f8697bff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86901992d0 con 0x7f86901006a0 2023-10-26T06:55:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f8697bff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8690199490 con 0x7f86901066b0 2023-10-26T06:55:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f869599b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86901006a0 0x7f869019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f869519a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8690101060 0x7f869019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f869599b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86901006a0 0x7f869019e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:59898/0 (socket says 172.21.15.89:59898) 2023-10-26T06:55:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f869599b700 1 -- 172.21.15.89:0/1272255987 learned_addr learned my addr 172.21.15.89:0/1272255987 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:42.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.147+0000 7f869619c700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 0x7f8690198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f869599b700 1 -- 172.21.15.89:0/1272255987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 msgr2=0x7f8690198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f869599b700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 0x7f8690198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f869599b700 1 -- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8690101060 msgr2=0x7f869019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f869599b700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8690101060 0x7f869019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f869599b700 1 -- 172.21.15.89:0/1272255987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688009c40 con 0x7f86901006a0 2023-10-26T06:55:42.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f869599b700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86901006a0 0x7f869019e9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f868c005910 tx=0x7f868c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f8686ffd700 1 -- 172.21.15.89:0/1272255987 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c01d070 con 0x7f86901006a0 2023-10-26T06:55:42.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f8686ffd700 1 -- 172.21.15.89:0/1272255987 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f868c022470 con 0x7f86901006a0 2023-10-26T06:55:42.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c0095f0 con 0x7f86901006a0 2023-10-26T06:55:42.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f8686ffd700 1 -- 172.21.15.89:0/1272255987 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c017650 con 0x7f86901006a0 2023-10-26T06:55:42.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.148+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86901a57e0 con 0x7f86901006a0 2023-10-26T06:55:42.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.149+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8690101de0 con 0x7f86901006a0 2023-10-26T06:55:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.151+0000 7f8686ffd700 1 -- 172.21.15.89:0/1272255987 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f868c0225e0 con 0x7f86901006a0 2023-10-26T06:55:42.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.152+0000 7f8686ffd700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f867c0a1670 0x7f867c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.152+0000 7f8686ffd700 1 -- 172.21.15.89:0/1272255987 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f868c0ce100 con 0x7f86901006a0 2023-10-26T06:55:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.153+0000 7f869519a700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f867c0a1670 0x7f867c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.153+0000 7f869519a700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f867c0a1670 0x7f867c0a3b30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8690199cc0 tx=0x7f8680009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.162 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.155+0000 7f8686ffd700 1 -- 172.21.15.89:0/1272255987 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f868c097460 con 0x7f86901006a0 2023-10-26T06:55:42.421 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.418+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8690067530 con 0x7f86901006a0 2023-10-26T06:55:42.422 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.419+0000 7f8686ffd700 1 -- 172.21.15.89:0/1272255987 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f868c09aa80 con 0x7f86901006a0 2023-10-26T06:55:42.422 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:55:42.422 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:55:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f867c0a1670 msgr2=0x7f867c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f867c0a1670 0x7f867c0a3b30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8690199cc0 tx=0x7f8680009260 comp rx=0 tx=0).stop 2023-10-26T06:55:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86901006a0 msgr2=0x7f869019e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.425 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86901006a0 0x7f869019e9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f868c005910 tx=0x7f868c004730 comp rx=0 tx=0).stop 2023-10-26T06:55:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 shutdown_connections 2023-10-26T06:55:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f867c0a1670 0x7f867c0a3b30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.426 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86901066b0 0x7f8690198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86901006a0 0x7f869019e9d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 --2- 172.21.15.89:0/1272255987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8690101060 0x7f869019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 >> 172.21.15.89:0/1272255987 conn(0x7f86900fc170 msgr2=0x7f86900fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.427 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.422+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 shutdown_connections 2023-10-26T06:55:42.428 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.423+0000 7f8697bff700 1 -- 172.21.15.89:0/1272255987 wait complete. 2023-10-26T06:55:42.428 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:55:42.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:55:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 -- 172.21.15.89:0/969002769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0105b30 msgr2=0x7f5fd010d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/969002769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0105b30 0x7f5fd010d540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800b0d0 tx=0x7f5fc800b3e0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 -- 172.21.15.89:0/969002769 shutdown_connections 2023-10-26T06:55:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/969002769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd0067d40 0x7f5fd0068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/969002769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0105b30 0x7f5fd010d540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/969002769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd0105170 0x7f5fd01055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 -- 172.21.15.89:0/969002769 >> 172.21.15.89:0/969002769 conn(0x7f5fd00767f0 msgr2=0x7f5fd0076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 -- 172.21.15.89:0/969002769 shutdown_connections 2023-10-26T06:55:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.647+0000 7f5fd67ff700 1 -- 172.21.15.89:0/969002769 wait complete. 2023-10-26T06:55:42.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.648+0000 7f5fd67ff700 1 Processor -- start 2023-10-26T06:55:42.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.648+0000 7f5fd67ff700 1 -- start start 2023-10-26T06:55:42.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fd67ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0067d40 0x7f5fd00ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fd67ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd0105170 0x7f5fd00ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fd67ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd0105b30 0x7f5fd0100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fd67ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01a5070 con 0x7f5fd0105b30 2023-10-26T06:55:42.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fd67ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01a51e0 con 0x7f5fd0105170 2023-10-26T06:55:42.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fd67ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01a53a0 con 0x7f5fd0067d40 2023-10-26T06:55:42.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0067d40 0x7f5fd00ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fcf7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd0105170 0x7f5fd00ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0067d40 0x7f5fd00ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60490/0 (socket says 172.21.15.89:60490) 2023-10-26T06:55:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fcffff700 1 -- 172.21.15.89:0/3921586573 learned_addr learned my addr 172.21.15.89:0/3921586573 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fd4d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd0105b30 0x7f5fd0100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.649+0000 7f5fcffff700 1 -- 172.21.15.89:0/3921586573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd0105170 msgr2=0x7f5fd00ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcffff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd0105170 0x7f5fd00ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcffff700 1 -- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd0105b30 msgr2=0x7f5fd0100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcffff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd0105b30 0x7f5fd0100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcffff700 1 -- 172.21.15.89:0/3921586573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc8009d00 con 0x7f5fd0067d40 2023-10-26T06:55:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd0105170 0x7f5fd00ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fd4d9c700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd0105b30 0x7f5fd0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcffff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0067d40 0x7f5fd00ff6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e9a0 tx=0x7f5fc000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcd7fa700 1 -- 172.21.15.89:0/3921586573 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0009790 con 0x7f5fd0067d40 2023-10-26T06:55:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcd7fa700 1 -- 172.21.15.89:0/3921586573 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc0010430 con 0x7f5fd0067d40 2023-10-26T06:55:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fcd7fa700 1 -- 172.21.15.89:0/3921586573 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0018680 con 0x7f5fd0067d40 2023-10-26T06:55:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd01a5680 con 0x7f5fd0067d40 2023-10-26T06:55:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.650+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd01a5b40 con 0x7f5fd0067d40 2023-10-26T06:55:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.653+0000 7f5fcd7fa700 1 -- 172.21.15.89:0/3921586573 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5fc00105a0 con 0x7f5fd0067d40 2023-10-26T06:55:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.655+0000 7f5fea1f7b80 1 Processor -- start 2023-10-26T06:55:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fcd7fa700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5fb80a15f0 0x7f5fb80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fea1f7b80 1 -- start start 2023-10-26T06:55:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fcd7fa700 1 -- 172.21.15.89:0/3921586573 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f5fc0014070 con 0x7f5fd0067d40 2023-10-26T06:55:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fd4d9c700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5fb80a15f0 0x7f5fb80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baefbe0 0x55b76baeffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baeaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76ba5d6e0 con 0x55b76baf0500 2023-10-26T06:55:42.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76ba43c60 con 0x55b76baefbe0 2023-10-26T06:55:42.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76ba41580 con 0x55b76baea5d0 2023-10-26T06:55:42.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baefbe0 0x55b76baeffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baefbe0 0x55b76baeffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:59924/0 (socket says 172.21.15.89:59924) 2023-10-26T06:55:42.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.656+0000 7f5fcffff700 1 -- 172.21.15.89:0/561326871 learned_addr learned my addr 172.21.15.89:0/561326871 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:42.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fd4d9c700 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baeaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fcffff700 1 -- 172.21.15.89:0/561326871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 msgr2=0x55b76baeaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fcffff700 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baeaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fcffff700 1 -- 172.21.15.89:0/561326871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 msgr2=0x55b76baea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fcffff700 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fcffff700 1 -- 172.21.15.89:0/561326871 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b76b9fa3f0 con 0x55b76baefbe0 2023-10-26T06:55:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fd4d9c700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5fb80a15f0 0x7f5fb80a3ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009fd0 tx=0x7f5fc800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fcffff700 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baefbe0 0x55b76baeffc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0096d10 tx=0x7f5fc0099f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb63d496e0cafa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fb67fc700 1 -- 172.21.15.89:0/561326871 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00b1070 con 0x55b76baefbe0 2023-10-26T06:55:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fb67fc700 1 -- 172.21.15.89:0/561326871 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc00a2630 con 0x55b76baefbe0 2023-10-26T06:55:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.657+0000 7f5fb67fc700 1 -- 172.21.15.89:0/561326871 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc009a740 con 0x55b76baefbe0 2023-10-26T06:55:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/561326871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baefbe0 msgr2=0x55b76baeffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baefbe0 0x55b76baeffc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0096d10 tx=0x7f5fc0099f10 comp rx=0 tx=0).stop 2023-10-26T06:55:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/561326871 shutdown_connections 2023-10-26T06:55:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baeaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baefbe0 0x55b76baeffc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/561326871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/561326871 >> 172.21.15.89:0/561326871 conn(0x55b76ba5e720 msgr2=0x55b76bae4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/561326871 shutdown_connections 2023-10-26T06:55:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.658+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/561326871 wait complete. 2023-10-26T06:55:42.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 -- start start 2023-10-26T06:55:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 Processor -- start 2023-10-26T06:55:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baea5d0 0x55b76baf6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76baf3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76ba3ce70 con 0x55b76baea5d0 2023-10-26T06:55:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76b9d9710 con 0x55b76baf0500 2023-10-26T06:55:42.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.659+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76b9d93b0 con 0x55b76baefbe0 2023-10-26T06:55:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fea1f7b80 10 client.0 _ll_register_callbacks cb 0x55b76b923a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baea5d0 0x55b76baf6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baea5d0 0x55b76baf6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59982/0 (socket says 172.21.15.89:59982) 2023-10-26T06:55:42.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 -- 172.21.15.89:0/2472362840 learned_addr learned my addr 172.21.15.89:0/2472362840 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fd4d9c700 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76baf3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 -- 172.21.15.89:0/2472362840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 msgr2=0x55b76baf3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 -- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 msgr2=0x55b76baf3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76baf3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.681 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 -- 172.21.15.89:0/2472362840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd01a5b40 con 0x55b76baea5d0 2023-10-26T06:55:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5fcffff700 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baea5d0 0x55b76baf6580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e970 tx=0x7f5fc00ca9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585fc1998bc0e41e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.660+0000 7f5faf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:55:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5faf7fe700 1 -- 172.21.15.89:0/2472362840 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00b8070 con 0x55b76baea5d0 2023-10-26T06:55:42.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5faf7fe700 1 -- 172.21.15.89:0/2472362840 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f5fc00b2070 con 0x55b76baea5d0 2023-10-26T06:55:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b76ba41ef0 con 0x55b76baea5d0 2023-10-26T06:55:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5faf7fe700 1 -- 172.21.15.89:0/2472362840 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fc00b1030 con 0x55b76baea5d0 2023-10-26T06:55:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 10 client.16476 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:42.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b76b867de0 con 0x55b76baea5d0 2023-10-26T06:55:42.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5faf7fe700 1 -- 172.21.15.89:0/2472362840 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b76ba41ef0 con 0x55b76baea5d0 2023-10-26T06:55:42.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5faf7fe700 1 -- 172.21.15.89:0/2472362840 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fc00b81d0 con 0x55b76baea5d0 2023-10-26T06:55:42.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:42.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:42.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 20 client.16476 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:42.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb03b10 0x55b76bb05f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 4 client.16476 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:42.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.661+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b76bab98b0 con 0x55b76bb03b10 2023-10-26T06:55:42.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.662+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb03b10 0x55b76bb05f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.662+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb03b10 0x55b76bb05f30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e18332d15307ac5 server_cookie=eb593a18b0891820 in_seq=0 out_seq=0 2023-10-26T06:55:42.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.662+0000 7f5faf7fe700 10 client.16476 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:42.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5faf7fe700 1 -- 172.21.15.89:0/2472362840 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b76bab98b0 con 0x55b76bb03b10 2023-10-26T06:55:42.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5faf7fe700 10 client.16476 handle_command_reply: tid=0 2023-10-26T06:55:42.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 client.16476 shutdown 2023-10-26T06:55:42.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baea5d0 msgr2=0x55b76baf6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baea5d0 0x55b76baf6580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e970 tx=0x7f5fc00ca9b0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 shutdown_connections 2023-10-26T06:55:42.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baea5d0 0x55b76baf6580 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb03b10 0x55b76bb05f30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76baf3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2472362840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5faf7fe700 0 client.16476 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:42.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 >> 172.21.15.89:0/2472362840 conn(0x55b76ba5e720 msgr2=0x55b76bae9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 shutdown_connections 2023-10-26T06:55:42.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2472362840 wait complete. 2023-10-26T06:55:42.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.663+0000 7f5fea1f7b80 20 client.16476 trim_cache size 0 max 16384 2023-10-26T06:55:42.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 Processor -- start 2023-10-26T06:55:42.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 -- start start 2023-10-26T06:55:42.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baedf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baefbe0 0x55b76baf3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76bafaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76badfaf0 con 0x55b76baefbe0 2023-10-26T06:55:42.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76b8bd920 con 0x55b76baf0500 2023-10-26T06:55:42.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76baf4c90 con 0x55b76baea5d0 2023-10-26T06:55:42.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fcf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baefbe0 0x55b76baf3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baedf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fcf7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baefbe0 0x55b76baf3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59998/0 (socket says 172.21.15.89:59998) 2023-10-26T06:55:42.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.677+0000 7f5fcf7fe700 1 -- 172.21.15.89:0/2182350195 learned_addr learned my addr 172.21.15.89:0/2182350195 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:42.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fd4d9c700 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76bafaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fcffff700 1 -- 172.21.15.89:0/2182350195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 msgr2=0x55b76bafaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fcffff700 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76bafaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fcffff700 1 -- 172.21.15.89:0/2182350195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baefbe0 msgr2=0x55b76baf3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fcffff700 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baefbe0 0x55b76baf3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fcffff700 1 -- 172.21.15.89:0/2182350195 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b76bb0ab20 con 0x55b76baea5d0 2023-10-26T06:55:42.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baefbe0 0x55b76baf3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:42.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fcffff700 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baedf40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00b85e0 tx=0x7f5fc0099f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5e8880f12d9c91e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fb6ffd700 1 -- 172.21.15.89:0/2182350195 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00b1020 con 0x55b76baea5d0 2023-10-26T06:55:42.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fb6ffd700 1 -- 172.21.15.89:0/2182350195 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc0098700 con 0x55b76baea5d0 2023-10-26T06:55:42.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.678+0000 7f5fb6ffd700 1 -- 172.21.15.89:0/2182350195 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00b8040 con 0x55b76baea5d0 2023-10-26T06:55:42.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2182350195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 msgr2=0x55b76baedf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baedf40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00b85e0 tx=0x7f5fc0099f10 comp rx=0 tx=0).stop 2023-10-26T06:55:42.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2182350195 shutdown_connections 2023-10-26T06:55:42.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baf0500 0x55b76bafaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baea5d0 0x55b76baedf40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/2182350195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baefbe0 0x55b76baf3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2182350195 >> 172.21.15.89:0/2182350195 conn(0x55b76ba5e720 msgr2=0x55b76baff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2182350195 shutdown_connections 2023-10-26T06:55:42.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/2182350195 wait complete. 2023-10-26T06:55:42.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.679+0000 7f5fea1f7b80 1 -- start start 2023-10-26T06:55:42.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 1 Processor -- start 2023-10-26T06:55:42.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baea5d0 0x55b76baf8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baf9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bb56a30 con 0x55b76baf0500 2023-10-26T06:55:42.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bb56ba0 con 0x55b76baea5d0 2023-10-26T06:55:42.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76bb56d60 con 0x55b76baefbe0 2023-10-26T06:55:42.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.680+0000 7f5fea1f7b80 10 client.0 _ll_register_callbacks cb 0x55b76b923a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:42.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baea5d0 0x55b76baf8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fd4d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baf9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcf7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baea5d0 0x55b76baf8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:59940/0 (socket says 172.21.15.89:59940) 2023-10-26T06:55:42.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 -- 172.21.15.89:0/3558152950 learned_addr learned my addr 172.21.15.89:0/3558152950 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:42.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 -- 172.21.15.89:0/3558152950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 msgr2=0x55b76baf93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 -- 172.21.15.89:0/3558152950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 msgr2=0x55b76baf9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:42.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baf9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 -- 172.21.15.89:0/3558152950 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc4004040 con 0x55b76baea5d0 2023-10-26T06:55:42.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fcffff700 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baea5d0 0x55b76baf8e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fc009a230 tx=0x7f5fc0096710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bffc6978a8f23a19 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:42.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5faf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:42.709 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5faf7fe700 1 -- 172.21.15.89:0/3558152950 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00b1070 con 0x55b76baea5d0 2023-10-26T06:55:42.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5faf7fe700 1 -- 172.21.15.89:0/3558152950 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f5fc0099500 con 0x55b76baea5d0 2023-10-26T06:55:42.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.681+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fc00a2630 con 0x55b76baea5d0 2023-10-26T06:55:42.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5faf7fe700 1 -- 172.21.15.89:0/3558152950 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fc00b8040 con 0x55b76baea5d0 2023-10-26T06:55:42.710 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 10 client.26537 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:42.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55b76b867de0 con 0x55b76baea5d0 2023-10-26T06:55:42.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5faf7fe700 1 -- 172.21.15.89:0/3558152950 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc00a2630 con 0x55b76baea5d0 2023-10-26T06:55:42.711 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5faf7fe700 1 -- 172.21.15.89:0/3558152950 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fc000ca10 con 0x55b76baea5d0 2023-10-26T06:55:42.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 10 client.26537 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:42.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 10 client.26537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:42.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 20 client.26537 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:42.712 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb58050 0x55b76bb5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:42.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 4 client.26537 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:55:42.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b76ba35e50 con 0x55b76bb58050 2023-10-26T06:55:42.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.682+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb58050 0x55b76bb5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:42.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fcf7fe700 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb58050 0x55b76bb5a440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6d781cd01dfd6f server_cookie=aef50e6079344a0c in_seq=0 out_seq=0 2023-10-26T06:55:42.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5faf7fe700 10 client.26537 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:42.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5faf7fe700 1 -- 172.21.15.89:0/3558152950 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b76ba35e50 con 0x55b76bb58050 2023-10-26T06:55:42.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5faf7fe700 10 client.26537 handle_command_reply: tid=0 2023-10-26T06:55:42.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 client.26537 shutdown 2023-10-26T06:55:42.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baea5d0 msgr2=0x55b76baf8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baea5d0 0x55b76baf8e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fc009a230 tx=0x7f5fc0096710 comp rx=0 tx=0).stop 2023-10-26T06:55:42.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 shutdown_connections 2023-10-26T06:55:42.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b76baf0500 0x55b76baf9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55b76bb58050 0x55b76bb5a440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b76baea5d0 0x55b76baf8e90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 --2- 172.21.15.89:0/3558152950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b76baefbe0 0x55b76baf93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 >> 172.21.15.89:0/3558152950 conn(0x55b76ba5e720 msgr2=0x55b76baeddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5faf7fe700 0 client.26537 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:42.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 shutdown_connections 2023-10-26T06:55:42.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 1 -- 172.21.15.89:0/3558152950 wait complete. 2023-10-26T06:55:42.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.683+0000 7f5fea1f7b80 20 client.26537 trim_cache size 0 max 16384 2023-10-26T06:55:42.721 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:55:42.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5fb80a15f0 msgr2=0x7f5fb80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5fb80a15f0 0x7f5fb80a3ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009fd0 tx=0x7f5fc800b040 comp rx=0 tx=0).stop 2023-10-26T06:55:42.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0067d40 msgr2=0x7f5fd00ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:42.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0067d40 0x7f5fd00ff6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e9a0 tx=0x7f5fc000ecb0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 shutdown_connections 2023-10-26T06:55:42.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f5fb80a15f0 0x7f5fb80a3ab0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fd0105b30 0x7f5fd0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fd0067d40 0x7f5fd00ff6b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 --2- 172.21.15.89:0/3921586573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd0105170 0x7f5fd00ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:42.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 >> 172.21.15.89:0/3921586573 conn(0x7f5fd00767f0 msgr2=0x7f5fd00fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:42.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 shutdown_connections 2023-10-26T06:55:42.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:42.684+0000 7f5fd67ff700 1 -- 172.21.15.89:0/3921586573 wait complete. 2023-10-26T06:55:43.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:43 smithi192 ceph-mon[131661]: pgmap v580: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:55:43.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:43 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1272255987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:55:43.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:43 smithi135 ceph-mon[131403]: pgmap v580: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:55:43.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:43 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1272255987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:55:43.726 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:55:43.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:55:43.787 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:43 smithi089 ceph-mon[95031]: pgmap v580: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:55:43.787 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:43 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1272255987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:55:43.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 -- 172.21.15.89:0/3484348478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430106730 msgr2=0x7f7430106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:43.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 --2- 172.21.15.89:0/3484348478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430106730 0x7f7430106b10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f74200098f0 tx=0x7f7420009c00 comp rx=0 tx=0).stop 2023-10-26T06:55:43.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 -- 172.21.15.89:0/3484348478 shutdown_connections 2023-10-26T06:55:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 --2- 172.21.15.89:0/3484348478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74301010e0 0x7f743010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 --2- 172.21.15.89:0/3484348478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430106730 0x7f7430106b10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 --2- 172.21.15.89:0/3484348478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7430100720 0x7f7430100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 -- 172.21.15.89:0/3484348478 >> 172.21.15.89:0/3484348478 conn(0x7f74300fc1b0 msgr2=0x7f74300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 -- 172.21.15.89:0/3484348478 shutdown_connections 2023-10-26T06:55:43.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.931+0000 7f74369ff700 1 -- 172.21.15.89:0/3484348478 wait complete. 2023-10-26T06:55:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.932+0000 7f74369ff700 1 Processor -- start 2023-10-26T06:55:43.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.932+0000 7f74369ff700 1 -- start start 2023-10-26T06:55:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f74369ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430100720 0x7f7430198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f74369ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74301010e0 0x7f7430199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f74369ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7430106730 0x7f743019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f74369ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743019dce0 con 0x7f74301010e0 2023-10-26T06:55:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f74369ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743019de50 con 0x7f7430100720 2023-10-26T06:55:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f74369ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743019e010 con 0x7f7430106730 2023-10-26T06:55:43.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74301010e0 0x7f7430199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430100720 0x7f7430198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430100720 0x7f7430198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:59944/0 (socket says 172.21.15.89:59944) 2023-10-26T06:55:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74301010e0 0x7f7430199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60024/0 (socket says 172.21.15.89:60024) 2023-10-26T06:55:43.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 -- 172.21.15.89:0/758703199 learned_addr learned my addr 172.21.15.89:0/758703199 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f7434f9c700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7430106730 0x7f743019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 -- 172.21.15.89:0/758703199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7430106730 msgr2=0x7f743019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:43.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7430106730 0x7f743019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 -- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74301010e0 msgr2=0x7f7430199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74301010e0 0x7f7430199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 -- 172.21.15.89:0/758703199 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420009520 con 0x7f7430100720 2023-10-26T06:55:43.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f7434f9c700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7430106730 0x7f743019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:55:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742f7fe700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74301010e0 0x7f7430199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.933+0000 7f742ffff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430100720 0x7f7430198dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7420005950 tx=0x7f742000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.934+0000 7f742d7fa700 1 -- 172.21.15.89:0/758703199 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742001c070 con 0x7f7430100720 2023-10-26T06:55:43.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.934+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743019e290 con 0x7f7430100720 2023-10-26T06:55:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.934+0000 7f742d7fa700 1 -- 172.21.15.89:0/758703199 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f742000bb20 con 0x7f7430100720 2023-10-26T06:55:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.934+0000 7f742d7fa700 1 -- 172.21.15.89:0/758703199 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200038e0 con 0x7f7430100720 2023-10-26T06:55:43.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.934+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74301a5b30 con 0x7f7430100720 2023-10-26T06:55:43.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.938+0000 7f742d7fa700 1 -- 172.21.15.89:0/758703199 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7420020030 con 0x7f7430100720 2023-10-26T06:55:43.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.939+0000 7f744a288b80 1 Processor -- start 2023-10-26T06:55:43.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.939+0000 7f744a288b80 1 -- start start 2023-10-26T06:55:43.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.939+0000 7f742d7fa700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f74180a15f0 0x7f74180a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.939+0000 7f742d7fa700 1 -- 172.21.15.89:0/758703199 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f74200cdf50 con 0x7f7430100720 2023-10-26T06:55:43.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f744a288b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f4c60 0x5614779f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f7434f9c700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f74180a15f0 0x7f74180a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f744a288b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f744a288b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779efad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f744a288b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561477962760 con 0x5614779f5580 2023-10-26T06:55:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f744a288b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x561477948ce0 con 0x5614779f4c60 2023-10-26T06:55:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f744a288b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561477946600 con 0x5614779ef650 2023-10-26T06:55:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f4c60 0x5614779f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779ef110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60034/0 (socket says 172.21.15.89:60034) 2023-10-26T06:55:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742f7fe700 1 -- 172.21.15.89:0/509022501 learned_addr learned my addr 172.21.15.89:0/509022501 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f7434f9c700 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742f7fe700 1 -- 172.21.15.89:0/509022501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 msgr2=0x5614779efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742f7fe700 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742f7fe700 1 -- 172.21.15.89:0/509022501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f4c60 msgr2=0x5614779f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.940+0000 7f742f7fe700 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f4c60 0x5614779f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.941+0000 7f742f7fe700 1 -- 172.21.15.89:0/509022501 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614778ff470 con 0x5614779f5580 2023-10-26T06:55:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.941+0000 7f7434f9c700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f74180a15f0 0x7f74180a3ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7428009fd0 tx=0x7f7428007b40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.941+0000 7f742f7fe700 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779ef110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7424011570 tx=0x7f7424011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57ae92b9adbd2f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.941+0000 7f74167fc700 1 -- 172.21.15.89:0/509022501 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424011ea0 con 0x5614779f5580 2023-10-26T06:55:43.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.941+0000 7f74167fc700 1 -- 172.21.15.89:0/509022501 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7424003680 con 0x5614779f5580 2023-10-26T06:55:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.941+0000 7f74167fc700 1 -- 172.21.15.89:0/509022501 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424019790 con 0x5614779f5580 2023-10-26T06:55:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 -- 172.21.15.89:0/509022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 msgr2=0x5614779ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779ef110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7424011570 tx=0x7f7424011880 comp rx=0 tx=0).stop 2023-10-26T06:55:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 -- 172.21.15.89:0/509022501 shutdown_connections 2023-10-26T06:55:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f4c60 0x5614779f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 --2- 172.21.15.89:0/509022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779ef110 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 -- 172.21.15.89:0/509022501 >> 172.21.15.89:0/509022501 conn(0x5614779637a0 msgr2=0x5614779e9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 -- 172.21.15.89:0/509022501 shutdown_connections 2023-10-26T06:55:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 -- 172.21.15.89:0/509022501 wait complete. 2023-10-26T06:55:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.942+0000 7f744a288b80 1 -- start start 2023-10-26T06:55:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 1 Processor -- start 2023-10-26T06:55:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779ef650 0x5614779f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x5614779f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561477941ef0 con 0x5614779f5580 2023-10-26T06:55:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614778de790 con 0x5614779ef650 2023-10-26T06:55:43.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614778de430 con 0x5614779f4c60 2023-10-26T06:55:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f744a288b80 10 client.0 _ll_register_callbacks cb 0x561477828b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f742ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779ef650 0x5614779f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f742f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x5614779f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f742ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779ef650 0x5614779f8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:59960/0 (socket says 172.21.15.89:59960) 2023-10-26T06:55:43.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f742ffff700 1 -- 172.21.15.89:0/2398784993 learned_addr learned my addr 172.21.15.89:0/2398784993 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:43.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.943+0000 7f7434f9c700 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f742ffff700 1 -- 172.21.15.89:0/2398784993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 msgr2=0x5614779f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f742ffff700 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x5614779f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f742ffff700 1 -- 172.21.15.89:0/2398784993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 msgr2=0x5614779f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f742ffff700 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f742ffff700 1 -- 172.21.15.89:0/2398784993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7424009d50 con 0x5614779ef650 2023-10-26T06:55:43.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f742ffff700 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779ef650 0x5614779f8820 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f742009acd0 tx=0x7f742009a930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b7f94b99c36af7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:43.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f740f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:43.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f740f7fe700 1 -- 172.21.15.89:0/2398784993 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742009f120 con 0x5614779ef650 2023-10-26T06:55:43.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f740f7fe700 1 -- 172.21.15.89:0/2398784993 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f74200a32d0 con 0x5614779ef650 2023-10-26T06:55:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561477946f70 con 0x5614779ef650 2023-10-26T06:55:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f740f7fe700 1 -- 172.21.15.89:0/2398784993 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74200cb930 con 0x5614779ef650 2023-10-26T06:55:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.944+0000 7f744a288b80 10 client.26549 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56147776ce60 con 0x5614779ef650 2023-10-26T06:55:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f740f7fe700 1 -- 172.21.15.89:0/2398784993 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561477946f70 con 0x5614779ef650 2023-10-26T06:55:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f740f7fe700 1 -- 172.21.15.89:0/2398784993 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f74200aed20 con 0x5614779ef650 2023-10-26T06:55:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f744a288b80 10 client.26549 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:43.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f744a288b80 10 client.26549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:43.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f744a288b80 20 client.26549 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:43.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f744a288b80 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a08b90 0x561477a0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f744a288b80 4 client.26549 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614779be930 con 0x561477a08b90 2023-10-26T06:55:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.945+0000 7f742f7fe700 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a08b90 0x561477a0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.946+0000 7f742f7fe700 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a08b90 0x561477a0af80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58775ebea833553 server_cookie=b16f1294209fddf9 in_seq=0 out_seq=0 2023-10-26T06:55:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.946+0000 7f740f7fe700 10 client.26549 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.946+0000 7f740f7fe700 1 -- 172.21.15.89:0/2398784993 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5614779be930 con 0x561477a08b90 2023-10-26T06:55:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.946+0000 7f740f7fe700 10 client.26549 handle_command_reply: tid=0 2023-10-26T06:55:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 client.26549 shutdown 2023-10-26T06:55:43.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779ef650 msgr2=0x5614779f8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:43.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779ef650 0x5614779f8820 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f742009acd0 tx=0x7f742009a930 comp rx=0 tx=0).stop 2023-10-26T06:55:43.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 shutdown_connections 2023-10-26T06:55:43.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f5580 0x5614779f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a08b90 0x561477a0af80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779ef650 0x5614779f8820 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 --2- 172.21.15.89:0/2398784993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x5614779f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f740f7fe700 0 client.26549 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:43.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 >> 172.21.15.89:0/2398784993 conn(0x5614779637a0 msgr2=0x5614779ee100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:43.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 shutdown_connections 2023-10-26T06:55:43.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 1 -- 172.21.15.89:0/2398784993 wait complete. 2023-10-26T06:55:43.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.947+0000 7f744a288b80 20 client.26549 trim_cache size 0 max 16384 2023-10-26T06:55:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.965+0000 7f744a288b80 1 Processor -- start 2023-10-26T06:55:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.965+0000 7f744a288b80 1 -- start start 2023-10-26T06:55:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.965+0000 7f744a288b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f744a288b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f4c60 0x5614779e3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f744a288b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x5614779f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f744a288b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614777c29a0 con 0x5614779f4c60 2023-10-26T06:55:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f744a288b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614779f4920 con 0x5614779f5580 2023-10-26T06:55:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f744a288b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614779fbac0 con 0x5614779ef650 2023-10-26T06:55:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60572/0 (socket says 172.21.15.89:60572) 2023-10-26T06:55:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 -- 172.21.15.89:0/2882146154 learned_addr learned my addr 172.21.15.89:0/2882146154 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742f7fe700 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f4c60 0x5614779e3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f7434f9c700 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x5614779f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 -- 172.21.15.89:0/2882146154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 msgr2=0x5614779f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x5614779f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 -- 172.21.15.89:0/2882146154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f4c60 msgr2=0x5614779e3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f4c60 0x5614779e3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 -- 172.21.15.89:0/2882146154 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561477755ce0 con 0x5614779ef650 2023-10-26T06:55:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.966+0000 7f742ffff700 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779fff30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7420099320 tx=0x7f742000bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f24088770c935db server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f7416ffd700 1 -- 172.21.15.89:0/2882146154 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200976e0 con 0x5614779ef650 2023-10-26T06:55:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f7416ffd700 1 -- 172.21.15.89:0/2882146154 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f742009b6d0 con 0x5614779ef650 2023-10-26T06:55:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f7416ffd700 1 -- 172.21.15.89:0/2882146154 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200b91d0 con 0x5614779ef650 2023-10-26T06:55:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 -- 172.21.15.89:0/2882146154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 msgr2=0x5614779fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779fff30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7420099320 tx=0x7f742000bc90 comp rx=0 tx=0).stop 2023-10-26T06:55:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 -- 172.21.15.89:0/2882146154 shutdown_connections 2023-10-26T06:55:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x5614779f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779ef650 0x5614779fff30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 --2- 172.21.15.89:0/2882146154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779f4c60 0x5614779e3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 -- 172.21.15.89:0/2882146154 >> 172.21.15.89:0/2882146154 conn(0x5614779637a0 msgr2=0x5614779e3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 -- 172.21.15.89:0/2882146154 shutdown_connections 2023-10-26T06:55:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.967+0000 7f744a288b80 1 -- 172.21.15.89:0/2882146154 wait complete. 2023-10-26T06:55:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 -- start start 2023-10-26T06:55:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 Processor -- start 2023-10-26T06:55:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779ef650 0x561477a56c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x561477a57160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x561477a51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561477a51750 con 0x5614779ef650 2023-10-26T06:55:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x561477a518c0 con 0x5614779f5580 2023-10-26T06:55:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.968+0000 7f744a288b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x561477a51a80 con 0x5614779f4c60 2023-10-26T06:55:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f744a288b80 10 client.0 _ll_register_callbacks cb 0x561477828b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x561477a57160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779ef650 0x561477a56c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f7434f9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x561477a51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x561477a57160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60576/0 (socket says 172.21.15.89:60576) 2023-10-26T06:55:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742f7fe700 1 -- 172.21.15.89:0/513424663 learned_addr learned my addr 172.21.15.89:0/513424663 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742ffff700 1 -- 172.21.15.89:0/513424663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 msgr2=0x561477a57160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742ffff700 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x561477a57160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742ffff700 1 -- 172.21.15.89:0/513424663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 msgr2=0x561477a51170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742ffff700 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x561477a51170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742ffff700 1 -- 172.21.15.89:0/513424663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56147776ce60 con 0x5614779ef650 2023-10-26T06:55:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f742ffff700 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779ef650 0x561477a56c20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f742009acd0 tx=0x7f7420005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d579bc9200b2a3aa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f740f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:55:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.969+0000 7f740f7fe700 1 -- 172.21.15.89:0/513424663 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420021590 con 0x5614779ef650 2023-10-26T06:55:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f740f7fe700 1 -- 172.21.15.89:0/513424663 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f74200b91d0 con 0x5614779ef650 2023-10-26T06:55:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614779f00c0 con 0x5614779ef650 2023-10-26T06:55:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f740f7fe700 1 -- 172.21.15.89:0/513424663 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74200a32d0 con 0x5614779ef650 2023-10-26T06:55:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 10 client.16500 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5614777932d0 con 0x5614779ef650 2023-10-26T06:55:43.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f740f7fe700 1 -- 172.21.15.89:0/513424663 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614779f00c0 con 0x5614779ef650 2023-10-26T06:55:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f740f7fe700 1 -- 172.21.15.89:0/513424663 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f742007f070 con 0x5614779ef650 2023-10-26T06:55:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:43.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 20 client.16500 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a5e680 0x561477a60a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 4 client.16500 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:55:43.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.970+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f7424003680 con 0x561477a5e680 2023-10-26T06:55:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.971+0000 7f742f7fe700 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a5e680 0x561477a60a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.971+0000 7f742f7fe700 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a5e680 0x561477a60a70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ebd90066390851 server_cookie=585ecd2f0e67bdd4 in_seq=0 out_seq=0 2023-10-26T06:55:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.971+0000 7f740f7fe700 10 client.16500 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:43.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f740f7fe700 1 -- 172.21.15.89:0/513424663 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f7424003680 con 0x561477a5e680 2023-10-26T06:55:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f740f7fe700 10 client.16500 handle_command_reply: tid=0 2023-10-26T06:55:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 client.16500 shutdown 2023-10-26T06:55:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779ef650 msgr2=0x561477a56c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:43.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779ef650 0x561477a56c20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f742009acd0 tx=0x7f7420005040 comp rx=0 tx=0).stop 2023-10-26T06:55:44.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 shutdown_connections 2023-10-26T06:55:44.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5614779ef650 0x561477a56c20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x561477a5e680 0x561477a60a70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614779f5580 0x561477a51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 --2- 172.21.15.89:0/513424663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614779f4c60 0x561477a57160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 >> 172.21.15.89:0/513424663 conn(0x5614779637a0 msgr2=0x5614779f3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f740f7fe700 0 client.16500 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 shutdown_connections 2023-10-26T06:55:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.972+0000 7f744a288b80 1 -- 172.21.15.89:0/513424663 wait complete. 2023-10-26T06:55:44.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f744a288b80 20 client.16500 trim_cache size 0 max 16384 2023-10-26T06:55:44.003 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:55:44.003 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:55:44.003 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "46cd4699-f8e4-4520-9afe-3b97c02e076c", 2023-10-26T06:55:44.004 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:55:44.004 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:55:44.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f74180a15f0 msgr2=0x7f74180a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f74180a15f0 0x7f74180a3ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7428009fd0 tx=0x7f7428007b40 comp rx=0 tx=0).stop 2023-10-26T06:55:44.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430100720 msgr2=0x7f7430198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430100720 0x7f7430198dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7420005950 tx=0x7f742000fa00 comp rx=0 tx=0).stop 2023-10-26T06:55:44.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 shutdown_connections 2023-10-26T06:55:44.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f74180a15f0 0x7f74180a3ab0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7430106730 0x7f743019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7430100720 0x7f7430198dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 --2- 172.21.15.89:0/758703199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74301010e0 0x7f7430199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 >> 172.21.15.89:0/758703199 conn(0x7f74300fc1b0 msgr2=0x7f74300fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 shutdown_connections 2023-10-26T06:55:44.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:43.973+0000 7f74369ff700 1 -- 172.21.15.89:0/758703199 wait complete. 2023-10-26T06:55:44.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:55:44.219 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 -- 172.21.15.89:0/97651337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808100740 msgr2=0x7ff808100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/97651337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808100740 0x7ff808100bc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80098f0 tx=0x7ff7f8009c00 comp rx=0 tx=0).stop 2023-10-26T06:55:44.220 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 -- 172.21.15.89:0/97651337 shutdown_connections 2023-10-26T06:55:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/97651337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808106750 0x7ff808106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/97651337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808101100 0x7ff80810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/97651337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808100740 0x7ff808100bc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 -- 172.21.15.89:0/97651337 >> 172.21.15.89:0/97651337 conn(0x7ff8080fc230 msgr2=0x7ff8080fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 -- 172.21.15.89:0/97651337 shutdown_connections 2023-10-26T06:55:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.213+0000 7ff80f9ff700 1 -- 172.21.15.89:0/97651337 wait complete. 2023-10-26T06:55:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.214+0000 7ff80f9ff700 1 Processor -- start 2023-10-26T06:55:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.214+0000 7ff80f9ff700 1 -- start start 2023-10-26T06:55:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80f9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808100740 0x7ff808198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80f9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808101100 0x7ff8081993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80f9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808106750 0x7ff80819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80f9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80819dd80 con 0x7ff808101100 2023-10-26T06:55:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80f9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80819def0 con 0x7ff808100740 2023-10-26T06:55:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80f9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80819e0b0 con 0x7ff808106750 2023-10-26T06:55:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808101100 0x7ff8081993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808101100 0x7ff8081993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60094/0 (socket says 172.21.15.89:60094) 2023-10-26T06:55:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2434798882 learned_addr learned my addr 172.21.15.89:0/2434798882 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80df9c700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808106750 0x7ff80819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.215+0000 7ff80d79b700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808100740 0x7ff808198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80df9c700 1 -- 172.21.15.89:0/2434798882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808100740 msgr2=0x7ff808198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80df9c700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808100740 0x7ff808198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80df9c700 1 -- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808101100 msgr2=0x7ff8081993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80df9c700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808101100 0x7ff8081993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80df9c700 1 -- 172.21.15.89:0/2434798882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009d00 con 0x7ff808106750 2023-10-26T06:55:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808101100 0x7ff8081993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:44.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80df9c700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808106750 0x7ff80819d740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff800009cd0 tx=0x7ff80000c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff7fe7fc700 1 -- 172.21.15.89:0/2434798882 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800017070 con 0x7ff808106750 2023-10-26T06:55:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff7fe7fc700 1 -- 172.21.15.89:0/2434798882 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff80000f040 con 0x7ff808106750 2023-10-26T06:55:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff7fe7fc700 1 -- 172.21.15.89:0/2434798882 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000e050 con 0x7ff808106750 2023-10-26T06:55:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f8009520 con 0x7ff808106750 2023-10-26T06:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.216+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8081a59f0 con 0x7ff808106750 2023-10-26T06:55:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.219+0000 7ff7fe7fc700 1 -- 172.21.15.89:0/2434798882 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff80000cd10 con 0x7ff808106750 2023-10-26T06:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff7fe7fc700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7f40a1670 0x7ff7f40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 Processor -- start 2023-10-26T06:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff7fe7fc700 1 -- 172.21.15.89:0/2434798882 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7ff8000cca20 con 0x7ff808106750 2023-10-26T06:55:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 -- start start 2023-10-26T06:55:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff80d79b700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7f40a1670 0x7ff7f40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbd47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbd4d10 0x555d6cbce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbcede0 0x555d6cbcf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cb396e0 con 0x555d6cbd43f0 2023-10-26T06:55:44.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cb1fc60 con 0x555d6cbcede0 2023-10-26T06:55:44.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.221+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cb1d580 con 0x555d6cbd4d10 2023-10-26T06:55:44.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbd4d10 0x555d6cbce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbd4d10 0x555d6cbce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60592/0 (socket says 172.21.15.89:60592) 2023-10-26T06:55:44.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2778257677 learned_addr learned my addr 172.21.15.89:0/2778257677 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:44.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80df9c700 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbcede0 0x555d6cbcf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80d79b700 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbd47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2778257677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbcede0 msgr2=0x555d6cbcf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80d79b700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7f40a1670 0x7ff7f40a3b30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff804009c30 tx=0x7ff804011440 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbcede0 0x555d6cbcf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2778257677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 msgr2=0x555d6cbd47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbd47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2778257677 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d6cad63f0 con 0x555d6cbd4d10 2023-10-26T06:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbd4d10 0x555d6cbce8a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005950 tx=0x7ff7f8005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f4777ea7d2aa66 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff7f37fe700 1 -- 172.21.15.89:0/2778257677 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f801c050 con 0x555d6cbd4d10 2023-10-26T06:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.222+0000 7ff7f37fe700 1 -- 172.21.15.89:0/2778257677 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f801d040 con 0x555d6cbd4d10 2023-10-26T06:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff7f37fe700 1 -- 172.21.15.89:0/2778257677 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80052a0 con 0x555d6cbd4d10 2023-10-26T06:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2778257677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbd4d10 msgr2=0x555d6cbce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbd4d10 0x555d6cbce8a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005950 tx=0x7ff7f8005e20 comp rx=0 tx=0).stop 2023-10-26T06:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2778257677 shutdown_connections 2023-10-26T06:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbd47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbcede0 0x555d6cbcf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2778257677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbd4d10 0x555d6cbce8a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2778257677 >> 172.21.15.89:0/2778257677 conn(0x555d6cb3a720 msgr2=0x555d6cbc0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2778257677 shutdown_connections 2023-10-26T06:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.223+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2778257677 wait complete. 2023-10-26T06:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.224+0000 7ff8232f1b80 1 -- start start 2023-10-26T06:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.224+0000 7ff8232f1b80 1 Processor -- start 2023-10-26T06:55:44.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cb18e70 con 0x555d6cbd4d10 2023-10-26T06:55:44.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cab5710 con 0x555d6cbd43f0 2023-10-26T06:55:44.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cab53b0 con 0x555d6cbcede0 2023-10-26T06:55:44.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff8232f1b80 10 client.0 _ll_register_callbacks cb 0x555d6c9ffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:44.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff80df9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:60018/0 (socket says 172.21.15.89:60018) 2023-10-26T06:55:44.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff80cf9a700 1 -- 172.21.15.89:0/3943488060 learned_addr learned my addr 172.21.15.89:0/3943488060 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:44.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.225+0000 7ff80d79b700 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff80cf9a700 1 -- 172.21.15.89:0/3943488060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 msgr2=0x555d6cbdac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff80cf9a700 1 -- 172.21.15.89:0/3943488060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 msgr2=0x555d6cbd8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff80cf9a700 1 -- 172.21.15.89:0/3943488060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80810df10 con 0x555d6cbd43f0 2023-10-26T06:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005e90 tx=0x7ff7f800b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2aee3fd2570fcd server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff7f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/3943488060 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f801c040 con 0x555d6cbd43f0 2023-10-26T06:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d6cb1def0 con 0x555d6cbd43f0 2023-10-26T06:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.226+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/3943488060 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7ff7f800f910 con 0x555d6cbd43f0 2023-10-26T06:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/3943488060 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f8005310 con 0x555d6cbd43f0 2023-10-26T06:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 10 client.26567 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555d6c943de0 con 0x555d6cbd43f0 2023-10-26T06:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/3943488060 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d6cb1def0 con 0x555d6cbd43f0 2023-10-26T06:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/3943488060 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff7f802a040 con 0x555d6cbd43f0 2023-10-26T06:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 10 client.26567 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 10 client.26567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 20 client.26567 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cbe8230 0x555d6cbea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 4 client.26567 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d6cb958b0 con 0x555d6cbe8230 2023-10-26T06:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.227+0000 7ff80d79b700 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cbe8230 0x555d6cbea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.228+0000 7ff7f0ff9700 10 client.26567 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.228+0000 7ff80d79b700 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cbe8230 0x555d6cbea650 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8313de708e943004 server_cookie=6b05c349629ac5c5 in_seq=0 out_seq=0 2023-10-26T06:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.228+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/3943488060 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d6cb958b0 con 0x555d6cbe8230 2023-10-26T06:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.228+0000 7ff7f0ff9700 10 client.26567 handle_command_reply: tid=0 2023-10-26T06:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.228+0000 7ff8232f1b80 1 client.26567 shutdown 2023-10-26T06:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 msgr2=0x555d6cbd7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005e90 tx=0x7ff7f800b7f0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 shutdown_connections 2023-10-26T06:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cbe8230 0x555d6cbea650 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/3943488060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7f00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff7f0ff9700 0 client.26567 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 >> 172.21.15.89:0/3943488060 conn(0x555d6cb3a720 msgr2=0x555d6cbcd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 shutdown_connections 2023-10-26T06:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 1 -- 172.21.15.89:0/3943488060 wait complete. 2023-10-26T06:55:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.229+0000 7ff8232f1b80 20 client.26567 trim_cache size 0 max 16384 2023-10-26T06:55:44.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.246+0000 7ff8232f1b80 1 Processor -- start 2023-10-26T06:55:44.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.246+0000 7ff8232f1b80 1 -- start start 2023-10-26T06:55:44.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbd7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.265 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbdf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd4d10 0x555d6cbd67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6c999920 con 0x555d6cbd43f0 2023-10-26T06:55:44.266 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cbe5190 con 0x555d6cbd4d10 2023-10-26T06:55:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cbc1680 con 0x555d6cbcede0 2023-10-26T06:55:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbdf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbdf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60128/0 (socket says 172.21.15.89:60128) 2023-10-26T06:55:44.267 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2322079782 learned_addr learned my addr 172.21.15.89:0/2322079782 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80d79b700 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbd7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80df9c700 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd4d10 0x555d6cbd67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2322079782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 msgr2=0x555d6cbd7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.268 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbd7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2322079782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd4d10 msgr2=0x555d6cbd67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd4d10 0x555d6cbd67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.269 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.247+0000 7ff80cf9a700 1 -- 172.21.15.89:0/2322079782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d6c92cc60 con 0x555d6cbd43f0 2023-10-26T06:55:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff80df9c700 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd4d10 0x555d6cbd67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:55:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff80d79b700 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbd7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:55:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbdf6f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005950 tx=0x7ff7f8009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0c08a12fe78cbe server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:44.270 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff7f3fff700 1 -- 172.21.15.89:0/2322079782 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f801c070 con 0x555d6cbd43f0 2023-10-26T06:55:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff7f3fff700 1 -- 172.21.15.89:0/2322079782 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f800b860 con 0x555d6cbd43f0 2023-10-26T06:55:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff7f3fff700 1 -- 172.21.15.89:0/2322079782 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8005670 con 0x555d6cbd43f0 2023-10-26T06:55:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2322079782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 msgr2=0x555d6cbdf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.271 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.248+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbdf6f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005950 tx=0x7ff7f8009fd0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2322079782 shutdown_connections 2023-10-26T06:55:44.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd4d10 0x555d6cbd67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbd7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.272 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/2322079782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd43f0 0x555d6cbdf6f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2322079782 >> 172.21.15.89:0/2322079782 conn(0x555d6cb3a720 msgr2=0x555d6cbc0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2322079782 shutdown_connections 2023-10-26T06:55:44.273 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 -- 172.21.15.89:0/2322079782 wait complete. 2023-10-26T06:55:44.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.249+0000 7ff8232f1b80 1 -- start start 2023-10-26T06:55:44.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.250+0000 7ff8232f1b80 1 Processor -- start 2023-10-26T06:55:44.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.250+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.274 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.250+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.250+0000 7ff8232f1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.250+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cbdfcb0 con 0x555d6cbd4d10 2023-10-26T06:55:44.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.250+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cbd7ec0 con 0x555d6cbd43f0 2023-10-26T06:55:44.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.250+0000 7ff8232f1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6cbd8080 con 0x555d6cbcede0 2023-10-26T06:55:44.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff8232f1b80 10 client.0 _ll_register_callbacks cb 0x555d6c9ffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:55:44.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.276 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:60630/0 (socket says 172.21.15.89:60630) 2023-10-26T06:55:44.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 -- 172.21.15.89:0/611236769 learned_addr learned my addr 172.21.15.89:0/611236769 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:55:44.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80df9c700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 -- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 msgr2=0x555d6cbd7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 -- 172.21.15.89:0/611236769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 msgr2=0x555d6cbd7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:55:44.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 -- 172.21.15.89:0/611236769 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80401e430 con 0x555d6cbcede0 2023-10-26T06:55:44.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80df9c700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:55:44.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff80d79b700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdf3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x555d6cbeaf60 tx=0x7ff804016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d30e5ed95ccb766 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:55:44.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff7f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:55:44.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/611236769 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804023040 con 0x555d6cbcede0 2023-10-26T06:55:44.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d6cb11e50 con 0x555d6cbcede0 2023-10-26T06:55:44.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.251+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/611236769 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8499+0+0 (secure 0 0 0) 0x7ff8040283a0 con 0x555d6cbcede0 2023-10-26T06:55:44.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/611236769 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8040285a0 con 0x555d6cbcede0 2023-10-26T06:55:44.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 10 client.26566 fetch_fsmap learned FSMap version 20 2023-10-26T06:55:44.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555d6c943de0 con 0x555d6cbcede0 2023-10-26T06:55:44.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/611236769 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d6cb11e50 con 0x555d6cbcede0 2023-10-26T06:55:44.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/611236769 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff804004c80 con 0x555d6cbcede0 2023-10-26T06:55:44.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 10 client.26566 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:55:44.282 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 10 client.26566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:55:44.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 20 client.26566 populate_metadata read hostname 'smithi089' 2023-10-26T06:55:44.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cc3d590 0x555d6cc3f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:55:44.283 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 4 client.26566 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:55:44.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d6cb11b70 con 0x555d6cc3d590 2023-10-26T06:55:44.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.252+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cc3d590 0x555d6cc3f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:55:44.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff80cf9a700 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cc3d590 0x555d6cc3f9b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5c24a485659166 server_cookie=834b80a396a92bfb in_seq=0 out_seq=0 2023-10-26T06:55:44.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff7f0ff9700 10 client.26566 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:44.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff7f0ff9700 1 -- 172.21.15.89:0/611236769 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555d6cb11b70 con 0x555d6cc3d590 2023-10-26T06:55:44.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff7f0ff9700 10 client.26566 handle_command_reply: tid=0 2023-10-26T06:55:44.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 client.26566 shutdown 2023-10-26T06:55:44.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 msgr2=0x555d6cbdf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdf3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x555d6cbeaf60 tx=0x7ff804016000 comp rx=0 tx=0).stop 2023-10-26T06:55:44.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 shutdown_connections 2023-10-26T06:55:44.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d6cbd4d10 0x555d6cbd7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x555d6cc3d590 0x555d6cc3f9b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d6cbcede0 0x555d6cbdf3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 --2- 172.21.15.89:0/611236769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d6cbd43f0 0x555d6cbd7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff7f0ff9700 0 client.26566 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:55:44.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 >> 172.21.15.89:0/611236769 conn(0x555d6cb3a720 msgr2=0x555d6cbd66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 shutdown_connections 2023-10-26T06:55:44.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 1 -- 172.21.15.89:0/611236769 wait complete. 2023-10-26T06:55:44.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.253+0000 7ff8232f1b80 20 client.26566 trim_cache size 0 max 16384 2023-10-26T06:55:44.289 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:55:44.290 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:55:44.290 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:55:44.290 INFO:teuthology.orchestra.run.smithi089.stdout: "46cd4699-f8e4-4520-9afe-3b97c02e076c": { 2023-10-26T06:55:44.290 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:55:44.291 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "46cd4699-f8e4-4520-9afe-3b97c02e076c", 2023-10-26T06:55:44.291 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:55:44.291 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:55:44.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:55:44.292 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:55:44.293 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7f40a1670 msgr2=0x7ff7f40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7f40a1670 0x7ff7f40a3b30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff804009c30 tx=0x7ff804011440 comp rx=0 tx=0).stop 2023-10-26T06:55:44.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808106750 msgr2=0x7ff80819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:55:44.294 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808106750 0x7ff80819d740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff800009cd0 tx=0x7ff80000c620 comp rx=0 tx=0).stop 2023-10-26T06:55:44.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 shutdown_connections 2023-10-26T06:55:44.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff7f40a1670 0x7ff7f40a3b30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff808106750 0x7ff80819d740 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.295 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808100740 0x7ff808198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 --2- 172.21.15.89:0/2434798882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff808101100 0x7ff8081993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:55:44.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 >> 172.21.15.89:0/2434798882 conn(0x7ff8080fc230 msgr2=0x7ff80810bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:55:44.296 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 shutdown_connections 2023-10-26T06:55:44.297 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:55:44.254+0000 7ff80f9ff700 1 -- 172.21.15.89:0/2434798882 wait complete. 2023-10-26T06:55:44.297 INFO:tasks.cephfs.filesystem:scrub status for tag:46cd4699-f8e4-4520-9afe-3b97c02e076c - {'path': '/', 'tag': '46cd4699-f8e4-4520-9afe-3b97c02e076c', 'options': 'recursive,force'} 2023-10-26T06:55:44.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:44 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:55:44.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:44 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:44.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:44 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:55:44.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:44 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:44.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:44 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:55:44.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:44 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:44.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:44 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:55:44.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:44 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:44.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:44 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:55:44.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:44 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:44.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:44 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:55:44.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:44 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:55:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:45 smithi192 ceph-mon[131661]: pgmap v581: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:55:45.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:45 smithi135 ceph-mon[131403]: pgmap v581: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:55:45.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:45 smithi089 ceph-mon[95031]: pgmap v581: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:55:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:47 smithi192 ceph-mon[131661]: pgmap v582: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:47.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:47 smithi135 ceph-mon[131403]: pgmap v582: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:47.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:47 smithi089 ceph-mon[95031]: pgmap v582: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:55:49.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:49 smithi192 ceph-mon[131661]: pgmap v583: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:55:49.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:49 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:55:49.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:49 smithi135 ceph-mon[131403]: pgmap v583: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:55:49.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:49 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:55:49.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:49 smithi089 ceph-mon[95031]: pgmap v583: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:55:49.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:49 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:55:51.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:51 smithi192 ceph-mon[131661]: pgmap v584: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:51 smithi135 ceph-mon[131403]: pgmap v584: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:51.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:51 smithi089 ceph-mon[95031]: pgmap v584: 97 pgs: 97 active+clean; 15 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:55:53.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:53 smithi192 ceph-mon[131661]: pgmap v585: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:55:53.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:53 smithi135 ceph-mon[131403]: pgmap v585: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:55:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:53 smithi089 ceph-mon[95031]: pgmap v585: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:55:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:55 smithi192 ceph-mon[131661]: pgmap v586: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:55:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:55 smithi135 ceph-mon[131403]: pgmap v586: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:55:55.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:55 smithi089 ceph-mon[95031]: pgmap v586: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:55:57.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:57 smithi192 ceph-mon[131661]: pgmap v587: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:55:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:57 smithi135 ceph-mon[131403]: pgmap v587: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:55:57.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:57 smithi089 ceph-mon[95031]: pgmap v587: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-10-26T06:55:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:55:59 smithi135 ceph-mon[131403]: pgmap v588: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:55:59.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:55:59 smithi089 ceph-mon[95031]: pgmap v588: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:55:59.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:55:59 smithi192 ceph-mon[131661]: pgmap v588: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:56:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:01 smithi135 ceph-mon[131403]: pgmap v589: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:56:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:01 smithi089 ceph-mon[95031]: pgmap v589: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:56:01.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:01 smithi192 ceph-mon[131661]: pgmap v589: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2023-10-26T06:56:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:03 smithi135 ceph-mon[131403]: pgmap v590: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 23 KiB/s wr, 4 op/s 2023-10-26T06:56:03.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:03 smithi089 ceph-mon[95031]: pgmap v590: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 23 KiB/s wr, 4 op/s 2023-10-26T06:56:03.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:03 smithi192 ceph-mon[131661]: pgmap v590: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 23 KiB/s wr, 4 op/s 2023-10-26T06:56:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:05 smithi135 ceph-mon[131403]: pgmap v591: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:05.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:05 smithi089 ceph-mon[95031]: pgmap v591: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:05.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:05 smithi192 ceph-mon[131661]: pgmap v591: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:07 smithi135 ceph-mon[131403]: pgmap v592: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:07.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:07 smithi089 ceph-mon[95031]: pgmap v592: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:07.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:07 smithi192 ceph-mon[131661]: pgmap v592: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:09.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:09 smithi135 ceph-mon[131403]: pgmap v593: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 10 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:09.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:09 smithi089 ceph-mon[95031]: pgmap v593: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 10 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:09.897 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:09 smithi192 ceph-mon[131661]: pgmap v593: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 10 KiB/s rd, 44 KiB/s wr, 16 op/s 2023-10-26T06:56:10.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:10 smithi135 ceph-mon[131403]: osdmap e93: 12 total, 12 up, 12 in 2023-10-26T06:56:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:10 smithi089 ceph-mon[95031]: osdmap e93: 12 total, 12 up, 12 in 2023-10-26T06:56:10.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:10 smithi192 ceph-mon[131661]: osdmap e93: 12 total, 12 up, 12 in 2023-10-26T06:56:11.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:11 smithi135 ceph-mon[131403]: pgmap v595: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:56:11.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:11 smithi089 ceph-mon[95031]: pgmap v595: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:56:11.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:11 smithi192 ceph-mon[131661]: pgmap v595: 97 pgs: 97 active+clean; 15 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:56:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:13 smithi135 ceph-mon[131403]: pgmap v596: 97 pgs: 16 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 51 active+clean; 14 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 11 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-10-26T06:56:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:13 smithi089 ceph-mon[95031]: pgmap v596: 97 pgs: 16 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 51 active+clean; 14 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 11 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-10-26T06:56:13.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:13 smithi192 ceph-mon[131661]: pgmap v596: 97 pgs: 16 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 51 active+clean; 14 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 11 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-10-26T06:56:14.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:56:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.527+0000 7f2ab659d700 1 -- 172.21.15.89:0/926420864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab0106690 msgr2=0x7f2ab0106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.527+0000 7f2ab659d700 1 --2- 172.21.15.89:0/926420864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab0106690 0x7f2ab0106a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0098f0 tx=0x7f2aac009c00 comp rx=0 tx=0).stop 2023-10-26T06:56:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.528+0000 7f2ab659d700 1 -- 172.21.15.89:0/926420864 shutdown_connections 2023-10-26T06:56:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.528+0000 7f2ab659d700 1 --2- 172.21.15.89:0/926420864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 0x7f2ab010d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.528+0000 7f2ab659d700 1 --2- 172.21.15.89:0/926420864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab0106690 0x7f2ab0106a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.528+0000 7f2ab659d700 1 --2- 172.21.15.89:0/926420864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab01005d0 0x7f2ab0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.528+0000 7f2ab659d700 1 -- 172.21.15.89:0/926420864 >> 172.21.15.89:0/926420864 conn(0x7f2ab00fc0c0 msgr2=0x7f2ab00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.528+0000 7f2ab659d700 1 -- 172.21.15.89:0/926420864 shutdown_connections 2023-10-26T06:56:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.528+0000 7f2ab659d700 1 -- 172.21.15.89:0/926420864 wait complete. 2023-10-26T06:56:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 Processor -- start 2023-10-26T06:56:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 -- start start 2023-10-26T06:56:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab01005d0 0x7f2ab019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 0x7f2ab019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab0106690 0x7f2ab0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0199120 con 0x7f2ab0106690 2023-10-26T06:56:14.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0199290 con 0x7f2ab0100f90 2023-10-26T06:56:14.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab659d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0199450 con 0x7f2ab01005d0 2023-10-26T06:56:14.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab01005d0 0x7f2ab019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab4d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 0x7f2ab019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab5d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab0106690 0x7f2ab0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab4d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 0x7f2ab019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42452/0 (socket says 172.21.15.89:42452) 2023-10-26T06:56:14.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab01005d0 0x7f2ab019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:54292/0 (socket says 172.21.15.89:54292) 2023-10-26T06:56:14.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.529+0000 7f2ab559b700 1 -- 172.21.15.89:0/1830615035 learned_addr learned my addr 172.21.15.89:0/1830615035 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:14.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab559b700 1 -- 172.21.15.89:0/1830615035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 msgr2=0x7f2ab019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 0x7f2ab019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab559b700 1 -- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab0106690 msgr2=0x7f2ab0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab0106690 0x7f2ab0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab559b700 1 -- 172.21.15.89:0/1830615035 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0009520 con 0x7f2ab01005d0 2023-10-26T06:56:14.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 0x7f2ab019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:14.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab5d9c700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab0106690 0x7f2ab0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:14.539 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab01005d0 0x7f2ab019e990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005950 tx=0x7f2aac00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2aa67fc700 1 -- 172.21.15.89:0/1830615035 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac01c070 con 0x7f2ab01005d0 2023-10-26T06:56:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2aa67fc700 1 -- 172.21.15.89:0/1830615035 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aac00baa0 con 0x7f2ab01005d0 2023-10-26T06:56:14.540 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa8009d00 con 0x7f2ab01005d0 2023-10-26T06:56:14.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.530+0000 7f2aa67fc700 1 -- 172.21.15.89:0/1830615035 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0038e0 con 0x7f2ab01005d0 2023-10-26T06:56:14.541 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.531+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac009520 con 0x7f2ab01005d0 2023-10-26T06:56:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.533+0000 7f2aa67fc700 1 -- 172.21.15.89:0/1830615035 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2aac020030 con 0x7f2ab01005d0 2023-10-26T06:56:14.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.534+0000 7f2aa67fc700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a9c0a15f0 0x7f2a9c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.535+0000 7f2aa67fc700 1 -- 172.21.15.89:0/1830615035 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f2aac0cdeb0 con 0x7f2ab01005d0 2023-10-26T06:56:14.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.535+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a9c0a15f0 0x7f2a9c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.535+0000 7f2acb05fb80 1 Processor -- start 2023-10-26T06:56:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.535+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a9c0a15f0 0x7f2a9c0a3ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009fd0 tx=0x7f2aa0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.535+0000 7f2acb05fb80 1 -- start start 2023-10-26T06:56:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec89140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec83820 0x55e00ec83c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ebf66e0 con 0x55e00ec89680 2023-10-26T06:56:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ebdcc60 con 0x55e00ec88d60 2023-10-26T06:56:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ebda580 con 0x55e00ec83820 2023-10-26T06:56:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab4d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec89140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab5d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec83820 0x55e00ec83c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec89140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42460/0 (socket says 172.21.15.89:42460) 2023-10-26T06:56:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab4d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec832e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47106/0 (socket says 172.21.15.89:47106) 2023-10-26T06:56:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 -- 172.21.15.89:0/3980910651 learned_addr learned my addr 172.21.15.89:0/3980910651 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 -- 172.21.15.89:0/3980910651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec83820 msgr2=0x55e00ec83c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 --2- 172.21.15.89:0/3980910651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec83820 0x55e00ec83c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 -- 172.21.15.89:0/3980910651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 msgr2=0x55e00ec832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 --2- 172.21.15.89:0/3980910651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.536+0000 7f2ab559b700 1 -- 172.21.15.89:0/3980910651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e00eb933f0 con 0x55e00ec88d60 2023-10-26T06:56:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.537+0000 7f2ab559b700 1 --2- 172.21.15.89:0/3980910651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec89140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2aac097920 tx=0x7f2aac0054c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d403a7c328c00d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.537+0000 7f2a9b7fe700 1 -- 172.21.15.89:0/3980910651 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac09b5b0 con 0x55e00ec88d60 2023-10-26T06:56:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.537+0000 7f2a9b7fe700 1 -- 172.21.15.89:0/3980910651 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aac09aeb0 con 0x55e00ec88d60 2023-10-26T06:56:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.537+0000 7f2a9b7fe700 1 -- 172.21.15.89:0/3980910651 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac099370 con 0x55e00ec88d60 2023-10-26T06:56:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 -- 172.21.15.89:0/3980910651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 msgr2=0x55e00ec89140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/3980910651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec89140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2aac097920 tx=0x7f2aac0054c0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 -- 172.21.15.89:0/3980910651 shutdown_connections 2023-10-26T06:56:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/3980910651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec83820 0x55e00ec83c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/3980910651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec89140 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/3980910651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 -- 172.21.15.89:0/3980910651 >> 172.21.15.89:0/3980910651 conn(0x55e00ebf7720 msgr2=0x55e00ec7dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 -- 172.21.15.89:0/3980910651 shutdown_connections 2023-10-26T06:56:14.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 -- 172.21.15.89:0/3980910651 wait complete. 2023-10-26T06:56:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 -- start start 2023-10-26T06:56:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.538+0000 7f2acb05fb80 1 Processor -- start 2023-10-26T06:56:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec83820 0x55e00ec8f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec8c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec89680 0x55e00ec8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ebd5e70 con 0x55e00ec83820 2023-10-26T06:56:14.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00eb72710 con 0x55e00ec88d60 2023-10-26T06:56:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00eb723b0 con 0x55e00ec89680 2023-10-26T06:56:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2acb05fb80 10 client.0 _ll_register_callbacks cb 0x55e00eabca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2ab4d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec8c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec83820 0x55e00ec8f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2ab5d9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec89680 0x55e00ec8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2ab4d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec8c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42462/0 (socket says 172.21.15.89:42462) 2023-10-26T06:56:14.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.539+0000 7f2ab4d9a700 1 -- 172.21.15.89:0/521761954 learned_addr learned my addr 172.21.15.89:0/521761954 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2ab4d9a700 1 -- 172.21.15.89:0/521761954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec89680 msgr2=0x55e00ec8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec89680 0x55e00ec8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2ab4d9a700 1 -- 172.21.15.89:0/521761954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec83820 msgr2=0x55e00ec8f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec83820 0x55e00ec8f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2ab4d9a700 1 -- 172.21.15.89:0/521761954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac009520 con 0x55e00ec88d60 2023-10-26T06:56:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec8c950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000ec10 tx=0x7f2aa00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab95d23943e4acab server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.562 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2a98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:56:14.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2a98ff9700 1 -- 172.21.15.89:0/521761954 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa001eaa0 con 0x55e00ec88d60 2023-10-26T06:56:14.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e00ebdaef0 con 0x55e00ec88d60 2023-10-26T06:56:14.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2a98ff9700 1 -- 172.21.15.89:0/521761954 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f2aa001ec00 con 0x55e00ec88d60 2023-10-26T06:56:14.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2a98ff9700 1 -- 172.21.15.89:0/521761954 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa001edc0 con 0x55e00ec88d60 2023-10-26T06:56:14.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2acb05fb80 10 client.26585 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:14.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.540+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e00ea00de0 con 0x55e00ec88d60 2023-10-26T06:56:14.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2a98ff9700 1 -- 172.21.15.89:0/521761954 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa0033020 con 0x55e00ec88d60 2023-10-26T06:56:14.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2a98ff9700 1 -- 172.21.15.89:0/521761954 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2aa001c480 con 0x55e00ec88d60 2023-10-26T06:56:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2acb05fb80 10 client.26585 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2acb05fb80 10 client.26585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:14.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2acb05fb80 20 client.26585 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ec9cd90 0x55e00ec9f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2acb05fb80 4 client.26585 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e00ec528b0 con 0x55e00ec9cd90 2023-10-26T06:56:14.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.541+0000 7f2ab559b700 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ec9cd90 0x55e00ec9f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.542+0000 7f2ab559b700 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ec9cd90 0x55e00ec9f1b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e47fb0db64833c4 server_cookie=6494ab95290b6 in_seq=0 out_seq=0 2023-10-26T06:56:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.542+0000 7f2a98ff9700 10 client.26585 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.542+0000 7f2a98ff9700 1 -- 172.21.15.89:0/521761954 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e00ec528b0 con 0x55e00ec9cd90 2023-10-26T06:56:14.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.542+0000 7f2a98ff9700 10 client.26585 handle_command_reply: tid=0 2023-10-26T06:56:14.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.542+0000 7f2acb05fb80 1 client.26585 shutdown 2023-10-26T06:56:14.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 msgr2=0x55e00ec8c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec8c950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000ec10 tx=0x7f2aa00047c0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 shutdown_connections 2023-10-26T06:56:14.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec83820 0x55e00ec8f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ec9cd90 0x55e00ec9f1b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec89680 0x55e00ec8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/521761954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec88d60 0x55e00ec8c950 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2a98ff9700 0 client.26585 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:14.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 >> 172.21.15.89:0/521761954 conn(0x55e00ebf7720 msgr2=0x55e00ec822d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:14.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 shutdown_connections 2023-10-26T06:56:14.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 1 -- 172.21.15.89:0/521761954 wait complete. 2023-10-26T06:56:14.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.543+0000 7f2acb05fb80 20 client.26585 trim_cache size 0 max 16384 2023-10-26T06:56:14.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 Processor -- start 2023-10-26T06:56:14.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 -- start start 2023-10-26T06:56:14.576 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec7da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec8c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ea56920 con 0x55e00ec89680 2023-10-26T06:56:14.577 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ec8af20 con 0x55e00ec83820 2023-10-26T06:56:14.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.560+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ec99100 con 0x55e00ec88d60 2023-10-26T06:56:14.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.578 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab4d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec7da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab5d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec8c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42472/0 (socket says 172.21.15.89:42472) 2023-10-26T06:56:14.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 -- 172.21.15.89:0/1252527348 learned_addr learned my addr 172.21.15.89:0/1252527348 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:14.579 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 -- 172.21.15.89:0/1252527348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 msgr2=0x55e00ec7da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1252527348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec7da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 -- 172.21.15.89:0/1252527348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 msgr2=0x55e00ec8c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.580 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1252527348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec8c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 -- 172.21.15.89:0/1252527348 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e00e9e9c60 con 0x55e00ec83820 2023-10-26T06:56:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1252527348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0973b0 tx=0x7f2aac00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=336c5bf0d05f9eb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2a9bfff700 1 -- 172.21.15.89:0/1252527348 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0b8040 con 0x55e00ec83820 2023-10-26T06:56:14.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2a9bfff700 1 -- 172.21.15.89:0/1252527348 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aac09f000 con 0x55e00ec83820 2023-10-26T06:56:14.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.561+0000 7f2a9bfff700 1 -- 172.21.15.89:0/1252527348 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0975c0 con 0x55e00ec83820 2023-10-26T06:56:14.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1252527348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 msgr2=0x55e00ec93db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1252527348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0973b0 tx=0x7f2aac00bfd0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1252527348 shutdown_connections 2023-10-26T06:56:14.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1252527348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec8c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1252527348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1252527348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec7da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1252527348 >> 172.21.15.89:0/1252527348 conn(0x55e00ebf7720 msgr2=0x55e00ec822d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:14.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1252527348 shutdown_connections 2023-10-26T06:56:14.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1252527348 wait complete. 2023-10-26T06:56:14.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.562+0000 7f2acb05fb80 1 -- start start 2023-10-26T06:56:14.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 1 Processor -- start 2023-10-26T06:56:14.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.585 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec92370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec92a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ec931a0 con 0x55e00ec89680 2023-10-26T06:56:14.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ecf0bd0 con 0x55e00ec83820 2023-10-26T06:56:14.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ecf0d10 con 0x55e00ec88d60 2023-10-26T06:56:14.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2acb05fb80 10 client.0 _ll_register_callbacks cb 0x55e00eabca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:14.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2ab559b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42488/0 (socket says 172.21.15.89:42488) 2023-10-26T06:56:14.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2ab559b700 1 -- 172.21.15.89:0/1739458700 learned_addr learned my addr 172.21.15.89:0/1739458700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:14.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec92370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.563+0000 7f2ab5d9c700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec92a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2ab559b700 1 -- 172.21.15.89:0/1739458700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 msgr2=0x55e00ec92370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec92370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2ab559b700 1 -- 172.21.15.89:0/1739458700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 msgr2=0x55e00ec92a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec92a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2ab559b700 1 -- 172.21.15.89:0/1739458700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e00eb933f0 con 0x55e00ec83820 2023-10-26T06:56:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2ab5d9c700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec92a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:14.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2ab559b700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2aac097920 tx=0x7f2aac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2011d1de656652 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2a98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:56:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2a98ff9700 1 -- 172.21.15.89:0/1739458700 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0b8070 con 0x55e00ec83820 2023-10-26T06:56:14.591 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2a98ff9700 1 -- 172.21.15.89:0/1739458700 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f2aac0a31b0 con 0x55e00ec83820 2023-10-26T06:56:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aac09aeb0 con 0x55e00ec83820 2023-10-26T06:56:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2a98ff9700 1 -- 172.21.15.89:0/1739458700 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aac0a10e0 con 0x55e00ec83820 2023-10-26T06:56:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2acb05fb80 10 client.26591 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:14.592 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55e00ea27250 con 0x55e00ec83820 2023-10-26T06:56:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.564+0000 7f2a98ff9700 1 -- 172.21.15.89:0/1739458700 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aac09aeb0 con 0x55e00ec83820 2023-10-26T06:56:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2a98ff9700 1 -- 172.21.15.89:0/1739458700 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2aac0cb8a0 con 0x55e00ec83820 2023-10-26T06:56:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2acb05fb80 10 client.26591 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2acb05fb80 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2acb05fb80 20 client.26591 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ecf1f80 0x55e00ecf4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2acb05fb80 4 client.26591 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:56:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e00ebcee50 con 0x55e00ecf1f80 2023-10-26T06:56:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ecf1f80 0x55e00ecf4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2ab4d9a700 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ecf1f80 0x55e00ecf4370 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c44dfff8a892f10 server_cookie=c334448a2c31c69c in_seq=0 out_seq=0 2023-10-26T06:56:14.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2a98ff9700 10 client.26591 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:14.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.565+0000 7f2a98ff9700 1 -- 172.21.15.89:0/1739458700 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e00ebcee50 con 0x55e00ecf1f80 2023-10-26T06:56:14.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2a98ff9700 10 client.26591 handle_command_reply: tid=0 2023-10-26T06:56:14.596 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 client.26591 shutdown 2023-10-26T06:56:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 msgr2=0x55e00ec93d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2aac097920 tx=0x7f2aac009c00 comp rx=0 tx=0).stop 2023-10-26T06:56:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 shutdown_connections 2023-10-26T06:56:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e00ec89680 0x55e00ec92a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55e00ecf1f80 0x55e00ecf4370 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e00ec83820 0x55e00ec93d70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 --2- 172.21.15.89:0/1739458700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e00ec88d60 0x55e00ec92370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2a98ff9700 0 client.26591 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 >> 172.21.15.89:0/1739458700 conn(0x55e00ebf7720 msgr2=0x55e00ec7da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 shutdown_connections 2023-10-26T06:56:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 1 -- 172.21.15.89:0/1739458700 wait complete. 2023-10-26T06:56:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.566+0000 7f2acb05fb80 20 client.26591 trim_cache size 0 max 16384 2023-10-26T06:56:14.600 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:56:14.600 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:56:14.601 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:56:14.601 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:56:14.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a9c0a15f0 msgr2=0x7f2a9c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a9c0a15f0 0x7f2a9c0a3ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009fd0 tx=0x7f2aa0009260 comp rx=0 tx=0).stop 2023-10-26T06:56:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab01005d0 msgr2=0x7f2ab019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab01005d0 0x7f2ab019e990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005950 tx=0x7f2aac00fa00 comp rx=0 tx=0).stop 2023-10-26T06:56:14.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 shutdown_connections 2023-10-26T06:56:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2a9c0a15f0 0x7f2a9c0a3ab0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ab0106690 0x7f2ab0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab01005d0 0x7f2ab019e990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 --2- 172.21.15.89:0/1830615035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ab0100f90 0x7f2ab019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 >> 172.21.15.89:0/1830615035 conn(0x7f2ab00fc0c0 msgr2=0x7f2ab00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 shutdown_connections 2023-10-26T06:56:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.567+0000 7f2ab659d700 1 -- 172.21.15.89:0/1830615035 wait complete. 2023-10-26T06:56:14.606 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:56:14.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:56:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 -- 172.21.15.89:0/944629162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058106510 msgr2=0x7f00581068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.813 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 --2- 172.21.15.89:0/944629162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058106510 0x7f00581068f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0054009990 tx=0x7f0054009ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 -- 172.21.15.89:0/944629162 shutdown_connections 2023-10-26T06:56:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 --2- 172.21.15.89:0/944629162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00580681f0 0x7f005810d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 --2- 172.21.15.89:0/944629162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058106510 0x7f00581068f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 --2- 172.21.15.89:0/944629162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0058067830 0x7f0058067cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 -- 172.21.15.89:0/944629162 >> 172.21.15.89:0/944629162 conn(0x7f0058074ee0 msgr2=0x7f00580752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 -- 172.21.15.89:0/944629162 shutdown_connections 2023-10-26T06:56:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.811+0000 7f005f39d700 1 -- 172.21.15.89:0/944629162 wait complete. 2023-10-26T06:56:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.812+0000 7f005f39d700 1 Processor -- start 2023-10-26T06:56:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.812+0000 7f005f39d700 1 -- start start 2023-10-26T06:56:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005f39d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058067830 0x7f005819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005f39d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00580681f0 0x7f005819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005f39d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0058106510 0x7f00581988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005f39d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058199090 con 0x7f00580681f0 2023-10-26T06:56:14.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005f39d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058199200 con 0x7f0058067830 2023-10-26T06:56:14.819 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005f39d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00581993c0 con 0x7f0058106510 2023-10-26T06:56:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058067830 0x7f005819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005db9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00580681f0 0x7f005819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058067830 0x7f005819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42504/0 (socket says 172.21.15.89:42504) 2023-10-26T06:56:14.820 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 -- 172.21.15.89:0/1089603866 learned_addr learned my addr 172.21.15.89:0/1089603866 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005eb9c700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0058106510 0x7f00581988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005db9a700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00580681f0 0x7f005819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47148/0 (socket says 172.21.15.89:47148) 2023-10-26T06:56:14.821 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 -- 172.21.15.89:0/1089603866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0058106510 msgr2=0x7f00581988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0058106510 0x7f00581988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 -- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00580681f0 msgr2=0x7f005819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00580681f0 0x7f005819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:14.822 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005e39b700 1 -- 172.21.15.89:0/1089603866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540095f0 con 0x7f0058067830 2023-10-26T06:56:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.813+0000 7f005eb9c700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0058106510 0x7f00581988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.814+0000 7f005e39b700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058067830 0x7f005819e820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0054000c00 tx=0x7f0054005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.823 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.814+0000 7f004f7fe700 1 -- 172.21.15.89:0/1089603866 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005401d070 con 0x7f0058067830 2023-10-26T06:56:14.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.814+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0058199640 con 0x7f0058067830 2023-10-26T06:56:14.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.814+0000 7f004f7fe700 1 -- 172.21.15.89:0/1089603866 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0054022470 con 0x7f0058067830 2023-10-26T06:56:14.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.814+0000 7f004f7fe700 1 -- 172.21.15.89:0/1089603866 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054017650 con 0x7f0058067830 2023-10-26T06:56:14.824 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.814+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00581a57a0 con 0x7f0058067830 2023-10-26T06:56:14.825 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.815+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0058101bc0 con 0x7f0058067830 2023-10-26T06:56:14.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.817+0000 7f004f7fe700 1 -- 172.21.15.89:0/1089603866 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f00540225e0 con 0x7f0058067830 2023-10-26T06:56:14.826 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.818+0000 7f004f7fe700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00440a1670 0x7f00440a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:14.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.819+0000 7f004f7fe700 1 -- 172.21.15.89:0/1089603866 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f00540ce4d0 con 0x7f0058067830 2023-10-26T06:56:14.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.819+0000 7f005db9a700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00440a1670 0x7f00440a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:14.827 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.819+0000 7f005db9a700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00440a1670 0x7f00440a3b30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f005819a030 tx=0x7f0048009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:14.828 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:14.821+0000 7f004f7fe700 1 -- 172.21.15.89:0/1089603866 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0054097810 con 0x7f0058067830 2023-10-26T06:56:15.078 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.077+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0058199da0 con 0x7f0058067830 2023-10-26T06:56:15.079 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.078+0000 7f004f7fe700 1 -- 172.21.15.89:0/1089603866 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f0054027090 con 0x7f0058067830 2023-10-26T06:56:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:56:15.080 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:56:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.080+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00440a1670 msgr2=0x7f00440a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.080+0000 7f005f39d700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00440a1670 0x7f00440a3b30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f005819a030 tx=0x7f0048009260 comp rx=0 tx=0).stop 2023-10-26T06:56:15.082 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.080+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058067830 msgr2=0x7f005819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.080+0000 7f005f39d700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058067830 0x7f005819e820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0054000c00 tx=0x7f0054005740 comp rx=0 tx=0).stop 2023-10-26T06:56:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.080+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 shutdown_connections 2023-10-26T06:56:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.080+0000 7f005f39d700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f00440a1670 0x7f00440a3b30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.083 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.080+0000 7f005f39d700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0058106510 0x7f00581988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.081+0000 7f005f39d700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0058067830 0x7f005819e820 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.081+0000 7f005f39d700 1 --2- 172.21.15.89:0/1089603866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00580681f0 0x7f005819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.081+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 >> 172.21.15.89:0/1089603866 conn(0x7f0058074ee0 msgr2=0x7f005810b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:15.084 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.081+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 shutdown_connections 2023-10-26T06:56:15.085 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.081+0000 7f005f39d700 1 -- 172.21.15.89:0/1089603866 wait complete. 2023-10-26T06:56:15.085 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:56:15.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:56:15.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.303+0000 7f39ba3ff700 1 -- 172.21.15.89:0/3419175918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b40734e0 msgr2=0x7f39b410ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.303+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/3419175918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b40734e0 0x7f39b410ca80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009990 tx=0x7f39a8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.303+0000 7f39ba3ff700 1 -- 172.21.15.89:0/3419175918 shutdown_connections 2023-10-26T06:56:15.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.303+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/3419175918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b410cfc0 0x7f39b410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.303+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/3419175918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39b4072bc0 0x7f39b4072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.303+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/3419175918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b40734e0 0x7f39b410ca80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.303+0000 7f39ba3ff700 1 -- 172.21.15.89:0/3419175918 >> 172.21.15.89:0/3419175918 conn(0x7f39b40fc1b0 msgr2=0x7f39b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.304+0000 7f39ba3ff700 1 -- 172.21.15.89:0/3419175918 shutdown_connections 2023-10-26T06:56:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.304+0000 7f39ba3ff700 1 -- 172.21.15.89:0/3419175918 wait complete. 2023-10-26T06:56:15.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 Processor -- start 2023-10-26T06:56:15.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 -- start start 2023-10-26T06:56:15.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b4072bc0 0x7f39b4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39b40734e0 0x7f39b4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b410cfc0 0x7f39b419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b419dd20 con 0x7f39b410cfc0 2023-10-26T06:56:15.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b419de90 con 0x7f39b4072bc0 2023-10-26T06:56:15.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39ba3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b419e050 con 0x7f39b40734e0 2023-10-26T06:56:15.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39b899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b410cfc0 0x7f39b419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39b899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b410cfc0 0x7f39b419d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47164/0 (socket says 172.21.15.89:47164) 2023-10-26T06:56:15.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39b899c700 1 -- 172.21.15.89:0/4021487957 learned_addr learned my addr 172.21.15.89:0/4021487957 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:15.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39b3fff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b4072bc0 0x7f39b4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.305+0000 7f39b37fe700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39b40734e0 0x7f39b4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b3fff700 1 -- 172.21.15.89:0/4021487957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39b40734e0 msgr2=0x7f39b4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b3fff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39b40734e0 0x7f39b4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b3fff700 1 -- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b410cfc0 msgr2=0x7f39b419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b3fff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b410cfc0 0x7f39b419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b3fff700 1 -- 172.21.15.89:0/4021487957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80095f0 con 0x7f39b4072bc0 2023-10-26T06:56:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b37fe700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39b40734e0 0x7f39b4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b899c700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b410cfc0 0x7f39b419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.306+0000 7f39b3fff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b4072bc0 0x7f39b4198e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39a400e9f0 tx=0x7f39a400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.308+0000 7f39b17fa700 1 -- 172.21.15.89:0/4021487957 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a400c9c0 con 0x7f39b4072bc0 2023-10-26T06:56:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.308+0000 7f39b17fa700 1 -- 172.21.15.89:0/4021487957 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a4010430 con 0x7f39b4072bc0 2023-10-26T06:56:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.308+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b419e330 con 0x7f39b4072bc0 2023-10-26T06:56:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.308+0000 7f39b17fa700 1 -- 172.21.15.89:0/4021487957 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4018680 con 0x7f39b4072bc0 2023-10-26T06:56:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.308+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b41a5b70 con 0x7f39b4072bc0 2023-10-26T06:56:15.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.310+0000 7f39b17fa700 1 -- 172.21.15.89:0/4021487957 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f39a400cb20 con 0x7f39b4072bc0 2023-10-26T06:56:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.312+0000 7f39b17fa700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f399c0a15f0 0x7f399c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.312+0000 7f39b17fa700 1 -- 172.21.15.89:0/4021487957 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f39a4014070 con 0x7f39b4072bc0 2023-10-26T06:56:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.312+0000 7f39cdcabb80 1 Processor -- start 2023-10-26T06:56:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.312+0000 7f39b37fe700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f399c0a15f0 0x7f399c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.312+0000 7f39cdcabb80 1 -- start start 2023-10-26T06:56:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b4715fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b4710090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4710a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b46836e0 con 0x55c8b4715be0 2023-10-26T06:56:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b4669c60 con 0x55c8b4716500 2023-10-26T06:56:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b4667580 con 0x55c8b47105d0 2023-10-26T06:56:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b4715fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b37fe700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f399c0a15f0 0x7f399c0a3ab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f39b419a350 tx=0x7f39a8005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b4715fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47172/0 (socket says 172.21.15.89:47172) 2023-10-26T06:56:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 -- 172.21.15.89:0/2351408971 learned_addr learned my addr 172.21.15.89:0/2351408971 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b899c700 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4710a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b37fe700 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b4710090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 -- 172.21.15.89:0/2351408971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 msgr2=0x55c8b4710a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4710a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 -- 172.21.15.89:0/2351408971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 msgr2=0x55c8b4710090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b4710090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b3fff700 1 -- 172.21.15.89:0/2351408971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b46203f0 con 0x55c8b4715be0 2023-10-26T06:56:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.313+0000 7f39b37fe700 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b4710090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.314+0000 7f39b3fff700 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b4715fc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f39a40a20b0 tx=0x7f39a4099ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b168732dbdc164 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.314+0000 7f399a7fc700 1 -- 172.21.15.89:0/2351408971 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a409e3a0 con 0x55c8b4715be0 2023-10-26T06:56:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.314+0000 7f399a7fc700 1 -- 172.21.15.89:0/2351408971 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a40a25e0 con 0x55c8b4715be0 2023-10-26T06:56:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.314+0000 7f399a7fc700 1 -- 172.21.15.89:0/2351408971 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a409a4b0 con 0x55c8b4715be0 2023-10-26T06:56:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2351408971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 msgr2=0x55c8b4715fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b4715fc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f39a40a20b0 tx=0x7f39a4099ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2351408971 shutdown_connections 2023-10-26T06:56:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b4715fc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4710a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2351408971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b4710090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2351408971 >> 172.21.15.89:0/2351408971 conn(0x55c8b4684720 msgr2=0x55c8b470ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2351408971 shutdown_connections 2023-10-26T06:56:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2351408971 wait complete. 2023-10-26T06:56:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 -- start start 2023-10-26T06:56:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.315+0000 7f39cdcabb80 1 Processor -- start 2023-10-26T06:56:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b4719880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4715be0 0x55c8b4719dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b4662e70 con 0x55c8b47105d0 2023-10-26T06:56:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b45ff710 con 0x55c8b4716500 2023-10-26T06:56:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b45ff3b0 con 0x55c8b4715be0 2023-10-26T06:56:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39cdcabb80 10 client.0 _ll_register_callbacks cb 0x55c8b4549a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b4719880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39b37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4715be0 0x55c8b4719dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b4719880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47188/0 (socket says 172.21.15.89:47188) 2023-10-26T06:56:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39b3fff700 1 -- 172.21.15.89:0/4032825208 learned_addr learned my addr 172.21.15.89:0/4032825208 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.316+0000 7f39b899c700 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39b37fe700 1 -- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 msgr2=0x55c8b471a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39b37fe700 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39b37fe700 1 -- 172.21.15.89:0/4032825208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 msgr2=0x55c8b4719880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39b37fe700 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b4719880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39b37fe700 1 -- 172.21.15.89:0/4032825208 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b41a5b70 con 0x55c8b4715be0 2023-10-26T06:56:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39b37fe700 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4715be0 0x55c8b4719dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39a800b2e0 tx=0x7f39a801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3796859c17e22f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39937fe700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:56:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39937fe700 1 -- 172.21.15.89:0/4032825208 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8024c80 con 0x55c8b4715be0 2023-10-26T06:56:15.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39937fe700 1 -- 172.21.15.89:0/4032825208 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f39a8024de0 con 0x55c8b4715be0 2023-10-26T06:56:15.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8b4667ef0 con 0x55c8b4715be0 2023-10-26T06:56:15.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39937fe700 1 -- 172.21.15.89:0/4032825208 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a8005050 con 0x55c8b4715be0 2023-10-26T06:56:15.340 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39cdcabb80 10 client.26602 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:15.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.317+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c8b448dde0 con 0x55c8b4715be0 2023-10-26T06:56:15.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39937fe700 1 -- 172.21.15.89:0/4032825208 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a803c020 con 0x55c8b4715be0 2023-10-26T06:56:15.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39937fe700 1 -- 172.21.15.89:0/4032825208 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39a8004cd0 con 0x55c8b4715be0 2023-10-26T06:56:15.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39cdcabb80 10 client.26602 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:15.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39cdcabb80 10 client.26602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:15.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39cdcabb80 20 client.26602 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:15.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b4729e30 0x55c8b472c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39cdcabb80 4 client.26602 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:15.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8b46df8b0 con 0x55c8b4729e30 2023-10-26T06:56:15.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.318+0000 7f39b3fff700 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b4729e30 0x55c8b472c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.319+0000 7f39937fe700 10 client.26602 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:15.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.319+0000 7f39b3fff700 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b4729e30 0x55c8b472c250 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a0a15120feccf6 server_cookie=c0eb9fe1d6eff20f in_seq=0 out_seq=0 2023-10-26T06:56:15.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.319+0000 7f39937fe700 1 -- 172.21.15.89:0/4032825208 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8b46df8b0 con 0x55c8b4729e30 2023-10-26T06:56:15.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.319+0000 7f39937fe700 10 client.26602 handle_command_reply: tid=0 2023-10-26T06:56:15.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.319+0000 7f39cdcabb80 1 client.26602 shutdown 2023-10-26T06:56:15.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4715be0 msgr2=0x55c8b4719dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4715be0 0x55c8b4719dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39a800b2e0 tx=0x7f39a801a050 comp rx=0 tx=0).stop 2023-10-26T06:56:15.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 shutdown_connections 2023-10-26T06:56:15.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b4719880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b4729e30 0x55c8b472c250 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/4032825208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4715be0 0x55c8b4719dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 >> 172.21.15.89:0/4032825208 conn(0x55c8b4684720 msgr2=0x55c8b470f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:15.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39937fe700 0 client.26602 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:15.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 shutdown_connections 2023-10-26T06:56:15.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 1 -- 172.21.15.89:0/4032825208 wait complete. 2023-10-26T06:56:15.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.320+0000 7f39cdcabb80 20 client.26602 trim_cache size 0 max 16384 2023-10-26T06:56:15.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.333+0000 7f39cdcabb80 1 Processor -- start 2023-10-26T06:56:15.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.333+0000 7f39cdcabb80 1 -- start start 2023-10-26T06:56:15.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b470fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4715be0 0x55c8b47136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4716500 0x55c8b4713bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b44e3920 con 0x55c8b47105d0 2023-10-26T06:56:15.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b471ae80 con 0x55c8b4715be0 2023-10-26T06:56:15.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b4717d60 con 0x55c8b4716500 2023-10-26T06:56:15.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4715be0 0x55c8b47136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4715be0 0x55c8b47136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42548/0 (socket says 172.21.15.89:42548) 2023-10-26T06:56:15.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 -- 172.21.15.89:0/2915970627 learned_addr learned my addr 172.21.15.89:0/2915970627 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:15.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b470fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b899c700 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4716500 0x55c8b4713bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 -- 172.21.15.89:0/2915970627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4716500 msgr2=0x55c8b4713bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4716500 0x55c8b4713bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 -- 172.21.15.89:0/2915970627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 msgr2=0x55c8b470fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b470fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.334+0000 7f39b37fe700 1 -- 172.21.15.89:0/2915970627 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b4476c60 con 0x55c8b4715be0 2023-10-26T06:56:15.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39b37fe700 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4715be0 0x55c8b47136b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009990 tx=0x7f39a800ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db38b026e08a51a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:15.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f399affd700 1 -- 172.21.15.89:0/2915970627 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8024c80 con 0x55c8b4715be0 2023-10-26T06:56:15.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f399affd700 1 -- 172.21.15.89:0/2915970627 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a803c070 con 0x55c8b4715be0 2023-10-26T06:56:15.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f399affd700 1 -- 172.21.15.89:0/2915970627 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a80053f0 con 0x55c8b4715be0 2023-10-26T06:56:15.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2915970627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4715be0 msgr2=0x55c8b47136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4715be0 0x55c8b47136b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009990 tx=0x7f39a800ffd0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2915970627 shutdown_connections 2023-10-26T06:56:15.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b47105d0 0x55c8b470fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b4716500 0x55c8b4713bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/2915970627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4715be0 0x55c8b47136b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.335+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2915970627 >> 172.21.15.89:0/2915970627 conn(0x55c8b4684720 msgr2=0x55c8b471a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:15.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.336+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2915970627 shutdown_connections 2023-10-26T06:56:15.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.336+0000 7f39cdcabb80 1 -- 172.21.15.89:0/2915970627 wait complete. 2023-10-26T06:56:15.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.336+0000 7f39cdcabb80 1 -- start start 2023-10-26T06:56:15.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.336+0000 7f39cdcabb80 1 Processor -- start 2023-10-26T06:56:15.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4720fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b471f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39cdcabb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b4720210 con 0x55c8b4715be0 2023-10-26T06:56:15.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b477dc90 con 0x55c8b4716500 2023-10-26T06:56:15.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39cdcabb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b477ddd0 con 0x55c8b47105d0 2023-10-26T06:56:15.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39cdcabb80 10 client.0 _ll_register_callbacks cb 0x55c8b4549a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:15.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4720fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4720fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:54386/0 (socket says 172.21.15.89:54386) 2023-10-26T06:56:15.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 -- 172.21.15.89:0/3466849094 learned_addr learned my addr 172.21.15.89:0/3466849094 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:15.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b899c700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b37fe700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b471f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 -- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 msgr2=0x55c8b471fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 -- 172.21.15.89:0/3466849094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 msgr2=0x55c8b471f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:15.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b471f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 -- 172.21.15.89:0/3466849094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8b448dde0 con 0x55c8b47105d0 2023-10-26T06:56:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b37fe700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b471f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39b3fff700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4720fb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39a4010ca0 tx=0x7f39a4099bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18505bb6262940d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:15.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.337+0000 7f39937fe700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:56:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39937fe700 1 -- 172.21.15.89:0/3466849094 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a409a020 con 0x55c8b47105d0 2023-10-26T06:56:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39937fe700 1 -- 172.21.15.89:0/3466849094 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f39a4010860 con 0x55c8b47105d0 2023-10-26T06:56:15.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39a40a25e0 con 0x55c8b47105d0 2023-10-26T06:56:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39937fe700 1 -- 172.21.15.89:0/3466849094 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a409e040 con 0x55c8b47105d0 2023-10-26T06:56:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 10 client.26608 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55c8b44b4250 con 0x55c8b47105d0 2023-10-26T06:56:15.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39937fe700 1 -- 172.21.15.89:0/3466849094 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a407d020 con 0x55c8b47105d0 2023-10-26T06:56:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39937fe700 1 -- 172.21.15.89:0/3466849094 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39a409a4b0 con 0x55c8b47105d0 2023-10-26T06:56:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 10 client.26608 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:15.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 10 client.26608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 20 client.26608 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b477f1b0 0x55c8b47815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 4 client.26608 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:56:15.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.338+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8b465be50 con 0x55c8b477f1b0 2023-10-26T06:56:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39b37fe700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b477f1b0 0x55c8b47815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39937fe700 10 client.26608 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:15.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39b37fe700 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b477f1b0 0x55c8b47815d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be83fc7c1c5d821a server_cookie=e6b475ab886484db in_seq=0 out_seq=0 2023-10-26T06:56:15.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39937fe700 1 -- 172.21.15.89:0/3466849094 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8b465be50 con 0x55c8b477f1b0 2023-10-26T06:56:15.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39937fe700 10 client.26608 handle_command_reply: tid=0 2023-10-26T06:56:15.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 client.26608 shutdown 2023-10-26T06:56:15.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 msgr2=0x55c8b4720fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4720fb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f39a4010ca0 tx=0x7f39a4099bd0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 shutdown_connections 2023-10-26T06:56:15.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55c8b477f1b0 0x55c8b47815d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8b4716500 0x55c8b471fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8b47105d0 0x55c8b4720fb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39937fe700 0 client.26608 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:15.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 --2- 172.21.15.89:0/3466849094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c8b4715be0 0x55c8b471f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 >> 172.21.15.89:0/3466849094 conn(0x55c8b4684720 msgr2=0x55c8b471a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:15.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 shutdown_connections 2023-10-26T06:56:15.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.339+0000 7f39cdcabb80 1 -- 172.21.15.89:0/3466849094 wait complete. 2023-10-26T06:56:15.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39cdcabb80 20 client.26608 trim_cache size 0 max 16384 2023-10-26T06:56:15.378 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:56:15.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f399c0a15f0 msgr2=0x7f399c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f399c0a15f0 0x7f399c0a3ab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f39b419a350 tx=0x7f39a8005fd0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b4072bc0 msgr2=0x7f39b4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:15.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b4072bc0 0x7f39b4198e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39a400e9f0 tx=0x7f39a400ed00 comp rx=0 tx=0).stop 2023-10-26T06:56:15.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 shutdown_connections 2023-10-26T06:56:15.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f399c0a15f0 0x7f399c0a3ab0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39b410cfc0 0x7f39b419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39b4072bc0 0x7f39b4198e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 --2- 172.21.15.89:0/4021487957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39b40734e0 0x7f39b4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 >> 172.21.15.89:0/4021487957 conn(0x7f39b40fc1b0 msgr2=0x7f39b40fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:15.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 shutdown_connections 2023-10-26T06:56:15.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:15.340+0000 7f39ba3ff700 1 -- 172.21.15.89:0/4021487957 wait complete. 2023-10-26T06:56:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:15 smithi089 ceph-mon[95031]: pgmap v597: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.8 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:15.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:15 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1089603866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:56:15.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:15 smithi192 ceph-mon[131661]: pgmap v597: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.8 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:15.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:15 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1089603866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:56:15.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:15 smithi135 ceph-mon[131403]: pgmap v597: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.8 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:15.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:15 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1089603866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:56:16.384 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:56:16.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:56:16.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 -- 172.21.15.89:0/665130662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50341005c0 msgr2=0x7f5034100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 --2- 172.21.15.89:0/665130662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50341005c0 0x7f5034100a40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50240098f0 tx=0x7f5024009c00 comp rx=0 tx=0).stop 2023-10-26T06:56:16.600 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 -- 172.21.15.89:0/665130662 shutdown_connections 2023-10-26T06:56:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 --2- 172.21.15.89:0/665130662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5034100f80 0x7f503410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 --2- 172.21.15.89:0/665130662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5034106660 0x7f5034106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 --2- 172.21.15.89:0/665130662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50341005c0 0x7f5034100a40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 -- 172.21.15.89:0/665130662 >> 172.21.15.89:0/665130662 conn(0x7f50340fc0f0 msgr2=0x7f50340fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 -- 172.21.15.89:0/665130662 shutdown_connections 2023-10-26T06:56:16.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.598+0000 7f503a99d700 1 -- 172.21.15.89:0/665130662 wait complete. 2023-10-26T06:56:16.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 Processor -- start 2023-10-26T06:56:16.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 -- start start 2023-10-26T06:56:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50341005c0 0x7f503419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5034100f80 0x7f503419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5034106660 0x7f5034198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5034199150 con 0x7f50341005c0 2023-10-26T06:56:16.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50341992c0 con 0x7f5034106660 2023-10-26T06:56:16.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.599+0000 7f503a99d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5034199480 con 0x7f5034100f80 2023-10-26T06:56:16.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50341005c0 0x7f503419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5034100f80 0x7f503419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5034100f80 0x7f503419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:54402/0 (socket says 172.21.15.89:54402) 2023-10-26T06:56:16.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50341005c0 0x7f503419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47222/0 (socket says 172.21.15.89:47222) 2023-10-26T06:56:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 -- 172.21.15.89:0/1398216084 learned_addr learned my addr 172.21.15.89:0/1398216084 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503a19c700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5034106660 0x7f5034198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 -- 172.21.15.89:0/1398216084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5034106660 msgr2=0x7f5034198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5034106660 0x7f5034198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 -- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50341005c0 msgr2=0x7f503419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50341005c0 0x7f503419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 -- 172.21.15.89:0/1398216084 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c009d00 con 0x7f5034100f80 2023-10-26T06:56:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503a19c700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5034106660 0x7f5034198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503999b700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50341005c0 0x7f503419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:16.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.600+0000 7f503919a700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5034100f80 0x7f503419eeb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5024005950 tx=0x7f502400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.601+0000 7f502affd700 1 -- 172.21.15.89:0/1398216084 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502401c070 con 0x7f5034100f80 2023-10-26T06:56:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.601+0000 7f502affd700 1 -- 172.21.15.89:0/1398216084 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502400bb20 con 0x7f5034100f80 2023-10-26T06:56:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.601+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5030009520 con 0x7f5034100f80 2023-10-26T06:56:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.601+0000 7f502affd700 1 -- 172.21.15.89:0/1398216084 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50240038e0 con 0x7f5034100f80 2023-10-26T06:56:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.601+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5024009520 con 0x7f5034100f80 2023-10-26T06:56:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.603+0000 7f502affd700 1 -- 172.21.15.89:0/1398216084 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5024020030 con 0x7f5034100f80 2023-10-26T06:56:16.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.605+0000 7f502affd700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f50200a1670 0x7f50200a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.605+0000 7f502affd700 1 -- 172.21.15.89:0/1398216084 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f50240cdf40 con 0x7f5034100f80 2023-10-26T06:56:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.605+0000 7f503999b700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f50200a1670 0x7f50200a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.605+0000 7f504f527b80 1 Processor -- start 2023-10-26T06:56:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f504f527b80 1 -- start start 2023-10-26T06:56:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f503999b700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f50200a1670 0x7f50200a3b30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fd0 tx=0x7f5030009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f504f527b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 0x56474f8af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f504f527b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8af700 0x56474f8a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f504f527b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f504f527b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f81c760 con 0x56474f8a98a0 2023-10-26T06:56:16.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f504f527b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f802ce0 con 0x56474f8aede0 2023-10-26T06:56:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f504f527b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f800600 con 0x56474f8af700 2023-10-26T06:56:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f503a19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f503a19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47234/0 (socket says 172.21.15.89:47234) 2023-10-26T06:56:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f503a19c700 1 -- 172.21.15.89:0/875300816 learned_addr learned my addr 172.21.15.89:0/875300816 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f503919a700 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8af700 0x56474f8a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.606+0000 7f503999b700 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 0x56474f8af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f503a19c700 1 -- 172.21.15.89:0/875300816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8af700 msgr2=0x56474f8a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f503a19c700 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8af700 0x56474f8a9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f503a19c700 1 -- 172.21.15.89:0/875300816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 msgr2=0x56474f8af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f503a19c700 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 0x56474f8af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f503a19c700 1 -- 172.21.15.89:0/875300816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56474f7b9470 con 0x56474f8a98a0 2023-10-26T06:56:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f503a19c700 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8a9d00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f502c0075f0 tx=0x7f502c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9198e37b529c5517 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f501ffff700 1 -- 172.21.15.89:0/875300816 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00e030 con 0x56474f8a98a0 2023-10-26T06:56:16.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f501ffff700 1 -- 172.21.15.89:0/875300816 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502c00f040 con 0x56474f8a98a0 2023-10-26T06:56:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.607+0000 7f501ffff700 1 -- 172.21.15.89:0/875300816 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c015bc0 con 0x56474f8a98a0 2023-10-26T06:56:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 -- 172.21.15.89:0/875300816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 msgr2=0x56474f8a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8a9d00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f502c0075f0 tx=0x7f502c0099e0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.627 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 -- 172.21.15.89:0/875300816 shutdown_connections 2023-10-26T06:56:16.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8a9d00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 0x56474f8af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.628 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 --2- 172.21.15.89:0/875300816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8af700 0x56474f8a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 -- 172.21.15.89:0/875300816 >> 172.21.15.89:0/875300816 conn(0x56474f81d7a0 msgr2=0x56474f8a3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 -- 172.21.15.89:0/875300816 shutdown_connections 2023-10-26T06:56:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.608+0000 7f504f527b80 1 -- 172.21.15.89:0/875300816 wait complete. 2023-10-26T06:56:16.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 -- start start 2023-10-26T06:56:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 Processor -- start 2023-10-26T06:56:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8aede0 0x56474f8b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f7fbef0 con 0x56474f8a98a0 2023-10-26T06:56:16.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f798790 con 0x56474f8af700 2023-10-26T06:56:16.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f798430 con 0x56474f8aede0 2023-10-26T06:56:16.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.609+0000 7f504f527b80 10 client.0 _ll_register_callbacks cb 0x56474f6e2b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:16.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47246/0 (socket says 172.21.15.89:47246) 2023-10-26T06:56:16.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 -- 172.21.15.89:0/1722321057 learned_addr learned my addr 172.21.15.89:0/1722321057 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503919a700 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8aede0 0x56474f8b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503a19c700 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 -- 172.21.15.89:0/1722321057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8aede0 msgr2=0x56474f8b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8aede0 0x56474f8b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 -- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 msgr2=0x56474f8b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 -- 172.21.15.89:0/1722321057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c009280 con 0x56474f8a98a0 2023-10-26T06:56:16.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f503999b700 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8b5620 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5030003810 tx=0x7f50300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c5351e3730ffc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.610+0000 7f501d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:56:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f501d7fa700 1 -- 172.21.15.89:0/1722321057 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503001f820 con 0x56474f8a98a0 2023-10-26T06:56:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f501d7fa700 1 -- 172.21.15.89:0/1722321057 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f503001f980 con 0x56474f8a98a0 2023-10-26T06:56:16.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56474f800f70 con 0x56474f8a98a0 2023-10-26T06:56:16.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f501d7fa700 1 -- 172.21.15.89:0/1722321057 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f503001fb40 con 0x56474f8a98a0 2023-10-26T06:56:16.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 10 client.16572 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:16.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56474f626e60 con 0x56474f8a98a0 2023-10-26T06:56:16.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f501d7fa700 1 -- 172.21.15.89:0/1722321057 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56474f800f70 con 0x56474f8a98a0 2023-10-26T06:56:16.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f501d7fa700 1 -- 172.21.15.89:0/1722321057 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f503001e410 con 0x56474f8a98a0 2023-10-26T06:56:16.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:16.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 20 client.16572 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f8c30f0 0x56474f8c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 4 client.16572 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:16.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.611+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56474f878930 con 0x56474f8c30f0 2023-10-26T06:56:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.612+0000 7f503919a700 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f8c30f0 0x56474f8c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.612+0000 7f501d7fa700 10 client.16572 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.612+0000 7f503919a700 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f8c30f0 0x56474f8c5510 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1274e7b12890795 server_cookie=bde414d31a990f9c in_seq=0 out_seq=0 2023-10-26T06:56:16.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f501d7fa700 1 -- 172.21.15.89:0/1722321057 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56474f878930 con 0x56474f8c30f0 2023-10-26T06:56:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f501d7fa700 10 client.16572 handle_command_reply: tid=0 2023-10-26T06:56:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 client.16572 shutdown 2023-10-26T06:56:16.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 msgr2=0x56474f8b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8b5620 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5030003810 tx=0x7f50300047c0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 shutdown_connections 2023-10-26T06:56:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8a98a0 0x56474f8b5620 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f8c30f0 0x56474f8c5510 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 --2- 172.21.15.89:0/1722321057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8aede0 0x56474f8b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f501d7fa700 0 client.16572 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 >> 172.21.15.89:0/1722321057 conn(0x56474f81d7a0 msgr2=0x56474f8a8350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 shutdown_connections 2023-10-26T06:56:16.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 1 -- 172.21.15.89:0/1722321057 wait complete. 2023-10-26T06:56:16.645 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.613+0000 7f504f527b80 20 client.16572 trim_cache size 0 max 16384 2023-10-26T06:56:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.630+0000 7f504f527b80 1 Processor -- start 2023-10-26T06:56:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f504f527b80 1 -- start start 2023-10-26T06:56:16.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f504f527b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f8ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f504f527b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 0x56474f8ac950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f504f527b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8af700 0x56474f8ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f504f527b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f67c9a0 con 0x56474f8af700 2023-10-26T06:56:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f504f527b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f8ba650 con 0x56474f8aede0 2023-10-26T06:56:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f504f527b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f8b3ee0 con 0x56474f8a98a0 2023-10-26T06:56:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f503a19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8af700 0x56474f8ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f503a19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8af700 0x56474f8ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47256/0 (socket says 172.21.15.89:47256) 2023-10-26T06:56:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.631+0000 7f503a19c700 1 -- 172.21.15.89:0/4029528711 learned_addr learned my addr 172.21.15.89:0/4029528711 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f503a19c700 1 -- 172.21.15.89:0/4029528711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 msgr2=0x56474f8ba110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:56:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f503a19c700 1 --2- 172.21.15.89:0/4029528711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f8ba110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f503a19c700 1 -- 172.21.15.89:0/4029528711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 msgr2=0x56474f8ac950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-26T06:56:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f503a19c700 1 --2- 172.21.15.89:0/4029528711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 0x56474f8ac950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f503a19c700 1 -- 172.21.15.89:0/4029528711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56474f8ca140 con 0x56474f8af700 2023-10-26T06:56:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f503a19c700 1 --2- 172.21.15.89:0/4029528711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8af700 0x56474f8ace90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f502c0075f0 tx=0x7f502c0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3c32144d09729d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f5028bff700 1 -- 172.21.15.89:0/4029528711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00e040 con 0x56474f8af700 2023-10-26T06:56:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f5028bff700 1 -- 172.21.15.89:0/4029528711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502c004c90 con 0x56474f8af700 2023-10-26T06:56:16.655 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.632+0000 7f5028bff700 1 -- 172.21.15.89:0/4029528711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c004030 con 0x56474f8af700 2023-10-26T06:56:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 -- 172.21.15.89:0/4029528711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8af700 msgr2=0x56474f8ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 --2- 172.21.15.89:0/4029528711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8af700 0x56474f8ace90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f502c0075f0 tx=0x7f502c0060e0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 -- 172.21.15.89:0/4029528711 shutdown_connections 2023-10-26T06:56:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 --2- 172.21.15.89:0/4029528711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8af700 0x56474f8ace90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 --2- 172.21.15.89:0/4029528711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f8ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 --2- 172.21.15.89:0/4029528711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8aede0 0x56474f8ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 -- 172.21.15.89:0/4029528711 >> 172.21.15.89:0/4029528711 conn(0x56474f81d7a0 msgr2=0x56474f8a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 -- 172.21.15.89:0/4029528711 shutdown_connections 2023-10-26T06:56:16.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 -- 172.21.15.89:0/4029528711 wait complete. 2023-10-26T06:56:16.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.633+0000 7f504f527b80 1 -- start start 2023-10-26T06:56:16.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 1 Processor -- start 2023-10-26T06:56:16.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f911850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8aede0 0x56474f911d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f8b9a60 con 0x56474f8aede0 2023-10-26T06:56:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f8b9bd0 con 0x56474f8af700 2023-10-26T06:56:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474f8b9d90 con 0x56474f8a98a0 2023-10-26T06:56:16.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f504f527b80 10 client.0 _ll_register_callbacks cb 0x56474f6e2b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f503999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f911850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f503919a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8aede0 0x56474f911d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f503999b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f911850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:54432/0 (socket says 172.21.15.89:54432) 2023-10-26T06:56:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f503999b700 1 -- 172.21.15.89:0/1460406308 learned_addr learned my addr 172.21.15.89:0/1460406308 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.634+0000 7f503a19c700 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f503919a700 1 -- 172.21.15.89:0/1460406308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 msgr2=0x56474f911850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.662 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f503919a700 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f911850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f503919a700 1 -- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 msgr2=0x56474f8b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f503919a700 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.663 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f503919a700 1 -- 172.21.15.89:0/1460406308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56474f7b9470 con 0x56474f8aede0 2023-10-26T06:56:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f503919a700 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8aede0 0x56474f911d90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f50240cb620 tx=0x7f5024005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245a2080a0afcf3b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f501d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:56:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f501d7fa700 1 -- 172.21.15.89:0/1460406308 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50240a3240 con 0x56474f8aede0 2023-10-26T06:56:16.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f501d7fa700 1 -- 172.21.15.89:0/1460406308 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f5024097650 con 0x56474f8aede0 2023-10-26T06:56:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f502c00f040 con 0x56474f8aede0 2023-10-26T06:56:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f501d7fa700 1 -- 172.21.15.89:0/1460406308 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5024097810 con 0x56474f8aede0 2023-10-26T06:56:16.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f504f527b80 10 client.16578 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x56474f60fce0 con 0x56474f8aede0 2023-10-26T06:56:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.635+0000 7f501d7fa700 1 -- 172.21.15.89:0/1460406308 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f502c00f040 con 0x56474f8aede0 2023-10-26T06:56:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f501d7fa700 1 -- 172.21.15.89:0/1460406308 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5024021380 con 0x56474f8aede0 2023-10-26T06:56:16.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f504f527b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f504f527b80 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f504f527b80 20 client.16578 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:16.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f504f527b80 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f919450 0x56474f91b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f504f527b80 4 client.16578 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:56:16.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56474f7f4ed0 con 0x56474f919450 2023-10-26T06:56:16.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.636+0000 7f503999b700 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f919450 0x56474f91b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.638+0000 7f503999b700 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f919450 0x56474f91b870 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217ccf780c3f0656 server_cookie=f9de80c9163be78a in_seq=0 out_seq=0 2023-10-26T06:56:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.638+0000 7f501d7fa700 10 client.16578 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.639+0000 7f501d7fa700 1 -- 172.21.15.89:0/1460406308 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56474f7f4ed0 con 0x56474f919450 2023-10-26T06:56:16.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.639+0000 7f501d7fa700 10 client.16578 handle_command_reply: tid=0 2023-10-26T06:56:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.639+0000 7f504f527b80 1 client.16578 shutdown 2023-10-26T06:56:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8aede0 msgr2=0x56474f911d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8aede0 0x56474f911d90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f50240cb620 tx=0x7f5024005570 comp rx=0 tx=0).stop 2023-10-26T06:56:16.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 shutdown_connections 2023-10-26T06:56:16.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x56474f919450 0x56474f91b870 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56474f8af700 0x56474f8b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56474f8a98a0 0x56474f911850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f501d7fa700 0 client.16578 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 --2- 172.21.15.89:0/1460406308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56474f8aede0 0x56474f911d90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 >> 172.21.15.89:0/1460406308 conn(0x56474f81d7a0 msgr2=0x56474f8ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 shutdown_connections 2023-10-26T06:56:16.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 1 -- 172.21.15.89:0/1460406308 wait complete. 2023-10-26T06:56:16.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.640+0000 7f504f527b80 20 client.16578 trim_cache size 0 max 16384 2023-10-26T06:56:16.673 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:56:16.674 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:56:16.674 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "9bd5c2cc-50f8-4841-837c-1586bbdfa54d", 2023-10-26T06:56:16.674 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:56:16.675 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:56:16.676 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f50200a1670 msgr2=0x7f50200a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f50200a1670 0x7f50200a3b30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fd0 tx=0x7f5030009260 comp rx=0 tx=0).stop 2023-10-26T06:56:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5034100f80 msgr2=0x7f503419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5034100f80 0x7f503419eeb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5024005950 tx=0x7f502400fa00 comp rx=0 tx=0).stop 2023-10-26T06:56:16.677 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 shutdown_connections 2023-10-26T06:56:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f50200a1670 0x7f50200a3b30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50341005c0 0x7f503419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.678 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5034106660 0x7f5034198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 --2- 172.21.15.89:0/1398216084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5034100f80 0x7f503419eeb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 >> 172.21.15.89:0/1398216084 conn(0x7f50340fc0f0 msgr2=0x7f50340fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.679 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 shutdown_connections 2023-10-26T06:56:16.680 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.641+0000 7f503a99d700 1 -- 172.21.15.89:0/1398216084 wait complete. 2023-10-26T06:56:16.680 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:56:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/3595659785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 msgr2=0x7f2df8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/3595659785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8100ba0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009990 tx=0x7f2dec009ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/3595659785 shutdown_connections 2023-10-26T06:56:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/3595659785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df81010e0 0x7f2df810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/3595659785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8106730 0x7f2df8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/3595659785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8100ba0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/3595659785 >> 172.21.15.89:0/3595659785 conn(0x7f2df80fc1b0 msgr2=0x7f2df80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.886 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/3595659785 shutdown_connections 2023-10-26T06:56:16.887 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.883+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/3595659785 wait complete. 2023-10-26T06:56:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.884+0000 7f2dfe1ff700 1 Processor -- start 2023-10-26T06:56:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.884+0000 7f2dfe1ff700 1 -- start start 2023-10-26T06:56:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2dfe1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2dfe1ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df81010e0 0x7f2df81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2dfe1ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df8106730 0x7f2df819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2dfe1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df819dca0 con 0x7f2df8100720 2023-10-26T06:56:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2dfe1ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df819de10 con 0x7f2df81010e0 2023-10-26T06:56:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2dfe1ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df819dfd0 con 0x7f2df8106730 2023-10-26T06:56:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47288/0 (socket says 172.21.15.89:47288) 2023-10-26T06:56:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 -- 172.21.15.89:0/2275069188 learned_addr learned my addr 172.21.15.89:0/2275069188 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df81010e0 0x7f2df81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df7fff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df8106730 0x7f2df819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 -- 172.21.15.89:0/2275069188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df8106730 msgr2=0x7f2df819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df8106730 0x7f2df819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 -- 172.21.15.89:0/2275069188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df81010e0 msgr2=0x7f2df81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df81010e0 0x7f2df81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.885+0000 7f2df77fe700 1 -- 172.21.15.89:0/2275069188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df0009c40 con 0x7f2df8100720 2023-10-26T06:56:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.886+0000 7f2df77fe700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8198d80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9a0 tx=0x7f2de800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.886+0000 7f2df4ff9700 1 -- 172.21.15.89:0/2275069188 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8009790 con 0x7f2df8100720 2023-10-26T06:56:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.886+0000 7f2df4ff9700 1 -- 172.21.15.89:0/2275069188 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de800ce80 con 0x7f2df8100720 2023-10-26T06:56:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.886+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dec0095f0 con 0x7f2df8100720 2023-10-26T06:56:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.886+0000 7f2df4ff9700 1 -- 172.21.15.89:0/2275069188 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8018680 con 0x7f2df8100720 2023-10-26T06:56:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.886+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df819e4a0 con 0x7f2df8100720 2023-10-26T06:56:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.889+0000 7f2df4ff9700 1 -- 172.21.15.89:0/2275069188 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2de8010430 con 0x7f2df8100720 2023-10-26T06:56:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.890+0000 7f2e11b7eb80 1 Processor -- start 2023-10-26T06:56:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.890+0000 7f2e11b7eb80 1 -- start start 2023-10-26T06:56:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df4ff9700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2de00a1670 0x7f2de00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959254be0 0x563959254fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x56395924f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56395924f5d0 0x56395924fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639591c26e0 con 0x563959255500 2023-10-26T06:56:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639591a8c60 con 0x563959254be0 2023-10-26T06:56:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639591a6580 con 0x56395924f5d0 2023-10-26T06:56:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df4ff9700 1 -- 172.21.15.89:0/2275069188 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f2de8014070 con 0x7f2df8100720 2023-10-26T06:56:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959254be0 0x563959254fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959254be0 0x563959254fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42652/0 (socket says 172.21.15.89:42652) 2023-10-26T06:56:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df7fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56395924f5d0 0x56395924fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df77fe700 1 -- 172.21.15.89:0/2643531736 learned_addr learned my addr 172.21.15.89:0/2643531736 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x56395924f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.891+0000 7f2df7fff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2de00a1670 0x7f2de00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df77fe700 1 -- 172.21.15.89:0/2643531736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56395924f5d0 msgr2=0x56395924fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df77fe700 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56395924f5d0 0x56395924fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df77fe700 1 -- 172.21.15.89:0/2643531736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 msgr2=0x56395924f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df77fe700 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x56395924f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df77fe700 1 -- 172.21.15.89:0/2643531736 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56395915f3f0 con 0x563959254be0 2023-10-26T06:56:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x56395924f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df77fe700 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959254be0 0x563959254fc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2de8096c40 tx=0x7f2de8099c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bccb1bce81f38409 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2df7fff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2de00a1670 0x7f2de00a3b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009550 tx=0x7f2df000b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2dddffb700 1 -- 172.21.15.89:0/2643531736 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de809a1d0 con 0x563959254be0 2023-10-26T06:56:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2dddffb700 1 -- 172.21.15.89:0/2643531736 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de809e320 con 0x563959254be0 2023-10-26T06:56:16.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.892+0000 7f2dddffb700 1 -- 172.21.15.89:0/2643531736 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de80cabc0 con 0x563959254be0 2023-10-26T06:56:16.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/2643531736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959254be0 msgr2=0x563959254fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959254be0 0x563959254fc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2de8096c40 tx=0x7f2de8099c40 comp rx=0 tx=0).stop 2023-10-26T06:56:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/2643531736 shutdown_connections 2023-10-26T06:56:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56395924f5d0 0x56395924fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959254be0 0x563959254fc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/2643531736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x56395924f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/2643531736 >> 172.21.15.89:0/2643531736 conn(0x5639591c3720 msgr2=0x563959249b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/2643531736 shutdown_connections 2023-10-26T06:56:16.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/2643531736 wait complete. 2023-10-26T06:56:16.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.893+0000 7f2e11b7eb80 1 -- start start 2023-10-26T06:56:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 1 Processor -- start 2023-10-26T06:56:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 0x563959258780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x563959258db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639591a1e70 con 0x563959255500 2023-10-26T06:56:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395913e710 con 0x56395924f5d0 2023-10-26T06:56:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395913e3b0 con 0x563959254be0 2023-10-26T06:56:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2e11b7eb80 10 client.0 _ll_register_callbacks cb 0x563959088a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42660/0 (socket says 172.21.15.89:42660) 2023-10-26T06:56:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.894+0000 7f2df77fe700 1 -- 172.21.15.89:0/1721029856 learned_addr learned my addr 172.21.15.89:0/1721029856 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df7fff700 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x563959258db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 0x563959258780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df77fe700 1 -- 172.21.15.89:0/1721029856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 msgr2=0x563959258780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.915 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df77fe700 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 0x563959258780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df77fe700 1 -- 172.21.15.89:0/1721029856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 msgr2=0x563959258db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df77fe700 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x563959258db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df77fe700 1 -- 172.21.15.89:0/1721029856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df819e4a0 con 0x56395924f5d0 2023-10-26T06:56:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2df77fe700 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925b490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2de80a2030 tx=0x7f2de80ca920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa487ba4506ca1d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2dd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:56:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.895+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/1721029856 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de809a630 con 0x56395924f5d0 2023-10-26T06:56:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639591a6ef0 con 0x56395924f5d0 2023-10-26T06:56:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/1721029856 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f2de809a1d0 con 0x56395924f5d0 2023-10-26T06:56:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/1721029856 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de80996c0 con 0x56395924f5d0 2023-10-26T06:56:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 10 client.26642 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563958fccde0 con 0x56395924f5d0 2023-10-26T06:56:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/1721029856 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639591a6ef0 con 0x56395924f5d0 2023-10-26T06:56:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/1721029856 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2de8095fc0 con 0x56395924f5d0 2023-10-26T06:56:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 10 client.26642 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 20 client.26642 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563959268ab0 0x56395926aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 4 client.26642 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:16.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56395921e8b0 con 0x563959268ab0 2023-10-26T06:56:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.896+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563959268ab0 0x56395926aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.897+0000 7f2dd6ffd700 10 client.26642 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.897+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563959268ab0 0x56395926aea0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73c1f921b8b8eef server_cookie=4c90db2b4207b090 in_seq=0 out_seq=0 2023-10-26T06:56:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/1721029856 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56395921e8b0 con 0x563959268ab0 2023-10-26T06:56:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2dd6ffd700 10 client.26642 handle_command_reply: tid=0 2023-10-26T06:56:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 client.26642 shutdown 2023-10-26T06:56:16.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 msgr2=0x56395925b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925b490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2de80a2030 tx=0x7f2de80ca920 comp rx=0 tx=0).stop 2023-10-26T06:56:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 shutdown_connections 2023-10-26T06:56:16.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959255500 0x563959258db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x563959268ab0 0x56395926aea0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925b490 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/1721029856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 0x563959258780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2dd6ffd700 0 client.26642 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 >> 172.21.15.89:0/1721029856 conn(0x5639591c3720 msgr2=0x56395924e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 shutdown_connections 2023-10-26T06:56:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/1721029856 wait complete. 2023-10-26T06:56:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.898+0000 7f2e11b7eb80 20 client.26642 trim_cache size 0 max 16384 2023-10-26T06:56:16.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.915+0000 7f2e11b7eb80 1 Processor -- start 2023-10-26T06:56:16.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.915+0000 7f2e11b7eb80 1 -- start start 2023-10-26T06:56:16.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.915+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56395924f5d0 0x56395925feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.915+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 0x56395924e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.915+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959255500 0x563959257050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563959022920 con 0x56395924f5d0 2023-10-26T06:56:16.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395924e700 con 0x563959255500 2023-10-26T06:56:16.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395924f490 con 0x563959254be0 2023-10-26T06:56:16.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56395924f5d0 0x56395925feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56395924f5d0 0x56395925feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47320/0 (socket says 172.21.15.89:47320) 2023-10-26T06:56:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 -- 172.21.15.89:0/3303888777 learned_addr learned my addr 172.21.15.89:0/3303888777 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 -- 172.21.15.89:0/3303888777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 msgr2=0x56395924e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 --2- 172.21.15.89:0/3303888777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 0x56395924e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 -- 172.21.15.89:0/3303888777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959255500 msgr2=0x563959257050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 --2- 172.21.15.89:0/3303888777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959255500 0x563959257050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 -- 172.21.15.89:0/3303888777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563958fb5c60 con 0x56395924f5d0 2023-10-26T06:56:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.916+0000 7f2df77fe700 1 --2- 172.21.15.89:0/3303888777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56395924f5d0 0x56395925feb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2de8000c00 tx=0x7f2de8010920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafb8af624d5d047 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2dde7fc700 1 -- 172.21.15.89:0/3303888777 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de809a1d0 con 0x56395924f5d0 2023-10-26T06:56:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2dde7fc700 1 -- 172.21.15.89:0/3303888777 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de80a2560 con 0x56395924f5d0 2023-10-26T06:56:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2dde7fc700 1 -- 172.21.15.89:0/3303888777 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8099920 con 0x56395924f5d0 2023-10-26T06:56:16.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3303888777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56395924f5d0 msgr2=0x56395925feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3303888777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56395924f5d0 0x56395925feb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2de8000c00 tx=0x7f2de8010920 comp rx=0 tx=0).stop 2023-10-26T06:56:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3303888777 shutdown_connections 2023-10-26T06:56:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3303888777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56395924f5d0 0x56395925feb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3303888777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563959255500 0x563959257050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3303888777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959254be0 0x56395924e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3303888777 >> 172.21.15.89:0/3303888777 conn(0x5639591c3720 msgr2=0x56395924f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3303888777 shutdown_connections 2023-10-26T06:56:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.917+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3303888777 wait complete. 2023-10-26T06:56:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 -- start start 2023-10-26T06:56:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 Processor -- start 2023-10-26T06:56:16.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959254be0 0x563959260050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959255500 0x56395925e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395925e780 con 0x563959254be0 2023-10-26T06:56:16.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395925e8f0 con 0x56395924f5d0 2023-10-26T06:56:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.918+0000 7f2e11b7eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395925eab0 con 0x563959255500 2023-10-26T06:56:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2e11b7eb80 10 client.0 _ll_register_callbacks cb 0x563959088a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:16.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959254be0 0x563959260050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:42674/0 (socket says 172.21.15.89:42674) 2023-10-26T06:56:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 -- 172.21.15.89:0/3237459284 learned_addr learned my addr 172.21.15.89:0/3237459284 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:16.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df7fff700 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959255500 0x56395925e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 -- 172.21.15.89:0/3237459284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959255500 msgr2=0x56395925e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959255500 0x56395925e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 -- 172.21.15.89:0/3237459284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959254be0 msgr2=0x563959260050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959254be0 0x563959260050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 -- 172.21.15.89:0/3237459284 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563958fccde0 con 0x56395924f5d0 2023-10-26T06:56:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959254be0 0x563959260050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:16.943 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2df77fe700 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925fb10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2de8099ca0 tx=0x7f2de8011f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecf5b6c2bb9020eb server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2dd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:56:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.919+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/3237459284 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de809a1d0 con 0x56395924f5d0 2023-10-26T06:56:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/3237459284 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f2de809a430 con 0x56395924f5d0 2023-10-26T06:56:16.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de809e320 con 0x56395924f5d0 2023-10-26T06:56:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/3237459284 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de809a5f0 con 0x56395924f5d0 2023-10-26T06:56:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 10 client.26645 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:16.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563958ff3250 con 0x56395924f5d0 2023-10-26T06:56:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/3237459284 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de809e320 con 0x56395924f5d0 2023-10-26T06:56:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/3237459284 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2de80992c0 con 0x56395924f5d0 2023-10-26T06:56:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 10 client.26645 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:16.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 10 client.26645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 20 client.26645 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5639592bd850 0x5639592bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 4 client.26645 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:56:16.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.920+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56395919ae50 con 0x5639592bd850 2023-10-26T06:56:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5639592bd850 0x5639592bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2df6ffd700 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5639592bd850 0x5639592bfc70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40084bd2c15cb0c server_cookie=bea4297d1cb20785 in_seq=0 out_seq=0 2023-10-26T06:56:16.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2dd6ffd700 10 client.26645 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2dd6ffd700 1 -- 172.21.15.89:0/3237459284 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56395919ae50 con 0x5639592bd850 2023-10-26T06:56:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2dd6ffd700 10 client.26645 handle_command_reply: tid=0 2023-10-26T06:56:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2e11b7eb80 1 client.26645 shutdown 2023-10-26T06:56:16.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 msgr2=0x56395925fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925fb10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2de8099ca0 tx=0x7f2de8011f60 comp rx=0 tx=0).stop 2023-10-26T06:56:16.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 shutdown_connections 2023-10-26T06:56:16.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.921+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x5639592bd850 0x5639592bfc70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563959255500 0x56395925e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56395924f5d0 0x56395925fb10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2e11b7eb80 1 --2- 172.21.15.89:0/3237459284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563959254be0 0x563959260050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dd6ffd700 0 client.26645 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:16.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 >> 172.21.15.89:0/3237459284 conn(0x5639591c3720 msgr2=0x563959249810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 shutdown_connections 2023-10-26T06:56:16.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2e11b7eb80 1 -- 172.21.15.89:0/3237459284 wait complete. 2023-10-26T06:56:16.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2e11b7eb80 20 client.26645 trim_cache size 0 max 16384 2023-10-26T06:56:16.953 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:56:16.953 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:56:16.953 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:56:16.954 INFO:teuthology.orchestra.run.smithi089.stdout: "9bd5c2cc-50f8-4841-837c-1586bbdfa54d": { 2023-10-26T06:56:16.954 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:56:16.954 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "9bd5c2cc-50f8-4841-837c-1586bbdfa54d", 2023-10-26T06:56:16.955 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:56:16.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:56:16.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:56:16.955 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:56:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2de00a1670 msgr2=0x7f2de00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2de00a1670 0x7f2de00a3b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009550 tx=0x7f2df000b040 comp rx=0 tx=0).stop 2023-10-26T06:56:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 msgr2=0x7f2df8198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:16.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8198d80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9a0 tx=0x7f2de800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 shutdown_connections 2023-10-26T06:56:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2de00a1670 0x7f2de00a3b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8100720 0x7f2df8198d80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df8106730 0x7f2df819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 --2- 172.21.15.89:0/2275069188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df81010e0 0x7f2df81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.922+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 >> 172.21.15.89:0/2275069188 conn(0x7f2df80fc1b0 msgr2=0x7f2df810be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.923+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 shutdown_connections 2023-10-26T06:56:16.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:16.923+0000 7f2dfe1ff700 1 -- 172.21.15.89:0/2275069188 wait complete. 2023-10-26T06:56:16.960 INFO:tasks.cephfs.filesystem:scrub status for tag:9bd5c2cc-50f8-4841-837c-1586bbdfa54d - {'path': '/', 'tag': '9bd5c2cc-50f8-4841-837c-1586bbdfa54d', 'options': 'recursive,force'} 2023-10-26T06:56:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:17 smithi089 ceph-mon[95031]: pgmap v598: 97 pgs: 16 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 46 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:17.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:17 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:56:17.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:17 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:17.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:17 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:56:17.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:17 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:17.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:17 smithi192 ceph-mon[131661]: pgmap v598: 97 pgs: 16 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 46 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:17.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:17 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:56:17.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:17 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:17.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:17 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:56:17.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:17 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:17.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:17 smithi135 ceph-mon[131403]: pgmap v598: 97 pgs: 16 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 46 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:17 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:56:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:17 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:17.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:17 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:56:17.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:17 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:19.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:19 smithi089 ceph-mon[95031]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:19.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:19 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:56:19.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:19 smithi192 ceph-mon[131661]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:19.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:19 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:56:19.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:19 smithi135 ceph-mon[131403]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:56:19.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:19 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:56:21.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:21 smithi192 ceph-mon[131661]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 194 B/s wr, 2 op/s 2023-10-26T06:56:21.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:21 smithi192 ceph-mon[131661]: osdmap e94: 12 total, 12 up, 12 in 2023-10-26T06:56:21.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:21 smithi135 ceph-mon[131403]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 194 B/s wr, 2 op/s 2023-10-26T06:56:21.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:21 smithi135 ceph-mon[131403]: osdmap e94: 12 total, 12 up, 12 in 2023-10-26T06:56:21.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:21 smithi089 ceph-mon[95031]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 194 B/s wr, 2 op/s 2023-10-26T06:56:21.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:21 smithi089 ceph-mon[95031]: osdmap e94: 12 total, 12 up, 12 in 2023-10-26T06:56:23.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:23 smithi192 ceph-mon[131661]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:56:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:23 smithi135 ceph-mon[131403]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:56:23.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:23 smithi089 ceph-mon[95031]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:56:25.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:25 smithi192 ceph-mon[131661]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:56:25.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:25 smithi135 ceph-mon[131403]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:56:25.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:25 smithi089 ceph-mon[95031]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:56:27.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:27 smithi192 ceph-mon[131661]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:27.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:27 smithi135 ceph-mon[131403]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:27.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:27 smithi089 ceph-mon[95031]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:29.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:29 smithi192 ceph-mon[131661]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:29 smithi135 ceph-mon[131403]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:29 smithi089 ceph-mon[95031]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:31.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:31 smithi192 ceph-mon[131661]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2023-10-26T06:56:31.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:31 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:56:31.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:31 smithi135 ceph-mon[131403]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2023-10-26T06:56:31.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:31 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:56:31.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:31 smithi089 ceph-mon[95031]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2023-10-26T06:56:31.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:31 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:56:32.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:56:32.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:56:32.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:56:32.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:56:32.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:56:32.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:56:32.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:56:32.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:56:32.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:56:33.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:33 smithi192 ceph-mon[131661]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:56:33.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:33 smithi135 ceph-mon[131403]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:56:34.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:33 smithi089 ceph-mon[95031]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2023-10-26T06:56:35.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:35 smithi192 ceph-mon[131661]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:35.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:35 smithi135 ceph-mon[131403]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:36.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:35 smithi089 ceph-mon[95031]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:37.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:37 smithi192 ceph-mon[131661]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:56:37.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:37 smithi135 ceph-mon[131403]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:56:38.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:37 smithi089 ceph-mon[95031]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:56:39.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:39 smithi192 ceph-mon[131661]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:39.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:39 smithi135 ceph-mon[131403]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:40.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:39 smithi089 ceph-mon[95031]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:41.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:41 smithi192 ceph-mon[131661]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:41.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:41 smithi135 ceph-mon[131403]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:42.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:41 smithi089 ceph-mon[95031]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:43.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:43 smithi192 ceph-mon[131661]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:43.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:43 smithi135 ceph-mon[131403]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:44.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:43 smithi089 ceph-mon[95031]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:45.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:45 smithi192 ceph-mon[131661]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:45.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:45 smithi135 ceph-mon[131403]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:46.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:45 smithi089 ceph-mon[95031]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:46.962 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:56:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 -- 172.21.15.89:0/2697225898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb41005d0 msgr2=0x7f2fb4100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.172 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/2697225898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb41005d0 0x7f2fb4100a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009990 tx=0x7f2fa8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 -- 172.21.15.89:0/2697225898 shutdown_connections 2023-10-26T06:56:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/2697225898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb4100f90 0x7f2fb410d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/2697225898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb4106690 0x7f2fb4106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.173 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/2697225898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb41005d0 0x7f2fb4100a50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 -- 172.21.15.89:0/2697225898 >> 172.21.15.89:0/2697225898 conn(0x7f2fb40fc0c0 msgr2=0x7f2fb40fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 -- 172.21.15.89:0/2697225898 shutdown_connections 2023-10-26T06:56:47.174 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.170+0000 7f2fb89fe700 1 -- 172.21.15.89:0/2697225898 wait complete. 2023-10-26T06:56:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.171+0000 7f2fb89fe700 1 Processor -- start 2023-10-26T06:56:47.176 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.171+0000 7f2fb89fe700 1 -- start start 2023-10-26T06:56:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb89fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb41005d0 0x7f2fb419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb89fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100f90 0x7f2fb419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb89fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb4106690 0x7f2fb4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.177 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb89fe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4199110 con 0x7f2fb41005d0 2023-10-26T06:56:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb89fe700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4199280 con 0x7f2fb4106690 2023-10-26T06:56:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb89fe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4199440 con 0x7f2fb4100f90 2023-10-26T06:56:47.178 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb41005d0 0x7f2fb419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100f90 0x7f2fb419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100f90 0x7f2fb419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:37956/0 (socket says 172.21.15.89:37956) 2023-10-26T06:56:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb41005d0 0x7f2fb419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48616/0 (socket says 172.21.15.89:48616) 2023-10-26T06:56:47.179 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 -- 172.21.15.89:0/3535263293 learned_addr learned my addr 172.21.15.89:0/3535263293 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb359e700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb4106690 0x7f2fb4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 -- 172.21.15.89:0/3535263293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb4106690 msgr2=0x7f2fb4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb4106690 0x7f2fb4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.180 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 -- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb41005d0 msgr2=0x7f2fb419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb41005d0 0x7f2fb419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.172+0000 7f2fb259c700 1 -- 172.21.15.89:0/3535263293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa80095f0 con 0x7f2fb4100f90 2023-10-26T06:56:47.181 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.173+0000 7f2fb259c700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100f90 0x7f2fb419eec0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005910 tx=0x7f2fa8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.173+0000 7f2f9bfff700 1 -- 172.21.15.89:0/3535263293 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa801d070 con 0x7f2fb4100f90 2023-10-26T06:56:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.173+0000 7f2f9bfff700 1 -- 172.21.15.89:0/3535263293 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa8022470 con 0x7f2fb4100f90 2023-10-26T06:56:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.173+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb41996c0 con 0x7f2fb4100f90 2023-10-26T06:56:47.182 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.173+0000 7f2f9bfff700 1 -- 172.21.15.89:0/3535263293 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8017650 con 0x7f2fb4100f90 2023-10-26T06:56:47.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.173+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb41a5970 con 0x7f2fb4100f90 2023-10-26T06:56:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.176+0000 7f2f9bfff700 1 -- 172.21.15.89:0/3535263293 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2fa80225e0 con 0x7f2fb4100f90 2023-10-26T06:56:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.178+0000 7f2fccbb5b80 1 Processor -- start 2023-10-26T06:56:47.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.178+0000 7f2fccbb5b80 1 -- start start 2023-10-26T06:56:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.178+0000 7f2f9bfff700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f9c0a1670 0x7f2f9c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e49a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb359e700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f9c0a1670 0x7f2f9c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2f9bfff700 1 -- 172.21.15.89:0/3535263293 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2fa80ce400 con 0x7f2fb4100f90 2023-10-26T06:56:47.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e494c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e4076e0 con 0x55608e49a680 2023-10-26T06:56:47.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e3edc60 con 0x55608e499d60 2023-10-26T06:56:47.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e3eb580 con 0x55608e494820 2023-10-26T06:56:47.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e49a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e49a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34744/0 (socket says 172.21.15.89:34744) 2023-10-26T06:56:47.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/2575938123 learned_addr learned my addr 172.21.15.89:0/2575938123 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb359e700 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e494c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb259c700 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/2575938123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 msgr2=0x55608e494c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.194 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e494c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/2575938123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 msgr2=0x55608e4942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.195 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.179+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/2575938123 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55608e3a43f0 con 0x55608e499d60 2023-10-26T06:56:47.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.180+0000 7f2fb359e700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f9c0a1670 0x7f2f9c0a3b30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009d10 tx=0x7f2fac004320 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.180+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e49a140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009d00 tx=0x7f2fa4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fa5194de1e3dff server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.180+0000 7f2f997fa700 1 -- 172.21.15.89:0/2575938123 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4019630 con 0x55608e499d60 2023-10-26T06:56:47.196 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.180+0000 7f2f997fa700 1 -- 172.21.15.89:0/2575938123 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa4019c50 con 0x55608e499d60 2023-10-26T06:56:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.180+0000 7f2f997fa700 1 -- 172.21.15.89:0/2575938123 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400eb30 con 0x55608e499d60 2023-10-26T06:56:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.180+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2575938123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 msgr2=0x55608e49a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.197 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.180+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e49a140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009d00 tx=0x7f2fa4011720 comp rx=0 tx=0).stop 2023-10-26T06:56:47.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2575938123 shutdown_connections 2023-10-26T06:56:47.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e494c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e49a140 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2575938123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2575938123 >> 172.21.15.89:0/2575938123 conn(0x55608e408720 msgr2=0x55608e48ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2575938123 shutdown_connections 2023-10-26T06:56:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2575938123 wait complete. 2023-10-26T06:56:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 -- start start 2023-10-26T06:56:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.181+0000 7f2fccbb5b80 1 Processor -- start 2023-10-26T06:56:47.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e499d60 0x55608e49d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e49dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.201 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e3e6e70 con 0x55608e49a680 2023-10-26T06:56:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e383710 con 0x55608e494820 2023-10-26T06:56:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e3833b0 con 0x55608e499d60 2023-10-26T06:56:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fccbb5b80 10 client.0 _ll_register_callbacks cb 0x55608e2cda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:47.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fb259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e499d60 0x55608e49d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fb259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e499d60 0x55608e49d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:37980/0 (socket says 172.21.15.89:37980) 2023-10-26T06:56:47.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fb359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e49dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fb259c700 1 -- 172.21.15.89:0/2254437577 learned_addr learned my addr 172.21.15.89:0/2254437577 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.182+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fb359e700 1 -- 172.21.15.89:0/2254437577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e499d60 msgr2=0x55608e49d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fb359e700 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e499d60 0x55608e49d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fb359e700 1 -- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 msgr2=0x55608e4a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fb359e700 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fb359e700 1 -- 172.21.15.89:0/2254437577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa4009960 con 0x55608e49a680 2023-10-26T06:56:47.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:47.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fb359e700 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e49dec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009550 tx=0x7f2fac004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c00b6abb5af9291 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2f96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:56:47.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2f96ff5700 1 -- 172.21.15.89:0/2254437577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac01a670 con 0x55608e49a680 2023-10-26T06:56:47.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55608e3ebef0 con 0x55608e49a680 2023-10-26T06:56:47.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2f96ff5700 1 -- 172.21.15.89:0/2254437577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2fac01a7d0 con 0x55608e49a680 2023-10-26T06:56:47.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2f96ff5700 1 -- 172.21.15.89:0/2254437577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fac01a990 con 0x55608e49a680 2023-10-26T06:56:47.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fccbb5b80 10 client.16608 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:47.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.183+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55608e211de0 con 0x55608e49a680 2023-10-26T06:56:47.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2f96ff5700 1 -- 172.21.15.89:0/2254437577 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fac032020 con 0x55608e49a680 2023-10-26T06:56:47.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2f96ff5700 1 -- 172.21.15.89:0/2254437577 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fac025c60 con 0x55608e49a680 2023-10-26T06:56:47.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2fccbb5b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:47.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2fccbb5b80 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:47.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2fccbb5b80 20 client.16608 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:47.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e4adfa0 0x55608e4b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2fccbb5b80 4 client.16608 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55608e4638b0 con 0x55608e4adfa0 2023-10-26T06:56:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.184+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e4adfa0 0x55608e4b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.185+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e4adfa0 0x55608e4b03c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ab8b96da0867a7 server_cookie=70a96007bd1b5479 in_seq=0 out_seq=0 2023-10-26T06:56:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.185+0000 7f2f96ff5700 10 client.16608 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.185+0000 7f2f96ff5700 1 -- 172.21.15.89:0/2254437577 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55608e4638b0 con 0x55608e4adfa0 2023-10-26T06:56:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.185+0000 7f2f96ff5700 10 client.16608 handle_command_reply: tid=0 2023-10-26T06:56:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.185+0000 7f2fccbb5b80 1 client.16608 shutdown 2023-10-26T06:56:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 msgr2=0x55608e49dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e49dec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009550 tx=0x7f2fac004b40 comp rx=0 tx=0).stop 2023-10-26T06:56:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 shutdown_connections 2023-10-26T06:56:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e49dec0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e4adfa0 0x55608e4b03c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/2254437577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e499d60 0x55608e49d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2f96ff5700 0 client.16608 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:47.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 >> 172.21.15.89:0/2254437577 conn(0x55608e408720 msgr2=0x55608e4932d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 shutdown_connections 2023-10-26T06:56:47.217 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/2254437577 wait complete. 2023-10-26T06:56:47.218 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.186+0000 7f2fccbb5b80 20 client.16608 trim_cache size 0 max 16384 2023-10-26T06:56:47.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 Processor -- start 2023-10-26T06:56:47.223 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 -- start start 2023-10-26T06:56:47.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e499d60 0x55608e4a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e49a680 0x55608e4978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.224 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e267920 con 0x55608e499d60 2023-10-26T06:56:47.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e487220 con 0x55608e494820 2023-10-26T06:56:47.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e48b460 con 0x55608e49a680 2023-10-26T06:56:47.225 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34760/0 (socket says 172.21.15.89:34760) 2023-10-26T06:56:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/341880525 learned_addr learned my addr 172.21.15.89:0/341880525 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fb259c700 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e499d60 0x55608e4a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.227 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.204+0000 7f2fb359e700 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e49a680 0x55608e4978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/341880525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e49a680 msgr2=0x55608e4978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.228 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e49a680 0x55608e4978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/341880525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e499d60 msgr2=0x55608e4a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.229 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e499d60 0x55608e4a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2fb2d9d700 1 -- 172.21.15.89:0/341880525 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55608e4b4ff0 con 0x55608e494820 2023-10-26T06:56:47.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a5060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4014040 tx=0x7f2fa40092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b345c13004082cf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2f99ffb700 1 -- 172.21.15.89:0/341880525 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400e3b0 con 0x55608e494820 2023-10-26T06:56:47.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2f99ffb700 1 -- 172.21.15.89:0/341880525 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa4028ce0 con 0x55608e494820 2023-10-26T06:56:47.231 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.205+0000 7f2f99ffb700 1 -- 172.21.15.89:0/341880525 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4019a10 con 0x55608e494820 2023-10-26T06:56:47.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/341880525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 msgr2=0x55608e4a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a5060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4014040 tx=0x7f2fa40092d0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/341880525 shutdown_connections 2023-10-26T06:56:47.233 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e49a680 0x55608e4978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e494820 0x55608e4a5060 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/341880525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e499d60 0x55608e4a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/341880525 >> 172.21.15.89:0/341880525 conn(0x55608e408720 msgr2=0x55608e493a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/341880525 shutdown_connections 2023-10-26T06:56:47.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.206+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/341880525 wait complete. 2023-10-26T06:56:47.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 -- start start 2023-10-26T06:56:47.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 Processor -- start 2023-10-26T06:56:47.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e4a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e4a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e501de0 con 0x55608e49a680 2023-10-26T06:56:47.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e501f20 con 0x55608e499d60 2023-10-26T06:56:47.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608e502060 con 0x55608e494820 2023-10-26T06:56:47.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fccbb5b80 10 client.0 _ll_register_callbacks cb 0x55608e2cda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:47.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fb359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.238 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.207+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e4a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb259c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e4a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48648/0 (socket says 172.21.15.89:48648) 2023-10-26T06:56:47.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 -- 172.21.15.89:0/3053642136 learned_addr learned my addr 172.21.15.89:0/3053642136 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.239 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e4a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:37996/0 (socket says 172.21.15.89:37996) 2023-10-26T06:56:47.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 -- 172.21.15.89:0/3053642136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 msgr2=0x55608e4a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e4a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 -- 172.21.15.89:0/3053642136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 msgr2=0x55608e4a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:47.240 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e4a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 -- 172.21.15.89:0/3053642136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55608e3a43f0 con 0x55608e49a680 2023-10-26T06:56:47.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e4a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:47.241 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fb359e700 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4a40c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2fac004a80 tx=0x7f2fac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1491980277dd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2f96ff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:56:47.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2f96ff5700 1 -- 172.21.15.89:0/3053642136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac025640 con 0x55608e49a680 2023-10-26T06:56:47.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2f96ff5700 1 -- 172.21.15.89:0/3053642136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2fac033080 con 0x55608e49a680 2023-10-26T06:56:47.242 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.208+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa4019c50 con 0x55608e49a680 2023-10-26T06:56:47.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2f96ff5700 1 -- 172.21.15.89:0/3053642136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fac007b70 con 0x55608e49a680 2023-10-26T06:56:47.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 10 client.16614 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:47.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55608e1fac60 con 0x55608e49a680 2023-10-26T06:56:47.243 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2f96ff5700 1 -- 172.21.15.89:0/3053642136 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fac044020 con 0x55608e49a680 2023-10-26T06:56:47.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2f96ff5700 1 -- 172.21.15.89:0/3053642136 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fac035070 con 0x55608e49a680 2023-10-26T06:56:47.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:47.244 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:47.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 20 client.16614 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:47.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e503860 0x55608e505c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 4 client.16614 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:56:47.245 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.209+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55608e3dfe50 con 0x55608e503860 2023-10-26T06:56:47.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e503860 0x55608e505c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2f96ff5700 10 client.16614 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:47.246 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fb2d9d700 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e503860 0x55608e505c80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537e149126f98127 server_cookie=885fd2c774915b99 in_seq=0 out_seq=0 2023-10-26T06:56:47.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2f96ff5700 1 -- 172.21.15.89:0/3053642136 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55608e3dfe50 con 0x55608e503860 2023-10-26T06:56:47.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2f96ff5700 10 client.16614 handle_command_reply: tid=0 2023-10-26T06:56:47.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 client.16614 shutdown 2023-10-26T06:56:47.247 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 msgr2=0x55608e4a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4a40c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2fac004a80 tx=0x7f2fac004ab0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 shutdown_connections 2023-10-26T06:56:47.248 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55608e49a680 0x55608e4a40c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55608e503860 0x55608e505c80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55608e494820 0x55608e4a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 --2- 172.21.15.89:0/3053642136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55608e499d60 0x55608e4a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.249 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2f96ff5700 0 client.16614 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:47.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.210+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 >> 172.21.15.89:0/3053642136 conn(0x55608e408720 msgr2=0x55608e49c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 shutdown_connections 2023-10-26T06:56:47.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fccbb5b80 1 -- 172.21.15.89:0/3053642136 wait complete. 2023-10-26T06:56:47.250 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fccbb5b80 20 client.16614 trim_cache size 0 max 16384 2023-10-26T06:56:47.251 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:56:47.251 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:56:47.252 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:56:47.252 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:56:47.253 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f9c0a1670 msgr2=0x7f2f9c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f9c0a1670 0x7f2f9c0a3b30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009d10 tx=0x7f2fac004320 comp rx=0 tx=0).stop 2023-10-26T06:56:47.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100f90 msgr2=0x7f2fb419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.254 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100f90 0x7f2fb419eec0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005910 tx=0x7f2fa8004730 comp rx=0 tx=0).stop 2023-10-26T06:56:47.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 shutdown_connections 2023-10-26T06:56:47.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2f9c0a1670 0x7f2f9c0a3b30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb41005d0 0x7f2fb419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.255 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb4106690 0x7f2fb4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 --2- 172.21.15.89:0/3535263293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100f90 0x7f2fb419eec0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.211+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 >> 172.21.15.89:0/3535263293 conn(0x7f2fb40fc0c0 msgr2=0x7f2fb40fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.212+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 shutdown_connections 2023-10-26T06:56:47.256 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.212+0000 7f2fb89fe700 1 -- 172.21.15.89:0/3535263293 wait complete. 2023-10-26T06:56:47.257 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:56:47.257 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:56:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 -- 172.21.15.89:0/2313159987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41066b0 msgr2=0x7f9ca4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 --2- 172.21.15.89:0/2313159987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41066b0 0x7f9ca4106a90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009990 tx=0x7f9ca0009ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 -- 172.21.15.89:0/2313159987 shutdown_connections 2023-10-26T06:56:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 --2- 172.21.15.89:0/2313159987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca41066b0 0x7f9ca4106a90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 --2- 172.21.15.89:0/2313159987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca4101060 0x7f9ca410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 --2- 172.21.15.89:0/2313159987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 0x7f9ca4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 -- 172.21.15.89:0/2313159987 >> 172.21.15.89:0/2313159987 conn(0x7f9ca40fc170 msgr2=0x7f9ca40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 -- 172.21.15.89:0/2313159987 shutdown_connections 2023-10-26T06:56:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.461+0000 7f9cabdff700 1 -- 172.21.15.89:0/2313159987 wait complete. 2023-10-26T06:56:47.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 Processor -- start 2023-10-26T06:56:47.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 -- start start 2023-10-26T06:56:47.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 0x7f9ca419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4101060 0x7f9ca419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca41066b0 0x7f9ca4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4199170 con 0x7f9ca4101060 2023-10-26T06:56:47.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca41992e0 con 0x7f9ca41006a0 2023-10-26T06:56:47.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.463+0000 7f9cabdff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca41994a0 con 0x7f9ca41066b0 2023-10-26T06:56:47.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 0x7f9ca419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 0x7f9ca419e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34784/0 (socket says 172.21.15.89:34784) 2023-10-26T06:56:47.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 -- 172.21.15.89:0/1476007090 learned_addr learned my addr 172.21.15.89:0/1476007090 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca939a700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4101060 0x7f9ca419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9caa39c700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca41066b0 0x7f9ca4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 -- 172.21.15.89:0/1476007090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca41066b0 msgr2=0x7f9ca4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca41066b0 0x7f9ca4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 -- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4101060 msgr2=0x7f9ca419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4101060 0x7f9ca419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 -- 172.21.15.89:0/1476007090 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94009520 con 0x7f9ca41006a0 2023-10-26T06:56:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9ca9b9b700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 0x7f9ca419e9e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0005910 tx=0x7f9ca0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9c9affd700 1 -- 172.21.15.89:0/1476007090 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca001d070 con 0x7f9ca41006a0 2023-10-26T06:56:47.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.464+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca00095f0 con 0x7f9ca41006a0 2023-10-26T06:56:47.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.465+0000 7f9c9affd700 1 -- 172.21.15.89:0/1476007090 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca0022470 con 0x7f9ca41006a0 2023-10-26T06:56:47.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.465+0000 7f9c9affd700 1 -- 172.21.15.89:0/1476007090 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0017650 con 0x7f9ca41006a0 2023-10-26T06:56:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.465+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca41a57f0 con 0x7f9ca41006a0 2023-10-26T06:56:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.465+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca4101de0 con 0x7f9ca41006a0 2023-10-26T06:56:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.467+0000 7f9c9affd700 1 -- 172.21.15.89:0/1476007090 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9ca00225e0 con 0x7f9ca41006a0 2023-10-26T06:56:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.469+0000 7f9c9affd700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c900a15f0 0x7f9c900a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.469+0000 7f9c9affd700 1 -- 172.21.15.89:0/1476007090 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f9ca00ce370 con 0x7f9ca41006a0 2023-10-26T06:56:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.469+0000 7f9ca939a700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c900a15f0 0x7f9c900a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.470+0000 7f9ca939a700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c900a15f0 0x7f9c900a3ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4199cd0 tx=0x7f9c94009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.478 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.472+0000 7f9c9affd700 1 -- 172.21.15.89:0/1476007090 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9ca00976d0 con 0x7f9ca41006a0 2023-10-26T06:56:47.747 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.746+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ca4067530 con 0x7f9ca41006a0 2023-10-26T06:56:47.748 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.747+0000 7f9c9affd700 1 -- 172.21.15.89:0/1476007090 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f9ca009acf0 con 0x7f9ca41006a0 2023-10-26T06:56:47.749 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:56:47.749 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:56:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c900a15f0 msgr2=0x7f9c900a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c900a15f0 0x7f9c900a3ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4199cd0 tx=0x7f9c94009260 comp rx=0 tx=0).stop 2023-10-26T06:56:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 msgr2=0x7f9ca419e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.751 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 0x7f9ca419e9e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0005910 tx=0x7f9ca0004730 comp rx=0 tx=0).stop 2023-10-26T06:56:47.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 shutdown_connections 2023-10-26T06:56:47.753 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f9c900a15f0 0x7f9c900a3ab0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ca41066b0 0x7f9ca4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ca41006a0 0x7f9ca419e9e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.754 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 --2- 172.21.15.89:0/1476007090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9ca4101060 0x7f9ca419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.749+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 >> 172.21.15.89:0/1476007090 conn(0x7f9ca40fc170 msgr2=0x7f9ca410bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.755 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.750+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 shutdown_connections 2023-10-26T06:56:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.750+0000 7f9cabdff700 1 -- 172.21.15.89:0/1476007090 wait complete. 2023-10-26T06:56:47.756 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:56:47.769 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:56:47.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:47 smithi192 ceph-mon[131661]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:56:47.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:47 smithi135 ceph-mon[131403]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:56:47.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/495643694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 msgr2=0x7f2e18106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/495643694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 0x7f2e18106b00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009990 tx=0x7f2e08009ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/495643694 shutdown_connections 2023-10-26T06:56:47.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/495643694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 0x7f2e18106b00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/495643694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e181010d0 0x7f2e1810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/495643694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 0x7f2e18100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/495643694 >> 172.21.15.89:0/495643694 conn(0x7f2e180fc200 msgr2=0x7f2e180fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:47.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/495643694 shutdown_connections 2023-10-26T06:56:47.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.974+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/495643694 wait complete. 2023-10-26T06:56:47.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.975+0000 7f2e1e9ff700 1 Processor -- start 2023-10-26T06:56:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.975+0000 7f2e1e9ff700 1 -- start start 2023-10-26T06:56:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e1e9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 0x7f2e1819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e1e9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e181010d0 0x7f2e1819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e1e9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 0x7f2e18196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e1e9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181970a0 con 0x7f2e18106720 2023-10-26T06:56:47.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e1e9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e18197210 con 0x7f2e181010d0 2023-10-26T06:56:47.982 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e1e9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181973d0 con 0x7f2e18100710 2023-10-26T06:56:47.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 0x7f2e1819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e181010d0 0x7f2e1819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 0x7f2e1819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38028/0 (socket says 172.21.15.89:38028) 2023-10-26T06:56:47.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 -- 172.21.15.89:0/3264888003 learned_addr learned my addr 172.21.15.89:0/3264888003 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e1cf9c700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 0x7f2e18196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 -- 172.21.15.89:0/3264888003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e181010d0 msgr2=0x7f2e1819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e181010d0 0x7f2e1819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 -- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 msgr2=0x7f2e18196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:47.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 0x7f2e18196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:47.985 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.976+0000 7f2e17fff700 1 -- 172.21.15.89:0/3264888003 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10009c40 con 0x7f2e18100710 2023-10-26T06:56:47.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.977+0000 7f2e17fff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 0x7f2e1819c910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005910 tx=0x7f2e08004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:47.986 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.977+0000 7f2e157fa700 1 -- 172.21.15.89:0/3264888003 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0801d070 con 0x7f2e18100710 2023-10-26T06:56:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.977+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e080095f0 con 0x7f2e18100710 2023-10-26T06:56:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.977+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181a3720 con 0x7f2e18100710 2023-10-26T06:56:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.977+0000 7f2e157fa700 1 -- 172.21.15.89:0/3264888003 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e08022470 con 0x7f2e18100710 2023-10-26T06:56:47.987 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.977+0000 7f2e157fa700 1 -- 172.21.15.89:0/3264888003 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e08017650 con 0x7f2e18100710 2023-10-26T06:56:47.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.982+0000 7f2e157fa700 1 -- 172.21.15.89:0/3264888003 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2e0800fac0 con 0x7f2e18100710 2023-10-26T06:56:47.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 Processor -- start 2023-10-26T06:56:47.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 -- start start 2023-10-26T06:56:47.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e01a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e0154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e015e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:47.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946df846e0 con 0x55946e019ee0 2023-10-26T06:56:47.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946df6ac60 con 0x55946e01a800 2023-10-26T06:56:47.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e32320b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946df68580 con 0x55946e0159e0 2023-10-26T06:56:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e1cf9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e01a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e0154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.998 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e015e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:47.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e0154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34816/0 (socket says 172.21.15.89:34816) 2023-10-26T06:56:47.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e17fff700 1 -- 172.21.15.89:0/2328125328 learned_addr learned my addr 172.21.15.89:0/2328125328 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:47.999 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.983+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e015e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38030/0 (socket says 172.21.15.89:38030) 2023-10-26T06:56:48.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e157fa700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2e000a15f0 0x7f2e000a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e157fa700 1 -- 172.21.15.89:0/3264888003 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2e080ce1d0 con 0x7f2e18100710 2023-10-26T06:56:48.000 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e177fe700 1 -- 172.21.15.89:0/2328125328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 msgr2=0x55946e0154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e177fe700 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e0154a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e1cf9c700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2e000a15f0 0x7f2e000a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e177fe700 1 -- 172.21.15.89:0/2328125328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 msgr2=0x55946e01a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e177fe700 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e01a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e177fe700 1 -- 172.21.15.89:0/2328125328 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55946df213f0 con 0x55946e0159e0 2023-10-26T06:56:48.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e17fff700 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e0154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e177fe700 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e015e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009c80 tx=0x7f2e0c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31eeafa212df6158 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2e1cf9c700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2e000a15f0 0x7f2e000a3ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e10004a60 tx=0x7f2e10004b40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:48.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.984+0000 7f2dfe7fc700 1 -- 172.21.15.89:0/2328125328 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c011cb0 con 0x55946e0159e0 2023-10-26T06:56:48.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2dfe7fc700 1 -- 172.21.15.89:0/2328125328 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e0c011e10 con 0x55946e0159e0 2023-10-26T06:56:48.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2dfe7fc700 1 -- 172.21.15.89:0/2328125328 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c005280 con 0x55946e0159e0 2023-10-26T06:56:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 -- 172.21.15.89:0/2328125328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 msgr2=0x55946e015e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e015e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009c80 tx=0x7f2e0c009f90 comp rx=0 tx=0).stop 2023-10-26T06:56:48.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 -- 172.21.15.89:0/2328125328 shutdown_connections 2023-10-26T06:56:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e01a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e015e20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2328125328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e0154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 -- 172.21.15.89:0/2328125328 >> 172.21.15.89:0/2328125328 conn(0x55946df85720 msgr2=0x55946e00bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 -- 172.21.15.89:0/2328125328 shutdown_connections 2023-10-26T06:56:48.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.985+0000 7f2e32320b80 1 -- 172.21.15.89:0/2328125328 wait complete. 2023-10-26T06:56:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.986+0000 7f2e32320b80 1 -- start start 2023-10-26T06:56:48.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.986+0000 7f2e32320b80 1 Processor -- start 2023-10-26T06:56:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e0159e0 0x55946e0218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e019ee0 0x55946e01eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e01a800 0x55946e01f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e32320b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946df63e70 con 0x55946e0159e0 2023-10-26T06:56:48.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e32320b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946df00710 con 0x55946e019ee0 2023-10-26T06:56:48.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e32320b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946df003b0 con 0x55946e01a800 2023-10-26T06:56:48.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e32320b80 10 client.0 _ll_register_callbacks cb 0x55946de4aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e0159e0 0x55946e0218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e0159e0 0x55946e0218a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48700/0 (socket says 172.21.15.89:48700) 2023-10-26T06:56:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e17fff700 1 -- 172.21.15.89:0/2185717865 learned_addr learned my addr 172.21.15.89:0/2185717865 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:48.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e177fe700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e019ee0 0x55946e01eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.987+0000 7f2e1cf9c700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e01a800 0x55946e01f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e17fff700 1 -- 172.21.15.89:0/2185717865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e01a800 msgr2=0x55946e01f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e17fff700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e01a800 0x55946e01f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e17fff700 1 -- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e019ee0 msgr2=0x55946e01eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e17fff700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e019ee0 0x55946e01eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e17fff700 1 -- 172.21.15.89:0/2185717865 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181a3720 con 0x55946e0159e0 2023-10-26T06:56:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e177fe700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e019ee0 0x55946e01eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:48.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e17fff700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e0159e0 0x55946e0218a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cb9e0 tx=0x7f2e080d19d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78df277d21b86702 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2df77fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-10-26T06:56:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2df77fe700 1 -- 172.21.15.89:0/2185717865 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0809b020 con 0x55946e0159e0 2023-10-26T06:56:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2df77fe700 1 -- 172.21.15.89:0/2185717865 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2e0809f3a0 con 0x55946e0159e0 2023-10-26T06:56:48.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55946df68ef0 con 0x55946e0159e0 2023-10-26T06:56:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2df77fe700 1 -- 172.21.15.89:0/2185717865 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e0809ad90 con 0x55946e0159e0 2023-10-26T06:56:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e32320b80 10 client.16632 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:48.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.988+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f2e0c0098e0 con 0x55946e0159e0 2023-10-26T06:56:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2df77fe700 1 -- 172.21.15.89:0/2185717865 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55946df68ef0 con 0x55946e0159e0 2023-10-26T06:56:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2df77fe700 1 -- 172.21.15.89:0/2185717865 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e0809a740 con 0x55946e0159e0 2023-10-26T06:56:48.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2e32320b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:48.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2e32320b80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:48.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2e32320b80 20 client.16632 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:48.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e02ee40 0x55946e031230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2e32320b80 4 client.16632 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:48.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55946dfe08b0 con 0x55946e02ee40 2023-10-26T06:56:48.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.989+0000 7f2e177fe700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e02ee40 0x55946e031230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.990+0000 7f2df77fe700 10 client.16632 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:48.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.990+0000 7f2e177fe700 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e02ee40 0x55946e031230 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb3b5a6bfe769db server_cookie=7102a15003504d4a in_seq=0 out_seq=0 2023-10-26T06:56:48.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.990+0000 7f2df77fe700 1 -- 172.21.15.89:0/2185717865 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55946dfe08b0 con 0x55946e02ee40 2023-10-26T06:56:48.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.990+0000 7f2df77fe700 10 client.16632 handle_command_reply: tid=0 2023-10-26T06:56:48.024 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.990+0000 7f2e32320b80 1 client.16632 shutdown 2023-10-26T06:56:48.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e0159e0 msgr2=0x55946e0218a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:48.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e0159e0 0x55946e0218a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cb9e0 tx=0x7f2e080d19d0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 shutdown_connections 2023-10-26T06:56:48.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e0159e0 0x55946e0218a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e02ee40 0x55946e031230 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e01a800 0x55946e01f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 --2- 172.21.15.89:0/2185717865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e019ee0 0x55946e01eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2df77fe700 0 client.16632 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:48.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 >> 172.21.15.89:0/2185717865 conn(0x55946df85720 msgr2=0x55946e015dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:48.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 shutdown_connections 2023-10-26T06:56:48.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 1 -- 172.21.15.89:0/2185717865 wait complete. 2023-10-26T06:56:48.027 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:47.991+0000 7f2e32320b80 20 client.16632 trim_cache size 0 max 16384 2023-10-26T06:56:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 Processor -- start 2023-10-26T06:56:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 -- start start 2023-10-26T06:56:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e0159e0 0x55946e01eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e019ee0 0x55946e0262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e01a800 0x55946e0213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946dde4920 con 0x55946e01a800 2023-10-26T06:56:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946e026840 con 0x55946e0159e0 2023-10-26T06:56:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.005+0000 7f2e32320b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946e01d0f0 con 0x55946e019ee0 2023-10-26T06:56:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e019ee0 0x55946e0262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e019ee0 0x55946e0262d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38034/0 (socket says 172.21.15.89:38034) 2023-10-26T06:56:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 -- 172.21.15.89:0/3424334951 learned_addr learned my addr 172.21.15.89:0/3424334951 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e17fff700 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e0159e0 0x55946e01eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e1cf9c700 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e01a800 0x55946e0213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 -- 172.21.15.89:0/3424334951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e0159e0 msgr2=0x55946e01eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e0159e0 0x55946e01eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 -- 172.21.15.89:0/3424334951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e01a800 msgr2=0x55946e0213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e01a800 0x55946e0213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 -- 172.21.15.89:0/3424334951 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55946dd8ede0 con 0x55946e019ee0 2023-10-26T06:56:48.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e17fff700 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e0159e0 0x55946e01eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.006+0000 7f2e177fe700 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e019ee0 0x55946e0262d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0049f0 tx=0x7f2e0c004a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5048741bc85d3d0b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2dfeffd700 1 -- 172.21.15.89:0/3424334951 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c004b00 con 0x55946e019ee0 2023-10-26T06:56:48.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2dfeffd700 1 -- 172.21.15.89:0/3424334951 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e0c004510 con 0x55946e019ee0 2023-10-26T06:56:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2dfeffd700 1 -- 172.21.15.89:0/3424334951 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c003c30 con 0x55946e019ee0 2023-10-26T06:56:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 -- 172.21.15.89:0/3424334951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e019ee0 msgr2=0x55946e0262d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:48.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e019ee0 0x55946e0262d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0049f0 tx=0x7f2e0c004a20 comp rx=0 tx=0).stop 2023-10-26T06:56:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 -- 172.21.15.89:0/3424334951 shutdown_connections 2023-10-26T06:56:48.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e01a800 0x55946e0213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e0159e0 0x55946e01eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.041 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 --2- 172.21.15.89:0/3424334951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e019ee0 0x55946e0262d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 -- 172.21.15.89:0/3424334951 >> 172.21.15.89:0/3424334951 conn(0x55946df85720 msgr2=0x55946e01e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 -- 172.21.15.89:0/3424334951 shutdown_connections 2023-10-26T06:56:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.007+0000 7f2e32320b80 1 -- 172.21.15.89:0/3424334951 wait complete. 2023-10-26T06:56:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 -- start start 2023-10-26T06:56:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 Processor -- start 2023-10-26T06:56:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e016e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e017350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e017890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946e026680 con 0x55946e019ee0 2023-10-26T06:56:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946e082c20 con 0x55946e01a800 2023-10-26T06:56:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55946e082d90 con 0x55946e0159e0 2023-10-26T06:56:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.008+0000 7f2e32320b80 10 client.0 _ll_register_callbacks cb 0x55946de4aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e016e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e017350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e016e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38042/0 (socket says 172.21.15.89:38042) 2023-10-26T06:56:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 -- 172.21.15.89:0/1427713838 learned_addr learned my addr 172.21.15.89:0/1427713838 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e1cf9c700 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e017890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 -- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 msgr2=0x55946e017890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e017890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 -- 172.21.15.89:0/1427713838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 msgr2=0x55946e017350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e017350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 -- 172.21.15.89:0/1427713838 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c0098e0 con 0x55946e0159e0 2023-10-26T06:56:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e177fe700 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e017350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2e17fff700 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e016e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e0809af50 tx=0x7f2e08099800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574e378fc30c1fc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.009+0000 7f2df77fe700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:56:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2df77fe700 1 -- 172.21.15.89:0/1427713838 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e080a32c0 con 0x55946e0159e0 2023-10-26T06:56:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2df77fe700 1 -- 172.21.15.89:0/1427713838 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f2e0809b950 con 0x55946e0159e0 2023-10-26T06:56:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55946e016450 con 0x55946e0159e0 2023-10-26T06:56:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2df77fe700 1 -- 172.21.15.89:0/1427713838 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e0809ad90 con 0x55946e0159e0 2023-10-26T06:56:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 10 client.26680 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7f2e0c01d040 con 0x55946e0159e0 2023-10-26T06:56:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2df77fe700 1 -- 172.21.15.89:0/1427713838 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55946e016450 con 0x55946e0159e0 2023-10-26T06:56:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2df77fe700 1 -- 172.21.15.89:0/1427713838 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e08084420 con 0x55946e0159e0 2023-10-26T06:56:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 10 client.26680 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 20 client.26680 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e084120 0x55946e086540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 4 client.26680 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:56:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f2e0c011e10 con 0x55946e084120 2023-10-26T06:56:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.010+0000 7f2e177fe700 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e084120 0x55946e086540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e177fe700 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e084120 0x55946e086540 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4e99a5b15bcf7b server_cookie=66fbd047a0607d0c in_seq=0 out_seq=0 2023-10-26T06:56:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2df77fe700 10 client.26680 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2df77fe700 1 -- 172.21.15.89:0/1427713838 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f2e0c011e10 con 0x55946e084120 2023-10-26T06:56:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2df77fe700 10 client.26680 handle_command_reply: tid=0 2023-10-26T06:56:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 client.26680 shutdown 2023-10-26T06:56:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 msgr2=0x55946e016e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e016e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e0809af50 tx=0x7f2e08099800 comp rx=0 tx=0).stop 2023-10-26T06:56:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 shutdown_connections 2023-10-26T06:56:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55946e084120 0x55946e086540 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55946e01a800 0x55946e017890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55946e0159e0 0x55946e016e10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2df77fe700 0 client.26680 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 --2- 172.21.15.89:0/1427713838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55946e019ee0 0x55946e017350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 >> 172.21.15.89:0/1427713838 conn(0x55946df85720 msgr2=0x55946e010080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.011+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 shutdown_connections 2023-10-26T06:56:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e32320b80 1 -- 172.21.15.89:0/1427713838 wait complete. 2023-10-26T06:56:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e32320b80 20 client.26680 trim_cache size 0 max 16384 2023-10-26T06:56:48.060 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:56:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2e000a15f0 msgr2=0x7f2e000a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2e000a15f0 0x7f2e000a3ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e10004a60 tx=0x7f2e10004b40 comp rx=0 tx=0).stop 2023-10-26T06:56:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 msgr2=0x7f2e1819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 0x7f2e1819c910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005910 tx=0x7f2e08004730 comp rx=0 tx=0).stop 2023-10-26T06:56:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 shutdown_connections 2023-10-26T06:56:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f2e000a15f0 0x7f2e000a3ab0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e18106720 0x7f2e18196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e18100710 0x7f2e1819c910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 --2- 172.21.15.89:0/3264888003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e181010d0 0x7f2e1819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:48.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 >> 172.21.15.89:0/3264888003 conn(0x7f2e180fc200 msgr2=0x7f2e180fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:48.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 shutdown_connections 2023-10-26T06:56:48.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:48.012+0000 7f2e1e9ff700 1 -- 172.21.15.89:0/3264888003 wait complete. 2023-10-26T06:56:48.064 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:47 smithi089 ceph-mon[95031]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:56:48.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:48 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1476007090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:56:48.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:48 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1476007090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:56:49.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:48 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1476007090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:56:49.066 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-26T06:56:49.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-26T06:56:49.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 -- 172.21.15.89:0/844857672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105170 msgr2=0x7efe681055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/844857672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105170 0x7efe681055f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0099c0 tx=0x7efe5c009cd0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 -- 172.21.15.89:0/844857672 shutdown_connections 2023-10-26T06:56:49.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/844857672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68105b30 0x7efe6810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.280 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/844857672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe68067d40 0x7efe68068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/844857672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105170 0x7efe681055f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 -- 172.21.15.89:0/844857672 >> 172.21.15.89:0/844857672 conn(0x7efe680767f0 msgr2=0x7efe68076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 -- 172.21.15.89:0/844857672 shutdown_connections 2023-10-26T06:56:49.281 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.277+0000 7efe6e9ff700 1 -- 172.21.15.89:0/844857672 wait complete. 2023-10-26T06:56:49.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.278+0000 7efe6e9ff700 1 Processor -- start 2023-10-26T06:56:49.284 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.278+0000 7efe6e9ff700 1 -- start start 2023-10-26T06:56:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe6e9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68067d40 0x7efe68198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe6e9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe68105170 0x7efe68199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe6e9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105b30 0x7efe6819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.285 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe6e9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6819dd00 con 0x7efe68105b30 2023-10-26T06:56:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe6e9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6819de70 con 0x7efe68105170 2023-10-26T06:56:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe6e9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6819e030 con 0x7efe68067d40 2023-10-26T06:56:49.286 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68067d40 0x7efe68198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68067d40 0x7efe68198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38062/0 (socket says 172.21.15.89:38062) 2023-10-26T06:56:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 -- 172.21.15.89:0/319646852 learned_addr learned my addr 172.21.15.89:0/319646852 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe677fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe68105170 0x7efe68199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.287 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe6cf9c700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105b30 0x7efe6819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 -- 172.21.15.89:0/319646852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe68105170 msgr2=0x7efe68199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe68105170 0x7efe68199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 -- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105b30 msgr2=0x7efe6819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.288 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105b30 0x7efe6819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe67fff700 1 -- 172.21.15.89:0/319646852 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe58009520 con 0x7efe68067d40 2023-10-26T06:56:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.279+0000 7efe677fe700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe68105170 0x7efe68199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:49.289 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.280+0000 7efe67fff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68067d40 0x7efe68198de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efe5800eb30 tx=0x7efe5800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.280+0000 7efe657fa700 1 -- 172.21.15.89:0/319646852 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58009880 con 0x7efe68067d40 2023-10-26T06:56:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.280+0000 7efe657fa700 1 -- 172.21.15.89:0/319646852 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe580044e0 con 0x7efe68067d40 2023-10-26T06:56:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.280+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c0095f0 con 0x7efe68067d40 2023-10-26T06:56:49.290 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.280+0000 7efe657fa700 1 -- 172.21.15.89:0/319646852 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58003d80 con 0x7efe68067d40 2023-10-26T06:56:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.280+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe681a5970 con 0x7efe68067d40 2023-10-26T06:56:49.291 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.283+0000 7efe657fa700 1 -- 172.21.15.89:0/319646852 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efe58003f20 con 0x7efe68067d40 2023-10-26T06:56:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.284+0000 7efe82383b80 1 Processor -- start 2023-10-26T06:56:49.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.284+0000 7efe657fa700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe500a1670 0x7efe500a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe657fa700 1 -- 172.21.15.89:0/319646852 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7efe58014070 con 0x7efe68067d40 2023-10-26T06:56:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe82383b80 1 -- start start 2023-10-26T06:56:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe6cf9c700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe500a1670 0x7efe500a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe82383b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c555040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe82383b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c555580 0x557e7c54f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe82383b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c54f650 0x557e7c54fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.300 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe82383b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c4c2760 con 0x557e7c54f650 2023-10-26T06:56:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe82383b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c4a8ce0 con 0x557e7c555580 2023-10-26T06:56:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe82383b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c4a6600 con 0x557e7c554c60 2023-10-26T06:56:49.301 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe677fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c555580 0x557e7c54f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe67fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c555040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.285+0000 7efe677fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c555580 0x557e7c54f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34866/0 (socket says 172.21.15.89:34866) 2023-10-26T06:56:49.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe67fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c555040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38074/0 (socket says 172.21.15.89:38074) 2023-10-26T06:56:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe677fe700 1 -- 172.21.15.89:0/1773231305 learned_addr learned my addr 172.21.15.89:0/1773231305 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe6cf9c700 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c54f650 0x557e7c54fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe6cf9c700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe500a1670 0x7efe500a3b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efe6000f520 tx=0x7efe6000b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe677fe700 1 -- 172.21.15.89:0/1773231305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 msgr2=0x557e7c555040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe677fe700 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c555040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe677fe700 1 -- 172.21.15.89:0/1773231305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c54f650 msgr2=0x557e7c54fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe677fe700 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c54f650 0x557e7c54fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe677fe700 1 -- 172.21.15.89:0/1773231305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e7c45f470 con 0x557e7c555580 2023-10-26T06:56:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe6cf9c700 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c54f650 0x557e7c54fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe677fe700 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c555580 0x557e7c54f110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0058e0 tx=0x7efe5c005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f861fbfc7048dd67 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe4e7fc700 1 -- 172.21.15.89:0/1773231305 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c01c050 con 0x557e7c555580 2023-10-26T06:56:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe4e7fc700 1 -- 172.21.15.89:0/1773231305 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe5c01d040 con 0x557e7c555580 2023-10-26T06:56:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.286+0000 7efe4e7fc700 1 -- 172.21.15.89:0/1773231305 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003ae0 con 0x557e7c555580 2023-10-26T06:56:49.306 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 -- 172.21.15.89:0/1773231305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c555580 msgr2=0x557e7c54f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c555580 0x557e7c54f110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0058e0 tx=0x7efe5c005f00 comp rx=0 tx=0).stop 2023-10-26T06:56:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 -- 172.21.15.89:0/1773231305 shutdown_connections 2023-10-26T06:56:49.307 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c54f650 0x557e7c54fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c555040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 --2- 172.21.15.89:0/1773231305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c555580 0x557e7c54f110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.308 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 -- 172.21.15.89:0/1773231305 >> 172.21.15.89:0/1773231305 conn(0x557e7c4c37a0 msgr2=0x557e7c549be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 -- 172.21.15.89:0/1773231305 shutdown_connections 2023-10-26T06:56:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.287+0000 7efe82383b80 1 -- 172.21.15.89:0/1773231305 wait complete. 2023-10-26T06:56:49.309 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.288+0000 7efe82383b80 1 -- start start 2023-10-26T06:56:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.288+0000 7efe82383b80 1 Processor -- start 2023-10-26T06:56:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.288+0000 7efe82383b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c54f650 0x557e7c55b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.310 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe82383b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c554c60 0x557e7c5587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe82383b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c558df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe82383b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c4a1ef0 con 0x557e7c555580 2023-10-26T06:56:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe82383b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c43e790 con 0x557e7c554c60 2023-10-26T06:56:49.311 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe82383b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c43e430 con 0x557e7c54f650 2023-10-26T06:56:49.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe82383b80 10 client.0 _ll_register_callbacks cb 0x557e7c388b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:49.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c54f650 0x557e7c55b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.312 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c54f650 0x557e7c55b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38078/0 (socket says 172.21.15.89:38078) 2023-10-26T06:56:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 -- 172.21.15.89:0/2211121310 learned_addr learned my addr 172.21.15.89:0/2211121310 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe6cf9c700 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c558df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe677fe700 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c554c60 0x557e7c5587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 -- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c554c60 msgr2=0x557e7c5587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c554c60 0x557e7c5587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 -- 172.21.15.89:0/2211121310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 msgr2=0x557e7c558df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c558df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.314 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 -- 172.21.15.89:0/2211121310 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6810de70 con 0x557e7c54f650 2023-10-26T06:56:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe67fff700 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c54f650 0x557e7c55b540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efe5809a540 tx=0x7efe5809a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b00b01468755f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.289+0000 7efe477fe700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:56:49.315 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe477fe700 1 -- 172.21.15.89:0/2211121310 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5809e700 con 0x557e7c54f650 2023-10-26T06:56:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe477fe700 1 -- 172.21.15.89:0/2211121310 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7efe580a2940 con 0x557e7c54f650 2023-10-26T06:56:49.316 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e7c4a6f70 con 0x557e7c54f650 2023-10-26T06:56:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe477fe700 1 -- 172.21.15.89:0/2211121310 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe5800cdd0 con 0x557e7c54f650 2023-10-26T06:56:49.317 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 10 client.26692 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557e7c2cce60 con 0x557e7c54f650 2023-10-26T06:56:49.318 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe477fe700 1 -- 172.21.15.89:0/2211121310 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e7c4a6f70 con 0x557e7c54f650 2023-10-26T06:56:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe477fe700 1 -- 172.21.15.89:0/2211121310 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe58099aa0 con 0x557e7c54f650 2023-10-26T06:56:49.319 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 10 client.26692 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:49.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 10 client.26692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:49.320 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 20 client.26692 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:49.321 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c568e30 0x557e7c56b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 4 client.26692 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:49.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.290+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e7c51e930 con 0x557e7c568e30 2023-10-26T06:56:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.291+0000 7efe677fe700 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c568e30 0x557e7c56b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.291+0000 7efe477fe700 10 client.26692 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.323 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.291+0000 7efe677fe700 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c568e30 0x557e7c56b250 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17bde2dd082d89b server_cookie=2e1bea059cd328af in_seq=0 out_seq=0 2023-10-26T06:56:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe477fe700 1 -- 172.21.15.89:0/2211121310 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e7c51e930 con 0x557e7c568e30 2023-10-26T06:56:49.324 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe477fe700 10 client.26692 handle_command_reply: tid=0 2023-10-26T06:56:49.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 client.26692 shutdown 2023-10-26T06:56:49.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c54f650 msgr2=0x557e7c55b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c54f650 0x557e7c55b540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efe5809a540 tx=0x7efe5809a000 comp rx=0 tx=0).stop 2023-10-26T06:56:49.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 shutdown_connections 2023-10-26T06:56:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c558df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c568e30 0x557e7c56b250 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c54f650 0x557e7c55b540 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe477fe700 0 client.26692 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 --2- 172.21.15.89:0/2211121310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c554c60 0x557e7c5587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 >> 172.21.15.89:0/2211121310 conn(0x557e7c4c37a0 msgr2=0x557e7c54e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 shutdown_connections 2023-10-26T06:56:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 1 -- 172.21.15.89:0/2211121310 wait complete. 2023-10-26T06:56:49.331 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.292+0000 7efe82383b80 20 client.26692 trim_cache size 0 max 16384 2023-10-26T06:56:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.309+0000 7efe82383b80 1 Processor -- start 2023-10-26T06:56:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.309+0000 7efe82383b80 1 -- start start 2023-10-26T06:56:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe82383b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c552e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe82383b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c55ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe82383b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c564d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe82383b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c3229a0 con 0x557e7c555580 2023-10-26T06:56:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe82383b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c54a140 con 0x557e7c54f650 2023-10-26T06:56:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe82383b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c54b680 con 0x557e7c554c60 2023-10-26T06:56:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c55ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c55ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38090/0 (socket says 172.21.15.89:38090) 2023-10-26T06:56:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 -- 172.21.15.89:0/593520320 learned_addr learned my addr 172.21.15.89:0/593520320 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe67fff700 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c552e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe6cf9c700 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c564d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 -- 172.21.15.89:0/593520320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 msgr2=0x557e7c552e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c552e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 -- 172.21.15.89:0/593520320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 msgr2=0x557e7c564d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c564d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.310+0000 7efe677fe700 1 -- 172.21.15.89:0/593520320 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e7c2b5ce0 con 0x557e7c554c60 2023-10-26T06:56:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe677fe700 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c55ff00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efe5c000c00 tx=0x7efe5c003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=113c2c9b4638cac8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe4effd700 1 -- 172.21.15.89:0/593520320 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c031b40 con 0x557e7c554c60 2023-10-26T06:56:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe4effd700 1 -- 172.21.15.89:0/593520320 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe5c005250 con 0x557e7c554c60 2023-10-26T06:56:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe4effd700 1 -- 172.21.15.89:0/593520320 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003bf0 con 0x557e7c554c60 2023-10-26T06:56:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 -- 172.21.15.89:0/593520320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 msgr2=0x557e7c55ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c55ff00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efe5c000c00 tx=0x7efe5c003680 comp rx=0 tx=0).stop 2023-10-26T06:56:49.350 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 -- 172.21.15.89:0/593520320 shutdown_connections 2023-10-26T06:56:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c564d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c552e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.351 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 --2- 172.21.15.89:0/593520320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c55ff00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 -- 172.21.15.89:0/593520320 >> 172.21.15.89:0/593520320 conn(0x557e7c4c37a0 msgr2=0x557e7c564650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 -- 172.21.15.89:0/593520320 shutdown_connections 2023-10-26T06:56:49.352 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.311+0000 7efe82383b80 1 -- 172.21.15.89:0/593520320 wait complete. 2023-10-26T06:56:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.312+0000 7efe82383b80 1 -- start start 2023-10-26T06:56:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.312+0000 7efe82383b80 1 Processor -- start 2023-10-26T06:56:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.312+0000 7efe82383b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c5b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.312+0000 7efe82383b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c5b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.312+0000 7efe82383b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c55ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.312+0000 7efe82383b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c55f520 con 0x557e7c555580 2023-10-26T06:56:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.312+0000 7efe82383b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c55f690 con 0x557e7c54f650 2023-10-26T06:56:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe82383b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7c55f850 con 0x557e7c554c60 2023-10-26T06:56:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe82383b80 10 client.0 _ll_register_callbacks cb 0x557e7c388b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c5b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe67fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c5b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe6cf9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c55ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c5b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38102/0 (socket says 172.21.15.89:38102) 2023-10-26T06:56:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 -- 172.21.15.89:0/1814350134 learned_addr learned my addr 172.21.15.89:0/1814350134 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 -- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 msgr2=0x557e7c5b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c5b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 -- 172.21.15.89:0/1814350134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 msgr2=0x557e7c55ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c55ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 -- 172.21.15.89:0/1814350134 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e7c45f470 con 0x557e7c554c60 2023-10-26T06:56:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe67fff700 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c5b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe677fe700 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c5b6510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe5c025040 tx=0x7efe5c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146dc7e915085af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.313+0000 7efe477fe700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:56:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe477fe700 1 -- 172.21.15.89:0/1814350134 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0053c0 con 0x557e7c554c60 2023-10-26T06:56:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe5c01d040 con 0x557e7c554c60 2023-10-26T06:56:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe477fe700 1 -- 172.21.15.89:0/1814350134 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7efe5c005520 con 0x557e7c554c60 2023-10-26T06:56:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe477fe700 1 -- 172.21.15.89:0/1814350134 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe5c0056e0 con 0x557e7c554c60 2023-10-26T06:56:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 10 client.26698 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x557e7c2f32d0 con 0x557e7c554c60 2023-10-26T06:56:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe477fe700 1 -- 172.21.15.89:0/1814350134 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe5c01d040 con 0x557e7c554c60 2023-10-26T06:56:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe477fe700 1 -- 172.21.15.89:0/1814350134 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe5c003cf0 con 0x557e7c554c60 2023-10-26T06:56:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 10 client.26698 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 10 client.26698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:49.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 20 client.26698 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:49.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c5bec60 0x557e7c5c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 4 client.26698 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-26T06:56:49.365 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.314+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e7c49aed0 con 0x557e7c5bec60 2023-10-26T06:56:49.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.315+0000 7efe67fff700 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c5bec60 0x557e7c5c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.315+0000 7efe67fff700 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c5bec60 0x557e7c5c1080 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12aab6dac11ecc8 server_cookie=34625732770cc521 in_seq=0 out_seq=0 2023-10-26T06:56:49.366 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.315+0000 7efe477fe700 10 client.26698 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe477fe700 1 -- 172.21.15.89:0/1814350134 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e7c49aed0 con 0x557e7c5bec60 2023-10-26T06:56:49.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe477fe700 10 client.26698 handle_command_reply: tid=0 2023-10-26T06:56:49.367 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 client.26698 shutdown 2023-10-26T06:56:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 msgr2=0x557e7c5b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c5b6510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe5c025040 tx=0x7efe5c005f00 comp rx=0 tx=0).stop 2023-10-26T06:56:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 shutdown_connections 2023-10-26T06:56:49.368 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e7c555580 0x557e7c55ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x557e7c5bec60 0x557e7c5c1080 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e7c54f650 0x557e7c5b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.369 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 --2- 172.21.15.89:0/1814350134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557e7c554c60 0x557e7c5b6510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 >> 172.21.15.89:0/1814350134 conn(0x557e7c4c37a0 msgr2=0x557e7c558650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe477fe700 0 client.26698 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.316+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 shutdown_connections 2023-10-26T06:56:49.370 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe82383b80 1 -- 172.21.15.89:0/1814350134 wait complete. 2023-10-26T06:56:49.371 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe82383b80 20 client.26698 trim_cache size 0 max 16384 2023-10-26T06:56:49.372 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:56:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2023-10-26T06:56:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "e77066ea-c4ac-4455-b83f-93c6f5faac8c", 2023-10-26T06:56:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2023-10-26T06:56:49.373 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:56:49.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe500a1670 msgr2=0x7efe500a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe500a1670 0x7efe500a3b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efe6000f520 tx=0x7efe6000b040 comp rx=0 tx=0).stop 2023-10-26T06:56:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68067d40 msgr2=0x7efe68198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68067d40 0x7efe68198de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efe5800eb30 tx=0x7efe5800ee40 comp rx=0 tx=0).stop 2023-10-26T06:56:49.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 shutdown_connections 2023-10-26T06:56:49.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7efe500a1670 0x7efe500a3b30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe68105b30 0x7efe6819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe68067d40 0x7efe68198de0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.376 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 --2- 172.21.15.89:0/319646852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe68105170 0x7efe68199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 >> 172.21.15.89:0/319646852 conn(0x7efe680767f0 msgr2=0x7efe68100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 shutdown_connections 2023-10-26T06:56:49.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.317+0000 7efe6e9ff700 1 -- 172.21.15.89:0/319646852 wait complete. 2023-10-26T06:56:49.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:56:49.581 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.579+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3237756162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 msgr2=0x7fc6bc108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.579+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3237756162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 0x7fc6bc108bd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009990 tx=0x7fc6ac009ca0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.580+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3237756162 shutdown_connections 2023-10-26T06:56:49.582 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.580+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3237756162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1031a0 0x7fc6bc10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.580+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3237756162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 0x7fc6bc108bd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.580+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3237756162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6bc1027e0 0x7fc6bc102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.583 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.580+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3237756162 >> 172.21.15.89:0/3237756162 conn(0x7fc6bc0fe290 msgr2=0x7fc6bc1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.580+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3237756162 shutdown_connections 2023-10-26T06:56:49.584 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.580+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3237756162 wait complete. 2023-10-26T06:56:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 Processor -- start 2023-10-26T06:56:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 -- start start 2023-10-26T06:56:49.586 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6bc1027e0 0x7fc6bc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1031a0 0x7fc6bc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 0x7fc6bc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1990b0 con 0x7fc6bc1027e0 2023-10-26T06:56:49.587 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc199220 con 0x7fc6bc1031a0 2023-10-26T06:56:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.581+0000 7fc6bb59e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1993e0 con 0x7fc6bc1087f0 2023-10-26T06:56:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6bc1027e0 0x7fc6bc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6bad9d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 0x7fc6bc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.588 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6bc1027e0 0x7fc6bc19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48786/0 (socket says 172.21.15.89:48786) 2023-10-26T06:56:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6ba59c700 1 -- 172.21.15.89:0/3051383097 learned_addr learned my addr 172.21.15.89:0/3051383097 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1031a0 0x7fc6bc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6bad9d700 1 -- 172.21.15.89:0/3051383097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1031a0 msgr2=0x7fc6bc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.589 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1031a0 0x7fc6bc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6bad9d700 1 -- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6bc1027e0 msgr2=0x7fc6bc19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6bc1027e0 0x7fc6bc19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.590 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6bad9d700 1 -- 172.21.15.89:0/3051383097 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac0095f0 con 0x7fc6bc1087f0 2023-10-26T06:56:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 0x7fc6bc198a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400efc0 tx=0x7fc6b400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6ab7fe700 1 -- 172.21.15.89:0/3051383097 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4018070 con 0x7fc6bc1087f0 2023-10-26T06:56:49.594 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.582+0000 7fc6ab7fe700 1 -- 172.21.15.89:0/3051383097 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6b400f040 con 0x7fc6bc1087f0 2023-10-26T06:56:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.583+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc199660 con 0x7fc6bc1087f0 2023-10-26T06:56:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.583+0000 7fc6ab7fe700 1 -- 172.21.15.89:0/3051383097 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400cd40 con 0x7fc6bc1087f0 2023-10-26T06:56:49.595 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.583+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc1a5910 con 0x7fc6bc1087f0 2023-10-26T06:56:49.601 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.586+0000 7fc6ab7fe700 1 -- 172.21.15.89:0/3051383097 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc6b4003a00 con 0x7fc6bc1087f0 2023-10-26T06:56:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 Processor -- start 2023-10-26T06:56:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 -- start start 2023-10-26T06:56:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6ab7fe700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc6a40a1670 0x7fc6a40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab95140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab95680 0x560f2ab8f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab8f820 0x560f2ab8fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2ab026e0 con 0x560f2ab8f820 2023-10-26T06:56:49.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2aae8c60 con 0x560f2ab95680 2023-10-26T06:56:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2aae6580 con 0x560f2ab94d60 2023-10-26T06:56:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.587+0000 7fc6ab7fe700 1 -- 172.21.15.89:0/3051383097 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fc6b40ccc10 con 0x7fc6bc1087f0 2023-10-26T06:56:49.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc6a40a1670 0x7fc6a40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab95140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab95140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38134/0 (socket says 172.21.15.89:38134) 2023-10-26T06:56:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1608002346 learned_addr learned my addr 172.21.15.89:0/1608002346 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab8f820 0x560f2ab8fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab95680 0x560f2ab8f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1608002346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab95680 msgr2=0x560f2ab8f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab95680 0x560f2ab8f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.606 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1608002346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab8f820 msgr2=0x560f2ab8fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab8f820 0x560f2ab8fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1608002346 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f2aa9f3f0 con 0x560f2ab94d60 2023-10-26T06:56:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab8f820 0x560f2ab8fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:49.607 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab95140 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b640 tx=0x7fc6ac00b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4deea8f3a79b0a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.588+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc6a40a1670 0x7fc6a40a3b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x560f2ab95ae0 tx=0x7fc6b00099b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6a8ff9700 1 -- 172.21.15.89:0/1608002346 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac01d070 con 0x560f2ab94d60 2023-10-26T06:56:49.608 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6a8ff9700 1 -- 172.21.15.89:0/1608002346 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6ac01e040 con 0x560f2ab94d60 2023-10-26T06:56:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6a8ff9700 1 -- 172.21.15.89:0/1608002346 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0049c0 con 0x560f2ab94d60 2023-10-26T06:56:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1608002346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 msgr2=0x560f2ab95140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab95140 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b640 tx=0x7fc6ac00b670 comp rx=0 tx=0).stop 2023-10-26T06:56:49.609 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1608002346 shutdown_connections 2023-10-26T06:56:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab8f820 0x560f2ab8fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab95140 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1608002346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab95680 0x560f2ab8f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.610 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1608002346 >> 172.21.15.89:0/1608002346 conn(0x560f2ab03720 msgr2=0x560f2ab89cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1608002346 shutdown_connections 2023-10-26T06:56:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.589+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1608002346 wait complete. 2023-10-26T06:56:49.611 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.590+0000 7fc6d4a05b80 1 -- start start 2023-10-26T06:56:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.590+0000 7fc6d4a05b80 1 Processor -- start 2023-10-26T06:56:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab98a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.612 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2aae1e70 con 0x560f2ab95680 2023-10-26T06:56:49.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2aa7e710 con 0x560f2ab8f820 2023-10-26T06:56:49.613 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2aa7e3b0 con 0x560f2ab94d60 2023-10-26T06:56:49.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6d4a05b80 10 client.0 _ll_register_callbacks cb 0x560f2a9c8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:49.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6b9d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.614 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6b9d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab98a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:38136/0 (socket says 172.21.15.89:38136) 2023-10-26T06:56:49.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6b9d9b700 1 -- 172.21.15.89:0/4164510679 learned_addr learned my addr 172.21.15.89:0/4164510679 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34942/0 (socket says 172.21.15.89:34942) 2023-10-26T06:56:49.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.615 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6ba59c700 1 -- 172.21.15.89:0/4164510679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 msgr2=0x560f2ab98a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab98a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6ba59c700 1 -- 172.21.15.89:0/4164510679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 msgr2=0x560f2ab991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6ba59c700 1 -- 172.21.15.89:0/4164510679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc10ffb0 con 0x560f2ab8f820 2023-10-26T06:56:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.591+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab98a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9b6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0098e0 tx=0x7fc6ac00be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99740863bbd801db server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:56:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/4164510679 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac01d070 con 0x560f2ab8f820 2023-10-26T06:56:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f2aae6ef0 con 0x560f2ab8f820 2023-10-26T06:56:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/4164510679 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fc6ac004640 con 0x560f2ab8f820 2023-10-26T06:56:49.618 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/4164510679 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ac004800 con 0x560f2ab8f820 2023-10-26T06:56:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6d4a05b80 10 client.26717 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560f2a90cde0 con 0x560f2ab8f820 2023-10-26T06:56:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/4164510679 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6ac03a020 con 0x560f2ab8f820 2023-10-26T06:56:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.592+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/4164510679 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6ac00fa60 con 0x560f2ab8f820 2023-10-26T06:56:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6d4a05b80 10 client.26717 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6d4a05b80 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6d4a05b80 20 client.26717 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2aba8ba0 0x560f2abaafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6d4a05b80 4 client.26717 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:56:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f2ab5e8b0 con 0x560f2aba8ba0 2023-10-26T06:56:49.621 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2aba8ba0 0x560f2abaafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2aba8ba0 0x560f2abaafc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e96c2bf7d3eb812 server_cookie=a10a2fb4c35dda0d in_seq=0 out_seq=0 2023-10-26T06:56:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.593+0000 7fc6a1ffb700 10 client.26717 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.594+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/4164510679 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f2ab5e8b0 con 0x560f2aba8ba0 2023-10-26T06:56:49.622 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.594+0000 7fc6a1ffb700 10 client.26717 handle_command_reply: tid=0 2023-10-26T06:56:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.594+0000 7fc6d4a05b80 1 client.26717 shutdown 2023-10-26T06:56:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 msgr2=0x560f2ab9b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9b6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0098e0 tx=0x7fc6ac00be00 comp rx=0 tx=0).stop 2023-10-26T06:56:49.623 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 shutdown_connections 2023-10-26T06:56:49.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2aba8ba0 0x560f2abaafc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9b6d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.624 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/4164510679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2ab98a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 >> 172.21.15.89:0/4164510679 conn(0x560f2ab03720 msgr2=0x560f2ab8e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6a1ffb700 0 client.26717 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 shutdown_connections 2023-10-26T06:56:49.625 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/4164510679 wait complete. 2023-10-26T06:56:49.626 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.595+0000 7fc6d4a05b80 20 client.26717 trim_cache size 0 max 16384 2023-10-26T06:56:49.629 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 Processor -- start 2023-10-26T06:56:49.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 -- start start 2023-10-26T06:56:49.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab94210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab94d60 0x560f2aba0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.630 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab95680 0x560f2ab9b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2a962920 con 0x560f2ab94d60 2023-10-26T06:56:49.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2ab997a0 con 0x560f2ab8f820 2023-10-26T06:56:49.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.612+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2ab99fc0 con 0x560f2ab95680 2023-10-26T06:56:49.631 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab94210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab94210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34944/0 (socket says 172.21.15.89:34944) 2023-10-26T06:56:49.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 -- 172.21.15.89:0/2841524017 learned_addr learned my addr 172.21.15.89:0/2841524017 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab95680 0x560f2ab9b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.632 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab94d60 0x560f2aba0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 -- 172.21.15.89:0/2841524017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab95680 msgr2=0x560f2ab9b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab95680 0x560f2ab9b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 -- 172.21.15.89:0/2841524017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab94d60 msgr2=0x560f2aba0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.633 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab94d60 0x560f2aba0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 -- 172.21.15.89:0/2841524017 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f2a8f5c60 con 0x560f2ab8f820 2023-10-26T06:56:49.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab94210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009ce0 tx=0x7fc6ac000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d54d6c0a46ff51 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.634 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.613+0000 7fc6a97fa700 1 -- 172.21.15.89:0/2841524017 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac01d070 con 0x560f2ab8f820 2023-10-26T06:56:49.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6a97fa700 1 -- 172.21.15.89:0/2841524017 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6ac03a070 con 0x560f2ab8f820 2023-10-26T06:56:49.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6a97fa700 1 -- 172.21.15.89:0/2841524017 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac00f6a0 con 0x560f2ab8f820 2023-10-26T06:56:49.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/2841524017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 msgr2=0x560f2ab94210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.635 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab94210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009ce0 tx=0x7fc6ac000bc0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/2841524017 shutdown_connections 2023-10-26T06:56:49.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab95680 0x560f2ab9b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab94210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.636 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/2841524017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab94d60 0x560f2aba0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/2841524017 >> 172.21.15.89:0/2841524017 conn(0x560f2ab03720 msgr2=0x560f2ab93b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/2841524017 shutdown_connections 2023-10-26T06:56:49.637 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.614+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/2841524017 wait complete. 2023-10-26T06:56:49.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 -- start start 2023-10-26T06:56:49.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 Processor -- start 2023-10-26T06:56:49.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.638 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2aba0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab9e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2ab9ea00 con 0x560f2ab95680 2023-10-26T06:56:49.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2ab9eb70 con 0x560f2ab8f820 2023-10-26T06:56:49.639 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2ab9ed30 con 0x560f2ab94d60 2023-10-26T06:56:49.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6d4a05b80 10 client.0 _ll_register_callbacks cb 0x560f2a9c8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:56:49.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.615+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:34948/0 (socket says 172.21.15.89:34948) 2023-10-26T06:56:49.640 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1821171594 learned_addr learned my addr 172.21.15.89:0/1821171594 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:49.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2aba0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab9e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1821171594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 msgr2=0x560f2aba0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.641 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2aba0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1821171594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 msgr2=0x560f2ab9e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:56:49.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab9e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.642 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 -- 172.21.15.89:0/1821171594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f2a90cde0 con 0x560f2ab8f820 2023-10-26T06:56:49.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6bad9d700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab9e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:56:49.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6ba59c700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9fc30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac02ce70 tx=0x7fc6ac02cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3655d16e9efad079 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:49.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:56:49.643 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1821171594 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac01d030 con 0x560f2ab8f820 2023-10-26T06:56:49.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.616+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6ac01e040 con 0x560f2ab8f820 2023-10-26T06:56:49.644 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.617+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1821171594 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fc6ac000e40 con 0x560f2ab8f820 2023-10-26T06:56:49.646 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1821171594 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ac02c500 con 0x560f2ab8f820 2023-10-26T06:56:49.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 10 client.26723 fetch_fsmap learned FSMap version 20 2023-10-26T06:56:49.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560f2a933250 con 0x560f2ab8f820 2023-10-26T06:56:49.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1821171594 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6ac01e040 con 0x560f2ab8f820 2023-10-26T06:56:49.647 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1821171594 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6ac004aa0 con 0x560f2ab8f820 2023-10-26T06:56:49.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 10 client.26723 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:56:49.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:56:49.648 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 20 client.26723 populate_metadata read hostname 'smithi089' 2023-10-26T06:56:49.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2abfd890 0x560f2abffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:49.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 4 client.26723 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:56:49.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.631+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f2aadae50 con 0x560f2abfd890 2023-10-26T06:56:49.649 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.632+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2abfd890 0x560f2abffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:49.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.632+0000 7fc6b9d9b700 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2abfd890 0x560f2abffcb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39bb214e72238ab5 server_cookie=e044a32ff35c3268 in_seq=0 out_seq=0 2023-10-26T06:56:49.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.632+0000 7fc6a1ffb700 10 client.26723 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.632+0000 7fc6a1ffb700 1 -- 172.21.15.89:0/1821171594 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560f2aadae50 con 0x560f2abfd890 2023-10-26T06:56:49.650 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.632+0000 7fc6a1ffb700 10 client.26723 handle_command_reply: tid=0 2023-10-26T06:56:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.632+0000 7fc6d4a05b80 1 client.26723 shutdown 2023-10-26T06:56:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 msgr2=0x560f2ab9fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9fc30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac02ce70 tx=0x7fc6ac02cf50 comp rx=0 tx=0).stop 2023-10-26T06:56:49.651 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 shutdown_connections 2023-10-26T06:56:49.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2ab95680 0x560f2ab9e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560f2abfd890 0x560f2abffcb0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.652 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f2ab8f820 0x560f2ab9fc30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 --2- 172.21.15.89:0/1821171594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560f2ab94d60 0x560f2aba0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6a1ffb700 0 client.26723 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:56:49.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 >> 172.21.15.89:0/1821171594 conn(0x560f2ab03720 msgr2=0x560f2ab83700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.653 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 shutdown_connections 2023-10-26T06:56:49.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 1 -- 172.21.15.89:0/1821171594 wait complete. 2023-10-26T06:56:49.654 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.633+0000 7fc6d4a05b80 20 client.26723 trim_cache size 0 max 16384 2023-10-26T06:56:49.654 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:56:49.655 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-26T06:56:49.655 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2023-10-26T06:56:49.655 INFO:teuthology.orchestra.run.smithi089.stdout: "e77066ea-c4ac-4455-b83f-93c6f5faac8c": { 2023-10-26T06:56:49.655 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2023-10-26T06:56:49.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "e77066ea-c4ac-4455-b83f-93c6f5faac8c", 2023-10-26T06:56:49.656 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2023-10-26T06:56:49.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:56:49.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:56:49.657 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:56:49.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc6a40a1670 msgr2=0x7fc6a40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.658 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc6a40a1670 0x7fc6a40a3b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x560f2ab95ae0 tx=0x7fc6b00099b0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 msgr2=0x7fc6bc198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:49.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 0x7fc6bc198a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400efc0 tx=0x7fc6b400c3a0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 shutdown_connections 2023-10-26T06:56:49.659 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc6a40a1670 0x7fc6a40a3b30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6bc1027e0 0x7fc6bc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6bc1087f0 0x7fc6bc198a50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.660 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 --2- 172.21.15.89:0/3051383097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6bc1031a0 0x7fc6bc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:49.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 >> 172.21.15.89:0/3051383097 conn(0x7fc6bc0fe290 msgr2=0x7fc6bc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:49.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 shutdown_connections 2023-10-26T06:56:49.661 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:49.634+0000 7fc6bb59e700 1 -- 172.21.15.89:0/3051383097 wait complete. 2023-10-26T06:56:49.662 INFO:tasks.cephfs.filesystem:scrub status for tag:e77066ea-c4ac-4455-b83f-93c6f5faac8c - {'path': '/', 'tag': 'e77066ea-c4ac-4455-b83f-93c6f5faac8c', 'options': 'recursive,force'} 2023-10-26T06:56:49.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:49 smithi089 ceph-mon[95031]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:49.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:49 smithi192 ceph-mon[131661]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:49.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:49 smithi135 ceph-mon[131403]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:50.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:50 smithi192 ceph-mon[131661]: scrub queued for path: / 2023-10-26T06:56:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:50 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:50 smithi192 ceph-mon[131661]: scrub summary: active paths [/] 2023-10-26T06:56:50.896 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:50 smithi192 ceph-mon[131661]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:50.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:50 smithi135 ceph-mon[131403]: scrub queued for path: / 2023-10-26T06:56:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:50 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:50 smithi135 ceph-mon[131403]: scrub summary: active paths [/] 2023-10-26T06:56:50.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:50 smithi135 ceph-mon[131403]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:51.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:50 smithi089 ceph-mon[95031]: scrub queued for path: / 2023-10-26T06:56:51.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:50 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:51.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:50 smithi089 ceph-mon[95031]: scrub summary: active paths [/] 2023-10-26T06:56:51.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:50 smithi089 ceph-mon[95031]: scrub summary: idle+waiting paths [/] 2023-10-26T06:56:51.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:51 smithi192 ceph-mon[131661]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:51.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:51 smithi135 ceph-mon[131403]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:52.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:51 smithi089 ceph-mon[95031]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:56:53.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-26T06:56:53.895 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:53 smithi192 ceph-mon[131661]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:56:53.922 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:53 smithi089 ceph-mon[95031]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:56:53.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:53 smithi135 ceph-mon[131403]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:56:54.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 -- 172.21.15.89:0/4064393523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c067fd0 msgr2=0x7f587c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 --2- 172.21.15.89:0/4064393523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c067fd0 0x7f587c0683b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f586c0099c0 tx=0x7f586c009cd0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 -- 172.21.15.89:0/4064393523 shutdown_connections 2023-10-26T06:56:54.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 --2- 172.21.15.89:0/4064393523 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c105ae0 0x7f587c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 --2- 172.21.15.89:0/4064393523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c067fd0 0x7f587c0683b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 --2- 172.21.15.89:0/4064393523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c105120 0x7f587c1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 -- 172.21.15.89:0/4064393523 >> 172.21.15.89:0/4064393523 conn(0x7f587c076790 msgr2=0x7f587c076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:54.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 -- 172.21.15.89:0/4064393523 shutdown_connections 2023-10-26T06:56:54.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.059+0000 7f58821ff700 1 -- 172.21.15.89:0/4064393523 wait complete. 2023-10-26T06:56:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.060+0000 7f58821ff700 1 Processor -- start 2023-10-26T06:56:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.060+0000 7f58821ff700 1 -- start start 2023-10-26T06:56:54.065 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f58821ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c067fd0 0x7f587c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f58821ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c105120 0x7f587c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f58821ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c105ae0 0x7f587c196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.066 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f58821ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c196ff0 con 0x7f587c105120 2023-10-26T06:56:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f58821ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c197160 con 0x7f587c067fd0 2023-10-26T06:56:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f58821ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c197320 con 0x7f587c105ae0 2023-10-26T06:56:54.067 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c067fd0 0x7f587c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c067fd0 0x7f587c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:43214/0 (socket says 172.21.15.89:43214) 2023-10-26T06:56:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 -- 172.21.15.89:0/4166524711 learned_addr learned my addr 172.21.15.89:0/4166524711 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587affd700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c105120 0x7f587c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.068 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587bfff700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c105ae0 0x7f587c196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 -- 172.21.15.89:0/4166524711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c105ae0 msgr2=0x7f587c196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c105ae0 0x7f587c196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 -- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c105120 msgr2=0x7f587c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.069 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c105120 0x7f587c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 -- 172.21.15.89:0/4166524711 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c0095f0 con 0x7f587c067fd0 2023-10-26T06:56:54.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.061+0000 7f587b7fe700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c067fd0 0x7f587c19c860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f586c005b40 tx=0x7f586c00b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:54.070 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.062+0000 7f5878ff9700 1 -- 172.21.15.89:0/4166524711 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c01d070 con 0x7f587c067fd0 2023-10-26T06:56:54.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.062+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c1975a0 con 0x7f587c067fd0 2023-10-26T06:56:54.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.062+0000 7f5878ff9700 1 -- 172.21.15.89:0/4166524711 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f586c00be00 con 0x7f587c067fd0 2023-10-26T06:56:54.071 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.062+0000 7f5878ff9700 1 -- 172.21.15.89:0/4166524711 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c00fb00 con 0x7f587c067fd0 2023-10-26T06:56:54.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.062+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587c1a3850 con 0x7f587c067fd0 2023-10-26T06:56:54.072 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.063+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587c108bd0 con 0x7f587c067fd0 2023-10-26T06:56:54.073 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.065+0000 7f5878ff9700 1 -- 172.21.15.89:0/4166524711 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f586c003680 con 0x7f587c067fd0 2023-10-26T06:56:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.066+0000 7f5878ff9700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58640a15f0 0x7f58640a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.066+0000 7f5878ff9700 1 -- 172.21.15.89:0/4166524711 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f586c0ce0c0 con 0x7f587c067fd0 2023-10-26T06:56:54.074 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.066+0000 7f587affd700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58640a15f0 0x7f58640a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.067+0000 7f587affd700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58640a15f0 0x7f58640a3ab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f587c197b50 tx=0x7f5870009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:54.075 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.068+0000 7f5878ff9700 1 -- 172.21.15.89:0/4166524711 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f586c097420 con 0x7f587c067fd0 2023-10-26T06:56:54.313 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.312+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f587c0676e0 con 0x7f58640a15f0 2023-10-26T06:56:54.322 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.321+0000 7f5878ff9700 1 -- 172.21.15.89:0/4166524711 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f587c0676e0 con 0x7f58640a15f0 2023-10-26T06:56:54.323 INFO:teuthology.orchestra.run.smithi089.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-26T00:00:00", "created": "2023-10-26T06:46:52", "first": "2023-10-26T06:47:00", "last": "2023-10-26T06:56:00", "last_pruned": "2023-10-26T06:56:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-10-26T06:56:54.325 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58640a15f0 msgr2=0x7f58640a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58640a15f0 0x7f58640a3ab0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f587c197b50 tx=0x7f5870009260 comp rx=0 tx=0).stop 2023-10-26T06:56:54.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c067fd0 msgr2=0x7f587c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.326 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c067fd0 0x7f587c19c860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f586c005b40 tx=0x7f586c00b9c0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 shutdown_connections 2023-10-26T06:56:54.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f58640a15f0 0x7f58640a3ab0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.327 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f587c105ae0 0x7f587c196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f587c067fd0 0x7f587c19c860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 --2- 172.21.15.89:0/4166524711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c105120 0x7f587c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 >> 172.21.15.89:0/4166524711 conn(0x7f587c076790 msgr2=0x7f587c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:54.328 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 shutdown_connections 2023-10-26T06:56:54.329 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.324+0000 7f58821ff700 1 -- 172.21.15.89:0/4166524711 wait complete. 2023-10-26T06:56:54.346 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-26T06:56:54.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 -- 172.21.15.89:0/38110285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550067fd0 msgr2=0x7f25500683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.547 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 --2- 172.21.15.89:0/38110285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550067fd0 0x7f25500683b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2540009920 tx=0x7f2540009c30 comp rx=0 tx=0).stop 2023-10-26T06:56:54.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 -- 172.21.15.89:0/38110285 shutdown_connections 2023-10-26T06:56:54.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 --2- 172.21.15.89:0/38110285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550067fd0 0x7f25500683b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 --2- 172.21.15.89:0/38110285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2550105ae0 0x7f255010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.548 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 --2- 172.21.15.89:0/38110285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550105120 0x7f25501055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 -- 172.21.15.89:0/38110285 >> 172.21.15.89:0/38110285 conn(0x7f2550076790 msgr2=0x7f2550076ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:54.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 -- 172.21.15.89:0/38110285 shutdown_connections 2023-10-26T06:56:54.549 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.545+0000 7f25557ff700 1 -- 172.21.15.89:0/38110285 wait complete. 2023-10-26T06:56:54.551 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.546+0000 7f25557ff700 1 Processor -- start 2023-10-26T06:56:54.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.546+0000 7f25557ff700 1 -- start start 2023-10-26T06:56:54.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f25557ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550067fd0 0x7f2550198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f25557ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550105120 0x7f2550199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.552 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f25557ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2550105ae0 0x7f255019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f25557ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255019dd60 con 0x7f2550105120 2023-10-26T06:56:54.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f25557ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255019ded0 con 0x7f2550067fd0 2023-10-26T06:56:54.553 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f25557ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255019e090 con 0x7f2550105ae0 2023-10-26T06:56:54.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550067fd0 0x7f2550198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254f7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2550105ae0 0x7f255019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.554 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550067fd0 0x7f2550198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:43230/0 (socket says 172.21.15.89:43230) 2023-10-26T06:56:54.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254e7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550105120 0x7f2550199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 -- 172.21.15.89:0/855434939 learned_addr learned my addr 172.21.15.89:0/855434939 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:56:54.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 -- 172.21.15.89:0/855434939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2550105ae0 msgr2=0x7f255019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.555 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2550105ae0 0x7f255019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 -- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550105120 msgr2=0x7f2550199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550105120 0x7f2550199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.556 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.547+0000 7f254effd700 1 -- 172.21.15.89:0/855434939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2540009520 con 0x7f2550067fd0 2023-10-26T06:56:54.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.548+0000 7f254f7fe700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2550105ae0 0x7f255019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:56:54.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.548+0000 7f254effd700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550067fd0 0x7f2550198e40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2540009230 tx=0x7f254000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:54.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.548+0000 7f2537fff700 1 -- 172.21.15.89:0/855434939 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254001c070 con 0x7f2550067fd0 2023-10-26T06:56:54.557 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.548+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255019e310 con 0x7f2550067fd0 2023-10-26T06:56:54.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.548+0000 7f2537fff700 1 -- 172.21.15.89:0/855434939 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254000baa0 con 0x7f2550067fd0 2023-10-26T06:56:54.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.548+0000 7f2537fff700 1 -- 172.21.15.89:0/855434939 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540004930 con 0x7f2550067fd0 2023-10-26T06:56:54.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.548+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a5bb0 con 0x7f2550067fd0 2023-10-26T06:56:54.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.549+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2550101df0 con 0x7f2550067fd0 2023-10-26T06:56:54.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.551+0000 7f2537fff700 1 -- 172.21.15.89:0/855434939 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2540020070 con 0x7f2550067fd0 2023-10-26T06:56:54.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.552+0000 7f2537fff700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25380a15f0 0x7f25380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:56:54.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.552+0000 7f2537fff700 1 -- 172.21.15.89:0/855434939 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f25400cdbf0 con 0x7f2550067fd0 2023-10-26T06:56:54.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.553+0000 7f254e7fc700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25380a15f0 0x7f25380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:56:54.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.553+0000 7f254e7fc700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25380a15f0 0x7f25380a3ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f255019a390 tx=0x7f2544009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:56:54.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.555+0000 7f2537fff700 1 -- 172.21.15.89:0/855434939 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2540096f50 con 0x7f2550067fd0 2023-10-26T06:56:54.794 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.793+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 --> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f255019a140 con 0x7f25380a15f0 2023-10-26T06:56:54.801 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.800+0000 7f2537fff700 1 -- 172.21.15.89:0/855434939 <== mgr.14217 v2:172.21.15.89:6800/1106080795 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f255019a140 con 0x7f25380a15f0 2023-10-26T06:56:54.801 INFO:teuthology.orchestra.run.smithi089.stdout:/ 1m 6m3h 2023-10-26T06:56:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25380a15f0 msgr2=0x7f25380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25380a15f0 0x7f25380a3ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f255019a390 tx=0x7f2544009260 comp rx=0 tx=0).stop 2023-10-26T06:56:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550067fd0 msgr2=0x7f2550198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:56:54.804 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550067fd0 0x7f2550198e40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2540009230 tx=0x7f254000fa00 comp rx=0 tx=0).stop 2023-10-26T06:56:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 shutdown_connections 2023-10-26T06:56:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f25380a15f0 0x7f25380a3ab0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2550105ae0 0x7f255019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.805 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2550067fd0 0x7f2550198e40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 --2- 172.21.15.89:0/855434939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2550105120 0x7f2550199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:56:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 >> 172.21.15.89:0/855434939 conn(0x7f2550076790 msgr2=0x7f25500fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:56:54.806 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 shutdown_connections 2023-10-26T06:56:54.807 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:56:54.802+0000 7f25557ff700 1 -- 172.21.15.89:0/855434939 wait complete. 2023-10-26T06:56:54.823 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-26T06:56:54.833 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-26T06:56:54.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:54 smithi135 ceph-mon[131403]: scrub summary: idle 2023-10-26T06:56:55.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:54 smithi089 ceph-mon[95031]: scrub summary: idle 2023-10-26T06:56:55.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:54 smithi192 ceph-mon[131661]: scrub summary: idle 2023-10-26T06:56:55.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:55 smithi135 ceph-mon[131403]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:56:55.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:55 smithi135 ceph-mon[131403]: from='client.26729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:56:56.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:55 smithi089 ceph-mon[95031]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:56:56.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:55 smithi089 ceph-mon[95031]: from='client.26729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:56:56.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:55 smithi192 ceph-mon[131661]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:56:56.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:55 smithi192 ceph-mon[131661]: from='client.26729 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:56:56.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:56 smithi135 ceph-mon[131403]: from='client.26735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:56:57.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:56 smithi089 ceph-mon[95031]: from='client.26735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:56:57.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:56 smithi192 ceph-mon[131661]: from='client.26735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-26T06:56:57.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:57 smithi135 ceph-mon[131403]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:58.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:57 smithi089 ceph-mon[95031]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:58.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:57 smithi192 ceph-mon[131661]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:56:58.967 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:56:58 smithi135 ceph-mon[131403]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2023-10-26T06:56:59.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:56:58 smithi089 ceph-mon[95031]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2023-10-26T06:56:59.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:56:58 smithi192 ceph-mon[131661]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2023-10-26T06:57:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:01 smithi192 ceph-mon[131661]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2023-10-26T06:57:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:01 smithi135 ceph-mon[131403]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2023-10-26T06:57:01.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:01 smithi089 ceph-mon[95031]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2023-10-26T06:57:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:03 smithi192 ceph-mon[131661]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2023-10-26T06:57:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:03 smithi135 ceph-mon[131403]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2023-10-26T06:57:03.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:03 smithi089 ceph-mon[95031]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2023-10-26T06:57:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:05 smithi192 ceph-mon[131661]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-10-26T06:57:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:05 smithi135 ceph-mon[131403]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-10-26T06:57:05.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:05 smithi089 ceph-mon[95031]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-10-26T06:57:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:07 smithi192 ceph-mon[131661]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-26T06:57:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:07 smithi135 ceph-mon[131403]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-26T06:57:07.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:07 smithi089 ceph-mon[95031]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-26T06:57:09.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:09 smithi192 ceph-mon[131661]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-26T06:57:09.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:09 smithi135 ceph-mon[131403]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-26T06:57:09.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:09 smithi089 ceph-mon[95031]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-26T06:57:10.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:10 smithi192 ceph-mon[131661]: osdmap e95: 12 total, 12 up, 12 in 2023-10-26T06:57:10.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:10 smithi135 ceph-mon[131403]: osdmap e95: 12 total, 12 up, 12 in 2023-10-26T06:57:10.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:10 smithi089 ceph-mon[95031]: osdmap e95: 12 total, 12 up, 12 in 2023-10-26T06:57:11.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:11 smithi192 ceph-mon[131661]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:57:11.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:11 smithi135 ceph-mon[131403]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:57:11.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:11 smithi089 ceph-mon[95031]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:57:13.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:13 smithi192 ceph-mon[131661]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:57:13.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:13 smithi135 ceph-mon[131403]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:57:13.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:13 smithi089 ceph-mon[95031]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 19 op/s 2023-10-26T06:57:15.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:15 smithi192 ceph-mon[131661]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 27 KiB/s wr, 5 op/s 2023-10-26T06:57:15.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:15 smithi135 ceph-mon[131403]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 27 KiB/s wr, 5 op/s 2023-10-26T06:57:15.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:15 smithi089 ceph-mon[95031]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 27 KiB/s wr, 5 op/s 2023-10-26T06:57:17.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:17 smithi192 ceph-mon[131661]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:57:17.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:17 smithi135 ceph-mon[131403]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:57:17.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:17 smithi089 ceph-mon[95031]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:57:19.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:19 smithi192 ceph-mon[131661]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:57:19.663 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-26T06:57:19.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:19 smithi135 ceph-mon[131403]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:57:19.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:19 smithi089 ceph-mon[95031]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2023-10-26T06:57:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 -- 172.21.15.89:0/4050260152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 msgr2=0x7fd610106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 --2- 172.21.15.89:0/4050260152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 0x7fd610106b10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009990 tx=0x7fd60c009ca0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 -- 172.21.15.89:0/4050260152 shutdown_connections 2023-10-26T06:57:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 --2- 172.21.15.89:0/4050260152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 0x7fd61010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 --2- 172.21.15.89:0/4050260152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 0x7fd610106b10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 --2- 172.21.15.89:0/4050260152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd610100720 0x7fd610100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 -- 172.21.15.89:0/4050260152 >> 172.21.15.89:0/4050260152 conn(0x7fd6100fc1b0 msgr2=0x7fd6100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 -- 172.21.15.89:0/4050260152 shutdown_connections 2023-10-26T06:57:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.869+0000 7fd617fff700 1 -- 172.21.15.89:0/4050260152 wait complete. 2023-10-26T06:57:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.870+0000 7fd617fff700 1 Processor -- start 2023-10-26T06:57:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd617fff700 1 -- start start 2023-10-26T06:57:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd617fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd610100720 0x7fd610198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd617fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 0x7fd6101992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd617fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 0x7fd61019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd617fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019dcc0 con 0x7fd610100720 2023-10-26T06:57:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd617fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019de30 con 0x7fd610106730 2023-10-26T06:57:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd617fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019dff0 con 0x7fd6101010e0 2023-10-26T06:57:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd61559a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 0x7fd6101992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd610100720 0x7fd610198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd61559a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 0x7fd6101992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:48302/0 (socket says 172.21.15.89:48302) 2023-10-26T06:57:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd61559a700 1 -- 172.21.15.89:0/3991846304 learned_addr learned my addr 172.21.15.89:0/3991846304 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.871+0000 7fd61659c700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 0x7fd61019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd615d9b700 1 -- 172.21.15.89:0/3991846304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 msgr2=0x7fd6101992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd615d9b700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 0x7fd6101992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd615d9b700 1 -- 172.21.15.89:0/3991846304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 msgr2=0x7fd61019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd615d9b700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 0x7fd61019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd615d9b700 1 -- 172.21.15.89:0/3991846304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c0095f0 con 0x7fd610100720 2023-10-26T06:57:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd61559a700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 0x7fd6101992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:57:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd615d9b700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd610100720 0x7fd610198da0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0098e0 tx=0x7fd60c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd606ffd700 1 -- 172.21.15.89:0/3991846304 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01d070 con 0x7fd610100720 2023-10-26T06:57:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd606ffd700 1 -- 172.21.15.89:0/3991846304 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60c022470 con 0x7fd610100720 2023-10-26T06:57:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61019e270 con 0x7fd610100720 2023-10-26T06:57:19.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd606ffd700 1 -- 172.21.15.89:0/3991846304 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017650 con 0x7fd610100720 2023-10-26T06:57:19.883 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.872+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6101a5b10 con 0x7fd610100720 2023-10-26T06:57:19.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.875+0000 7fd606ffd700 1 -- 172.21.15.89:0/3991846304 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd60c0225e0 con 0x7fd610100720 2023-10-26T06:57:19.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.877+0000 7fd606ffd700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5fc0a15f0 0x7fd5fc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.877+0000 7fd606ffd700 1 -- 172.21.15.89:0/3991846304 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8543+0+0 (secure 0 0 0) 0x7fd60c0ce420 con 0x7fd610100720 2023-10-26T06:57:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.877+0000 7fd61559a700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5fc0a15f0 0x7fd5fc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.877+0000 7fd62b9e1b80 1 Processor -- start 2023-10-26T06:57:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd61559a700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5fc0a15f0 0x7fd5fc0a3ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd61019a2f0 tx=0x7fd600009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd62b9e1b80 1 -- start start 2023-10-26T06:57:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837fbe0 0x55bc9837ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc9837a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc9837aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc982ed6e0 con 0x55bc98380500 2023-10-26T06:57:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc982d3c60 con 0x55bc9837a5d0 2023-10-26T06:57:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc982d1580 con 0x55bc9837fbe0 2023-10-26T06:57:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837fbe0 0x55bc9837ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837fbe0 0x55bc9837ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33412/0 (socket says 172.21.15.89:33412) 2023-10-26T06:57:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd615d9b700 1 -- 172.21.15.89:0/2035705729 learned_addr learned my addr 172.21.15.89:0/2035705729 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd61559a700 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc9837a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.878+0000 7fd61659c700 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc9837aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd615d9b700 1 -- 172.21.15.89:0/2035705729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 msgr2=0x55bc9837aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd615d9b700 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc9837aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd615d9b700 1 -- 172.21.15.89:0/2035705729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 msgr2=0x55bc9837a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd615d9b700 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc9837a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd615d9b700 1 -- 172.21.15.89:0/2035705729 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9828a3f0 con 0x55bc9837fbe0 2023-10-26T06:57:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd615d9b700 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837fbe0 0x55bc9837ffc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd60c006010 tx=0x7fd60c0977e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed14e44e38e1adde server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd5fbfff700 1 -- 172.21.15.89:0/2035705729 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c09bb20 con 0x55bc9837fbe0 2023-10-26T06:57:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd5fbfff700 1 -- 172.21.15.89:0/2035705729 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60c09b420 con 0x55bc9837fbe0 2023-10-26T06:57:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.879+0000 7fd5fbfff700 1 -- 172.21.15.89:0/2035705729 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c097c90 con 0x55bc9837fbe0 2023-10-26T06:57:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2035705729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837fbe0 msgr2=0x55bc9837ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837fbe0 0x55bc9837ffc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd60c006010 tx=0x7fd60c0977e0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2035705729 shutdown_connections 2023-10-26T06:57:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc9837aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837fbe0 0x55bc9837ffc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2035705729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc9837a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2035705729 >> 172.21.15.89:0/2035705729 conn(0x55bc982ee720 msgr2=0x55bc98374b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2035705729 shutdown_connections 2023-10-26T06:57:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2035705729 wait complete. 2023-10-26T06:57:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.880+0000 7fd62b9e1b80 1 -- start start 2023-10-26T06:57:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 1 Processor -- start 2023-10-26T06:57:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc98386480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc98383770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc98380500 0x55bc98383eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc982cce70 con 0x55bc9837fbe0 2023-10-26T06:57:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc98269710 con 0x55bc9837a5d0 2023-10-26T06:57:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc982693b0 con 0x55bc98380500 2023-10-26T06:57:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd62b9e1b80 10 client.0 _ll_register_callbacks cb 0x55bc981b3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:57:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc98386480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd61559a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc98383770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd61659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc98380500 0x55bc98383eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc98386480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52274/0 (socket says 172.21.15.89:52274) 2023-10-26T06:57:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.881+0000 7fd615d9b700 1 -- 172.21.15.89:0/3075083759 learned_addr learned my addr 172.21.15.89:0/3075083759 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd615d9b700 1 -- 172.21.15.89:0/3075083759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc98380500 msgr2=0x55bc98383eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd615d9b700 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc98380500 0x55bc98383eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd615d9b700 1 -- 172.21.15.89:0/3075083759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 msgr2=0x55bc98383770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd615d9b700 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc98383770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd615d9b700 1 -- 172.21.15.89:0/3075083759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6101a5b10 con 0x55bc9837a5d0 2023-10-26T06:57:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd615d9b700 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc98386480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000c00 tx=0x7fd60c022b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63653da189cf2362 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:19.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd5f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:57:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd5f97fa700 1 -- 172.21.15.89:0/3075083759 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c09bb20 con 0x55bc9837a5d0 2023-10-26T06:57:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd5f97fa700 1 -- 172.21.15.89:0/3075083759 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8543+0+0 (secure 0 0 0) 0x7fd60c09f330 con 0x55bc9837a5d0 2023-10-26T06:57:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc982d1ef0 con 0x55bc9837a5d0 2023-10-26T06:57:19.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.882+0000 7fd5f97fa700 1 -- 172.21.15.89:0/3075083759 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd60c09f570 con 0x55bc9837a5d0 2023-10-26T06:57:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 10 client.26747 fetch_fsmap learned FSMap version 20 2023-10-26T06:57:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x55bc980f7de0 con 0x55bc9837a5d0 2023-10-26T06:57:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd5f97fa700 1 -- 172.21.15.89:0/3075083759 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc982d1ef0 con 0x55bc9837a5d0 2023-10-26T06:57:19.906 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd5f97fa700 1 -- 172.21.15.89:0/3075083759 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd60c09a980 con 0x55bc9837a5d0 2023-10-26T06:57:19.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 10 client.26747 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:57:19.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:57:19.907 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 20 client.26747 populate_metadata read hostname 'smithi089' 2023-10-26T06:57:19.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983936b0 0x55bc98395ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 4 client.26747 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:57:19.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc983498b0 con 0x55bc983936b0 2023-10-26T06:57:19.908 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.883+0000 7fd61559a700 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983936b0 0x55bc98395ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.884+0000 7fd5f97fa700 10 client.26747 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:19.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.884+0000 7fd61559a700 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983936b0 0x55bc98395ad0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e525bf6a8debda0 server_cookie=2c2e3f4e6fc428cf in_seq=0 out_seq=0 2023-10-26T06:57:19.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.884+0000 7fd5f97fa700 1 -- 172.21.15.89:0/3075083759 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc983498b0 con 0x55bc983936b0 2023-10-26T06:57:19.909 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.884+0000 7fd5f97fa700 10 client.26747 handle_command_reply: tid=0 2023-10-26T06:57:19.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 client.26747 shutdown 2023-10-26T06:57:19.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 msgr2=0x55bc98386480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc98386480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000c00 tx=0x7fd60c022b50 comp rx=0 tx=0).stop 2023-10-26T06:57:19.910 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 shutdown_connections 2023-10-26T06:57:19.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983936b0 0x55bc98395ad0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc98380500 0x55bc98383eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.911 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837a5d0 0x55bc98386480 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd5f97fa700 0 client.26747 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/3075083759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc98383770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 >> 172.21.15.89:0/3075083759 conn(0x55bc982ee720 msgr2=0x55bc98379080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:19.912 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 shutdown_connections 2023-10-26T06:57:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/3075083759 wait complete. 2023-10-26T06:57:19.913 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.885+0000 7fd62b9e1b80 20 client.26747 trim_cache size 0 max 16384 2023-10-26T06:57:19.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.903+0000 7fd62b9e1b80 1 Processor -- start 2023-10-26T06:57:19.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd62b9e1b80 1 -- start start 2023-10-26T06:57:19.918 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc9836f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc9838ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc98380500 0x55bc98379930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.919 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9814d920 con 0x55bc9837fbe0 2023-10-26T06:57:19.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9837b030 con 0x55bc98380500 2023-10-26T06:57:19.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc98384d70 con 0x55bc9837a5d0 2023-10-26T06:57:19.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd61659c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc98380500 0x55bc98379930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.920 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd61559a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc9838ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc9836f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd61659c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc98380500 0x55bc98379930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52276/0 (socket says 172.21.15.89:52276) 2023-10-26T06:57:19.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd61659c700 1 -- 172.21.15.89:0/2495008950 learned_addr learned my addr 172.21.15.89:0/2495008950 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:19.921 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd61659c700 1 -- 172.21.15.89:0/2495008950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 msgr2=0x55bc9836f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.904+0000 7fd61659c700 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc9836f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd61659c700 1 -- 172.21.15.89:0/2495008950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 msgr2=0x55bc9838ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd61659c700 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc9838ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd61659c700 1 -- 172.21.15.89:0/2495008950 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc98374d70 con 0x55bc98380500 2023-10-26T06:57:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd61559a700 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc9838ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:57:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd615d9b700 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc9836f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:57:19.923 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd61659c700 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc98380500 0x55bc98379930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd608012930 tx=0x7fd608009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac7042054a59914 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:19.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd604ff9700 1 -- 172.21.15.89:0/2495008950 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800e040 con 0x55bc98380500 2023-10-26T06:57:19.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd604ff9700 1 -- 172.21.15.89:0/2495008950 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60800f040 con 0x55bc98380500 2023-10-26T06:57:19.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.905+0000 7fd604ff9700 1 -- 172.21.15.89:0/2495008950 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080079d0 con 0x55bc98380500 2023-10-26T06:57:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2495008950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc98380500 msgr2=0x55bc98379930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc98380500 0x55bc98379930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd608012930 tx=0x7fd608009920 comp rx=0 tx=0).stop 2023-10-26T06:57:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2495008950 shutdown_connections 2023-10-26T06:57:19.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc98380500 0x55bc98379930 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc9836f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/2495008950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc9837fbe0 0x55bc9838ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2495008950 >> 172.21.15.89:0/2495008950 conn(0x55bc982ee720 msgr2=0x55bc98379080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:19.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2495008950 shutdown_connections 2023-10-26T06:57:19.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/2495008950 wait complete. 2023-10-26T06:57:19.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 -- start start 2023-10-26T06:57:19.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.906+0000 7fd62b9e1b80 1 Processor -- start 2023-10-26T06:57:19.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc98389310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837fbe0 0x55bc98389850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd62b9e1b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc98389d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc983e74f0 con 0x55bc98380500 2023-10-26T06:57:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc983e7660 con 0x55bc9837fbe0 2023-10-26T06:57:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd62b9e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc983e7820 con 0x55bc9837a5d0 2023-10-26T06:57:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd62b9e1b80 10 client.0 _ll_register_callbacks cb 0x55bc981b3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:57:19.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc98389310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc98389310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33446/0 (socket says 172.21.15.89:33446) 2023-10-26T06:57:19.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 -- 172.21.15.89:0/465854052 learned_addr learned my addr 172.21.15.89:0/465854052 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:19.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd61559a700 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837fbe0 0x55bc98389850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd61659c700 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc98389d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 -- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837fbe0 msgr2=0x55bc98389850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837fbe0 0x55bc98389850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.931 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 -- 172.21.15.89:0/465854052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 msgr2=0x55bc98389d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc98389d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 -- 172.21.15.89:0/465854052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc980f7de0 con 0x55bc9837a5d0 2023-10-26T06:57:19.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd615d9b700 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc98389310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009990 tx=0x7fd60c09ab40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8011a3540bec47e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.907+0000 7fd5f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:57:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd5f97fa700 1 -- 172.21.15.89:0/465854052 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c09bb20 con 0x55bc9837a5d0 2023-10-26T06:57:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd60c09b420 con 0x55bc9837a5d0 2023-10-26T06:57:19.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd5f97fa700 1 -- 172.21.15.89:0/465854052 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8543+0+0 (secure 0 0 0) 0x7fd60c09f330 con 0x55bc9837a5d0 2023-10-26T06:57:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd5f97fa700 1 -- 172.21.15.89:0/465854052 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd60c009290 con 0x55bc9837a5d0 2023-10-26T06:57:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 10 client.26746 fetch_fsmap learned FSMap version 20 2023-10-26T06:57:19.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fd600017070 con 0x55bc9837a5d0 2023-10-26T06:57:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd5f97fa700 1 -- 172.21.15.89:0/465854052 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60c09b420 con 0x55bc9837a5d0 2023-10-26T06:57:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd5f97fa700 1 -- 172.21.15.89:0/465854052 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd60c09b680 con 0x55bc9837a5d0 2023-10-26T06:57:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 10 client.26746 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:57:19.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:57:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 20 client.26746 populate_metadata read hostname 'smithi089' 2023-10-26T06:57:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983e8b50 0x55bc983eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:19.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 4 client.26746 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-26T06:57:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.908+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bc982c5e50 con 0x55bc983e8b50 2023-10-26T06:57:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd61559a700 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983e8b50 0x55bc983eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd61559a700 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983e8b50 0x55bc983eaf70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac6340a31ec6d622 server_cookie=127f358bf12c7318 in_seq=0 out_seq=0 2023-10-26T06:57:19.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd5f97fa700 10 client.26746 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd5f97fa700 1 -- 172.21.15.89:0/465854052 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bc982c5e50 con 0x55bc983e8b50 2023-10-26T06:57:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd5f97fa700 10 client.26746 handle_command_reply: tid=0 2023-10-26T06:57:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 client.26746 shutdown 2023-10-26T06:57:19.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 msgr2=0x55bc98389310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc98389310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009990 tx=0x7fd60c09ab40 comp rx=0 tx=0).stop 2023-10-26T06:57:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 shutdown_connections 2023-10-26T06:57:19.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bc98380500 0x55bc98389d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x55bc983e8b50 0x55bc983eaf70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc9837a5d0 0x55bc98389310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 --2- 172.21.15.89:0/465854052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9837fbe0 0x55bc98389850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 >> 172.21.15.89:0/465854052 conn(0x55bc982ee720 msgr2=0x55bc9836e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:19.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd5f97fa700 0 client.26746 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:19.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.909+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 shutdown_connections 2023-10-26T06:57:19.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd62b9e1b80 1 -- 172.21.15.89:0/465854052 wait complete. 2023-10-26T06:57:19.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd62b9e1b80 20 client.26746 trim_cache size 0 max 16384 2023-10-26T06:57:19.942 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:57:19.942 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2023-10-26T06:57:19.943 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2023-10-26T06:57:19.943 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:57:19.944 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5fc0a15f0 msgr2=0x7fd5fc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5fc0a15f0 0x7fd5fc0a3ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd61019a2f0 tx=0x7fd600009260 comp rx=0 tx=0).stop 2023-10-26T06:57:19.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd610100720 msgr2=0x7fd610198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:19.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd610100720 0x7fd610198da0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0098e0 tx=0x7fd60c005740 comp rx=0 tx=0).stop 2023-10-26T06:57:19.945 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 shutdown_connections 2023-10-26T06:57:19.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd5fc0a15f0 0x7fd5fc0a3ab0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd610100720 0x7fd610198da0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.946 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd610106730 0x7fd61019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 --2- 172.21.15.89:0/3991846304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6101010e0 0x7fd6101992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:19.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 >> 172.21.15.89:0/3991846304 conn(0x7fd6100fc1b0 msgr2=0x7fd61010be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:19.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 shutdown_connections 2023-10-26T06:57:19.947 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:19.910+0000 7fd617fff700 1 -- 172.21.15.89:0/3991846304 wait complete. 2023-10-26T06:57:19.948 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-26T06:57:19.948 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:57:20.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 -- 172.21.15.89:0/2231897945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 msgr2=0x7fe8f4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/2231897945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4106a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009990 tx=0x7fe8f0009ca0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 -- 172.21.15.89:0/2231897945 shutdown_connections 2023-10-26T06:57:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/2231897945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8f4101060 0x7fe8f410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/2231897945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4106a90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/2231897945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8f41006a0 0x7fe8f4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 -- 172.21.15.89:0/2231897945 >> 172.21.15.89:0/2231897945 conn(0x7fe8f40fc170 msgr2=0x7fe8f40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 -- 172.21.15.89:0/2231897945 shutdown_connections 2023-10-26T06:57:20.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.149+0000 7fe8fbbff700 1 -- 172.21.15.89:0/2231897945 wait complete. 2023-10-26T06:57:20.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.150+0000 7fe8fbbff700 1 Processor -- start 2023-10-26T06:57:20.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.150+0000 7fe8fbbff700 1 -- start start 2023-10-26T06:57:20.155 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.150+0000 7fe8fbbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8f41006a0 0x7fe8f4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.150+0000 7fe8fbbff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8f4101060 0x7fe8f4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.150+0000 7fe8fbbff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.150+0000 7fe8fbbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f41a4e80 con 0x7fe8f41006a0 2023-10-26T06:57:20.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.150+0000 7fe8fbbff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f41a4ff0 con 0x7fe8f41066b0 2023-10-26T06:57:20.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fbbff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f41a51b0 con 0x7fe8f4101060 2023-10-26T06:57:20.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8f999b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8f41006a0 0x7fe8f4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8f919a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8f4101060 0x7fe8f4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52292/0 (socket says 172.21.15.89:52292) 2023-10-26T06:57:20.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 -- 172.21.15.89:0/1900107023 learned_addr learned my addr 172.21.15.89:0/1900107023 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:20.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 -- 172.21.15.89:0/1900107023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8f4101060 msgr2=0x7fe8f4073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8f4101060 0x7fe8f4073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 -- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8f41006a0 msgr2=0x7fe8f4072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.159 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8f41006a0 0x7fe8f4072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 -- 172.21.15.89:0/1900107023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f00095f0 con 0x7fe8f41066b0 2023-10-26T06:57:20.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8fa19c700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4073570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00ef30 tx=0x7fe8ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.151+0000 7fe8eaffd700 1 -- 172.21.15.89:0/1900107023 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec018070 con 0x7fe8f41066b0 2023-10-26T06:57:20.160 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.152+0000 7fe8eaffd700 1 -- 172.21.15.89:0/1900107023 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8ec00f040 con 0x7fe8f41066b0 2023-10-26T06:57:20.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.152+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f41a5430 con 0x7fe8f41066b0 2023-10-26T06:57:20.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.152+0000 7fe8eaffd700 1 -- 172.21.15.89:0/1900107023 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00cd00 con 0x7fe8f41066b0 2023-10-26T06:57:20.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.152+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f41a58f0 con 0x7fe8f41066b0 2023-10-26T06:57:20.161 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.153+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f4108c30 con 0x7fe8f41066b0 2023-10-26T06:57:20.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.154+0000 7fe8eaffd700 1 -- 172.21.15.89:0/1900107023 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe8ec003a00 con 0x7fe8f41066b0 2023-10-26T06:57:20.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.156+0000 7fe8eaffd700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8e00a15f0 0x7fe8e00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.156+0000 7fe8eaffd700 1 -- 172.21.15.89:0/1900107023 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8543+0+0 (secure 0 0 0) 0x7fe8ec0ccdc0 con 0x7fe8f41066b0 2023-10-26T06:57:20.163 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.156+0000 7fe8f999b700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8e00a15f0 0x7fe8e00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.157+0000 7fe8f999b700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8e00a15f0 0x7fe8e00a3ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0006010 tx=0x7fe8f000b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.165 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.158+0000 7fe8eaffd700 1 -- 172.21.15.89:0/1900107023 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe8ec096100 con 0x7fe8f41066b0 2023-10-26T06:57:20.432 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.431+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8f4101de0 con 0x7fe8f41066b0 2023-10-26T06:57:20.433 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.432+0000 7fe8eaffd700 1 -- 172.21.15.89:0/1900107023 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7fe8ec099720 con 0x7fe8f41066b0 2023-10-26T06:57:20.433 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:57:20.434 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:57:20.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8e00a15f0 msgr2=0x7fe8e00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.436 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8e00a15f0 0x7fe8e00a3ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0006010 tx=0x7fe8f000b260 comp rx=0 tx=0).stop 2023-10-26T06:57:20.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 msgr2=0x7fe8f4073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4073570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00ef30 tx=0x7fe8ec00c3a0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.437 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 shutdown_connections 2023-10-26T06:57:20.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fe8e00a15f0 0x7fe8e00a3ab0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe8f41006a0 0x7fe8f4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41066b0 0x7fe8f4073570 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.438 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 --2- 172.21.15.89:0/1900107023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8f4101060 0x7fe8f4073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 >> 172.21.15.89:0/1900107023 conn(0x7fe8f40fc170 msgr2=0x7fe8f40fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 shutdown_connections 2023-10-26T06:57:20.439 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.435+0000 7fe8fbbff700 1 -- 172.21.15.89:0/1900107023 wait complete. 2023-10-26T06:57:20.440 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:57:20.455 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-26T06:57:20.664 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 -- 172.21.15.89:0/2474278372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c111730 msgr2=0x7fd82c113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 --2- 172.21.15.89:0/2474278372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c111730 0x7fd82c113b20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd82400b0a0 tx=0x7fd82400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 -- 172.21.15.89:0/2474278372 shutdown_connections 2023-10-26T06:57:20.665 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 --2- 172.21.15.89:0/2474278372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c111730 0x7fd82c113b20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 --2- 172.21.15.89:0/2474278372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c072b90 0x7fd82c072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 --2- 172.21.15.89:0/2474278372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82c0734b0 0x7fd82c1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.666 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 -- 172.21.15.89:0/2474278372 >> 172.21.15.89:0/2474278372 conn(0x7fd82c0fc1b0 msgr2=0x7fd82c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 -- 172.21.15.89:0/2474278372 shutdown_connections 2023-10-26T06:57:20.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.663+0000 7fd8347ff700 1 -- 172.21.15.89:0/2474278372 wait complete. 2023-10-26T06:57:20.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 Processor -- start 2023-10-26T06:57:20.669 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 -- start start 2023-10-26T06:57:20.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c072b90 0x7fd82c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c0734b0 0x7fd82c19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82c111730 0x7fd82c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.670 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c1a2290 con 0x7fd82c111730 2023-10-26T06:57:20.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c1a2400 con 0x7fd82c072b90 2023-10-26T06:57:20.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.664+0000 7fd8347ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c1a25c0 con 0x7fd82c0734b0 2023-10-26T06:57:20.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c072b90 0x7fd82c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c072b90 0x7fd82c19d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52310/0 (socket says 172.21.15.89:52310) 2023-10-26T06:57:20.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 -- 172.21.15.89:0/399485700 learned_addr learned my addr 172.21.15.89:0/399485700 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:20.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd831d9a700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c0734b0 0x7fd82c19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd832d9c700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82c111730 0x7fd82c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 -- 172.21.15.89:0/399485700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c0734b0 msgr2=0x7fd82c19d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c0734b0 0x7fd82c19d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 -- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82c111730 msgr2=0x7fd82c1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82c111730 0x7fd82c1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 -- 172.21.15.89:0/399485700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824009d00 con 0x7fd82c072b90 2023-10-26T06:57:20.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd83259b700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c072b90 0x7fd82c19d370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82800e9a0 tx=0x7fd82800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd8237fe700 1 -- 172.21.15.89:0/399485700 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828009790 con 0x7fd82c072b90 2023-10-26T06:57:20.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.665+0000 7fd8237fe700 1 -- 172.21.15.89:0/399485700 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82800cbd0 con 0x7fd82c072b90 2023-10-26T06:57:20.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.666+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c1a28a0 con 0x7fd82c072b90 2023-10-26T06:57:20.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.666+0000 7fd8237fe700 1 -- 172.21.15.89:0/399485700 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828018680 con 0x7fd82c072b90 2023-10-26T06:57:20.675 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.666+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c1aa0e0 con 0x7fd82c072b90 2023-10-26T06:57:20.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.669+0000 7fd8237fe700 1 -- 172.21.15.89:0/399485700 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd82800cd40 con 0x7fd82c072b90 2023-10-26T06:57:20.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 Processor -- start 2023-10-26T06:57:20.682 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 -- start start 2023-10-26T06:57:20.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df5be0 0x560d47df5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df05d0 0x560d47df0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.683 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47d636e0 con 0x560d47df5be0 2023-10-26T06:57:20.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47d49c60 con 0x560d47df6500 2023-10-26T06:57:20.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd84807ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47d47580 con 0x560d47df05d0 2023-10-26T06:57:20.684 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd83259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.670+0000 7fd83259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52312/0 (socket says 172.21.15.89:52312) 2023-10-26T06:57:20.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd83259b700 1 -- 172.21.15.89:0/1556623012 learned_addr learned my addr 172.21.15.89:0/1556623012 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:20.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd832d9c700 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df5be0 0x560d47df5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.685 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd831d9a700 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df05d0 0x560d47df0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd8237fe700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd8180a1670 0x7fd8180a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd8237fe700 1 -- 172.21.15.89:0/399485700 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fd828014070 con 0x7fd82c072b90 2023-10-26T06:57:20.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd831d9a700 1 -- 172.21.15.89:0/1556623012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 msgr2=0x560d47df0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.686 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd831d9a700 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd831d9a700 1 -- 172.21.15.89:0/1556623012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df5be0 msgr2=0x560d47df5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd831d9a700 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df5be0 0x560d47df5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd831d9a700 1 -- 172.21.15.89:0/1556623012 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d47d003f0 con 0x560d47df05d0 2023-10-26T06:57:20.687 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd832d9c700 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df5be0 0x560d47df5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:57:20.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd832d9c700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd8180a1670 0x7fd8180a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd831d9a700 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df05d0 0x560d47df0a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd81c005950 tx=0x7fd81c00bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635df236cc1b956c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.671+0000 7fd820ff9700 1 -- 172.21.15.89:0/1556623012 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c010c40 con 0x560d47df05d0 2023-10-26T06:57:20.688 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd820ff9700 1 -- 172.21.15.89:0/1556623012 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81c015070 con 0x560d47df05d0 2023-10-26T06:57:20.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd820ff9700 1 -- 172.21.15.89:0/1556623012 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c005020 con 0x560d47df05d0 2023-10-26T06:57:20.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 -- 172.21.15.89:0/1556623012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df05d0 msgr2=0x560d47df0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd832d9c700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd8180a1670 0x7fd8180a3b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd824009850 tx=0x7fd8240099e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.689 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df05d0 0x560d47df0a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd81c005950 tx=0x7fd81c00bee0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 -- 172.21.15.89:0/1556623012 shutdown_connections 2023-10-26T06:57:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df5be0 0x560d47df5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df05d0 0x560d47df0a50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.690 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1556623012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 -- 172.21.15.89:0/1556623012 >> 172.21.15.89:0/1556623012 conn(0x560d47d64720 msgr2=0x560d47deab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.672+0000 7fd84807ab80 1 -- 172.21.15.89:0/1556623012 shutdown_connections 2023-10-26T06:57:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.673+0000 7fd84807ab80 1 -- 172.21.15.89:0/1556623012 wait complete. 2023-10-26T06:57:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.673+0000 7fd84807ab80 1 -- start start 2023-10-26T06:57:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.673+0000 7fd84807ab80 1 Processor -- start 2023-10-26T06:57:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47dfc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47df9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.693 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd84807ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47d42e70 con 0x560d47df05d0 2023-10-26T06:57:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd84807ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47cdf710 con 0x560d47df6500 2023-10-26T06:57:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd84807ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47cdf3b0 con 0x560d47df5be0 2023-10-26T06:57:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd84807ab80 10 client.0 _ll_register_callbacks cb 0x560d47c29a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:57:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47df9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd83259b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47dfc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47df9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33500/0 (socket says 172.21.15.89:33500) 2023-10-26T06:57:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 -- 172.21.15.89:0/15574483 learned_addr learned my addr 172.21.15.89:0/15574483 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:20.696 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd832d9c700 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 -- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 msgr2=0x560d47df9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.697 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 -- 172.21.15.89:0/15574483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 msgr2=0x560d47dfc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47dfc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.674+0000 7fd831d9a700 1 -- 172.21.15.89:0/15574483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c00b860 con 0x560d47df5be0 2023-10-26T06:57:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd831d9a700 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47df9860 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd81c000c00 tx=0x7fd81c010c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60c1a5c00fccd4fe server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.698 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd815ffb700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2023-10-26T06:57:20.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd815ffb700 1 -- 172.21.15.89:0/15574483 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c020070 con 0x560d47df5be0 2023-10-26T06:57:20.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d47d47ef0 con 0x560d47df5be0 2023-10-26T06:57:20.699 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd815ffb700 1 -- 172.21.15.89:0/15574483 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fd81c017070 con 0x560d47df5be0 2023-10-26T06:57:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd815ffb700 1 -- 172.21.15.89:0/15574483 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd81c005ce0 con 0x560d47df5be0 2023-10-26T06:57:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd84807ab80 10 client.26764 fetch_fsmap learned FSMap version 20 2023-10-26T06:57:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x560d47b6dde0 con 0x560d47df5be0 2023-10-26T06:57:20.700 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.675+0000 7fd815ffb700 1 -- 172.21.15.89:0/15574483 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81c031080 con 0x560d47df5be0 2023-10-26T06:57:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd815ffb700 1 -- 172.21.15.89:0/15574483 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd81c011440 con 0x560d47df5be0 2023-10-26T06:57:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd84807ab80 10 client.26764 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:57:20.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd84807ab80 10 client.26764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:57:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd84807ab80 20 client.26764 populate_metadata read hostname 'smithi089' 2023-10-26T06:57:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd84807ab80 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e09d30 0x560d47e0c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd84807ab80 4 client.26764 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T06:57:20.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d47dbf8b0 con 0x560d47e09d30 2023-10-26T06:57:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd83259b700 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e09d30 0x560d47e0c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd815ffb700 10 client.26764 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:20.703 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.676+0000 7fd83259b700 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e09d30 0x560d47e0c150 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0a95a285a1c6e3 server_cookie=2b383d19c5866580 in_seq=0 out_seq=0 2023-10-26T06:57:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd815ffb700 1 -- 172.21.15.89:0/15574483 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560d47dbf8b0 con 0x560d47e09d30 2023-10-26T06:57:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd815ffb700 10 client.26764 handle_command_reply: tid=0 2023-10-26T06:57:20.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 client.26764 shutdown 2023-10-26T06:57:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 msgr2=0x560d47df9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47df9860 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd81c000c00 tx=0x7fd81c010c70 comp rx=0 tx=0).stop 2023-10-26T06:57:20.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 shutdown_connections 2023-10-26T06:57:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47dfc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e09d30 0x560d47e0c150 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df6500 0x560d47df9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 --2- 172.21.15.89:0/15574483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47df9860 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd815ffb700 0 client.26764 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.677+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 >> 172.21.15.89:0/15574483 conn(0x560d47d64720 msgr2=0x560d47def080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.678+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 shutdown_connections 2023-10-26T06:57:20.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.678+0000 7fd84807ab80 1 -- 172.21.15.89:0/15574483 wait complete. 2023-10-26T06:57:20.708 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.678+0000 7fd84807ab80 20 client.26764 trim_cache size 0 max 16384 2023-10-26T06:57:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.692+0000 7fd84807ab80 1 Processor -- start 2023-10-26T06:57:20.713 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.692+0000 7fd84807ab80 1 -- start start 2023-10-26T06:57:20.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47df51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df5be0 0x560d47de55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.714 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df6500 0x560d47e00eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd84807ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47bc3920 con 0x560d47df05d0 2023-10-26T06:57:20.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd84807ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47debe20 con 0x560d47df5be0 2023-10-26T06:57:20.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd84807ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47dfae10 con 0x560d47df6500 2023-10-26T06:57:20.715 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47df51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47df51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48004/0 (socket says 172.21.15.89:48004) 2023-10-26T06:57:20.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd831d9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df5be0 0x560d47de55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 -- 172.21.15.89:0/1480008902 learned_addr learned my addr 172.21.15.89:0/1480008902 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:20.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd832d9c700 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df6500 0x560d47e00eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 -- 172.21.15.89:0/1480008902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df6500 msgr2=0x560d47e00eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df6500 0x560d47e00eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.717 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 -- 172.21.15.89:0/1480008902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df5be0 msgr2=0x560d47de55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df5be0 0x560d47de55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd83259b700 1 -- 172.21.15.89:0/1480008902 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d47b56c60 con 0x560d47df05d0 2023-10-26T06:57:20.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd832d9c700 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df6500 0x560d47e00eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:57:20.718 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.693+0000 7fd831d9a700 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df5be0 0x560d47de55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:57:20.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd83259b700 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47df51b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd8280967c0 tx=0x7fd8280cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a7867df040aa85 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd8217fa700 1 -- 172.21.15.89:0/1480008902 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828099c20 con 0x560d47df05d0 2023-10-26T06:57:20.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd8217fa700 1 -- 172.21.15.89:0/1480008902 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8280b0020 con 0x560d47df05d0 2023-10-26T06:57:20.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd8217fa700 1 -- 172.21.15.89:0/1480008902 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82809a520 con 0x560d47df05d0 2023-10-26T06:57:20.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd84807ab80 1 -- 172.21.15.89:0/1480008902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 msgr2=0x560d47df51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47df51b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd8280967c0 tx=0x7fd8280cf040 comp rx=0 tx=0).stop 2023-10-26T06:57:20.720 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd84807ab80 1 -- 172.21.15.89:0/1480008902 shutdown_connections 2023-10-26T06:57:20.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df05d0 0x560d47df51b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df6500 0x560d47e00eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.721 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd84807ab80 1 --2- 172.21.15.89:0/1480008902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df5be0 0x560d47de55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.694+0000 7fd84807ab80 1 -- 172.21.15.89:0/1480008902 >> 172.21.15.89:0/1480008902 conn(0x560d47d64720 msgr2=0x560d47deaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.695+0000 7fd84807ab80 1 -- 172.21.15.89:0/1480008902 shutdown_connections 2023-10-26T06:57:20.722 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.695+0000 7fd84807ab80 1 -- 172.21.15.89:0/1480008902 wait complete. 2023-10-26T06:57:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.695+0000 7fd84807ab80 1 -- start start 2023-10-26T06:57:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.695+0000 7fd84807ab80 1 Processor -- start 2023-10-26T06:57:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df05d0 0x560d47e01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47dff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd84807ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df6500 0x560d47dffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd84807ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47e00350 con 0x560d47df6500 2023-10-26T06:57:20.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd84807ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47e5db70 con 0x560d47df05d0 2023-10-26T06:57:20.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd84807ab80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d47e5dcb0 con 0x560d47df5be0 2023-10-26T06:57:20.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd84807ab80 10 client.0 _ll_register_callbacks cb 0x560d47c29a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T06:57:20.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df05d0 0x560d47e01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df05d0 0x560d47e01090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52340/0 (socket says 172.21.15.89:52340) 2023-10-26T06:57:20.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 -- 172.21.15.89:0/2009118735 learned_addr learned my addr 172.21.15.89:0/2009118735 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:20.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd831d9a700 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47dff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd832d9c700 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df6500 0x560d47dffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 -- 172.21.15.89:0/2009118735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 msgr2=0x560d47dff530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47dff530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 -- 172.21.15.89:0/2009118735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df6500 msgr2=0x560d47dffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T06:57:20.727 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df6500 0x560d47dffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 -- 172.21.15.89:0/2009118735 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d47b6dde0 con 0x560d47df05d0 2023-10-26T06:57:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd83259b700 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df05d0 0x560d47e01090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd828099fc0 tx=0x7fd8280a26c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1a8fbea33ca6a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.696+0000 7fd815ffb700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2023-10-26T06:57:20.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd815ffb700 1 -- 172.21.15.89:0/2009118735 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82809a9c0 con 0x560d47df05d0 2023-10-26T06:57:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd815ffb700 1 -- 172.21.15.89:0/2009118735 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fd8280b1070 con 0x560d47df05d0 2023-10-26T06:57:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd81c015070 con 0x560d47df05d0 2023-10-26T06:57:20.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd815ffb700 1 -- 172.21.15.89:0/2009118735 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82809a2c0 con 0x560d47df05d0 2023-10-26T06:57:20.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 10 client.26777 fetch_fsmap learned FSMap version 20 2023-10-26T06:57:20.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x7fd82c1aa0e0 con 0x560d47df05d0 2023-10-26T06:57:20.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd815ffb700 1 -- 172.21.15.89:0/2009118735 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd828082020 con 0x560d47df05d0 2023-10-26T06:57:20.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd815ffb700 1 -- 172.21.15.89:0/2009118735 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd8280a21c0 con 0x560d47df05d0 2023-10-26T06:57:20.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 10 client.26777 fetch_fsmap finished waiting for FSMap version 20 2023-10-26T06:57:20.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 10 client.26777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-26T06:57:20.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 20 client.26777 populate_metadata read hostname 'smithi089' 2023-10-26T06:57:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e5f050 0x560d47e61470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 4 client.26777 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-26T06:57:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.697+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 --> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d47d3be50 con 0x560d47e5f050 2023-10-26T06:57:20.732 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd831d9a700 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e5f050 0x560d47e61470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd815ffb700 10 client.26777 ms_handle_connect on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd831d9a700 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e5f050 0x560d47e61470 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb81d8f9a5a4e3f server_cookie=ea534c075bff932a in_seq=0 out_seq=0 2023-10-26T06:57:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd815ffb700 1 -- 172.21.15.89:0/2009118735 <== mds.0 v2:172.21.15.135:6838/609246385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d47d3be50 con 0x560d47e5f050 2023-10-26T06:57:20.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd815ffb700 10 client.26777 handle_command_reply: tid=0 2023-10-26T06:57:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 client.26777 shutdown 2023-10-26T06:57:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df05d0 msgr2=0x560d47e01090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.734 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df05d0 0x560d47e01090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd828099fc0 tx=0x7fd8280a26c0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 shutdown_connections 2023-10-26T06:57:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560d47df6500 0x560d47dffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:6838/609246385,v1:172.21.15.135:6839/609246385] conn(0x560d47e5f050 0x560d47e61470 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d47df05d0 0x560d47e01090 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 --2- 172.21.15.89:0/2009118735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d47df5be0 0x560d47dff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd815ffb700 0 client.26777 ms_handle_reset on v2:172.21.15.135:6838/609246385 2023-10-26T06:57:20.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.698+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 >> 172.21.15.89:0/2009118735 conn(0x560d47d64720 msgr2=0x560d47df09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 shutdown_connections 2023-10-26T06:57:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd84807ab80 1 -- 172.21.15.89:0/2009118735 wait complete. 2023-10-26T06:57:20.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd84807ab80 20 client.26777 trim_cache size 0 max 16384 2023-10-26T06:57:20.738 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2023-10-26T06:57:20.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd8180a1670 msgr2=0x7fd8180a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd8180a1670 0x7fd8180a3b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd824009850 tx=0x7fd8240099e0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c072b90 msgr2=0x7fd82c19d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:20.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c072b90 0x7fd82c19d370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82800e9a0 tx=0x7fd82800ecb0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.740 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 shutdown_connections 2023-10-26T06:57:20.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd8180a1670 0x7fd8180a3b30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82c111730 0x7fd82c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd82c072b90 0x7fd82c19d370 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 --2- 172.21.15.89:0/399485700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c0734b0 0x7fd82c19d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:20.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.699+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 >> 172.21.15.89:0/399485700 conn(0x7fd82c0fc1b0 msgr2=0x7fd82c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:20.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.700+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 shutdown_connections 2023-10-26T06:57:20.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:20.700+0000 7fd8347ff700 1 -- 172.21.15.89:0/399485700 wait complete. 2023-10-26T06:57:21.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:21 smithi192 ceph-mon[131661]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 192 B/s wr, 2 op/s 2023-10-26T06:57:21.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:21 smithi192 ceph-mon[131661]: osdmap e96: 12 total, 12 up, 12 in 2023-10-26T06:57:21.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:21 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1900107023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:57:21.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:21 smithi135 ceph-mon[131403]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 192 B/s wr, 2 op/s 2023-10-26T06:57:21.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:21 smithi135 ceph-mon[131403]: osdmap e96: 12 total, 12 up, 12 in 2023-10-26T06:57:21.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:21 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1900107023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:57:21.743 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-10-26T06:57:21.744 INFO:tasks.fwd_scrub:done joining 2023-10-26T06:57:21.745 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-26T06:57:21.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:57:21.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:21 smithi089 ceph-mon[95031]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 192 B/s wr, 2 op/s 2023-10-26T06:57:21.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:21 smithi089 ceph-mon[95031]: osdmap e96: 12 total, 12 up, 12 in 2023-10-26T06:57:21.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:21 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1900107023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:57:21.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 -- 172.21.15.89:0/791152004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 msgr2=0x7fd250106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 --2- 172.21.15.89:0/791152004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 0x7fd250106ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b0a0 tx=0x7fd24400b3b0 comp rx=0 tx=0).stop 2023-10-26T06:57:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 -- 172.21.15.89:0/791152004 shutdown_connections 2023-10-26T06:57:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 --2- 172.21.15.89:0/791152004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2501010b0 0x7fd25010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 --2- 172.21.15.89:0/791152004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 0x7fd250106ae0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 --2- 172.21.15.89:0/791152004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 0x7fd250100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 -- 172.21.15.89:0/791152004 >> 172.21.15.89:0/791152004 conn(0x7fd2500fc1a0 msgr2=0x7fd2500fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 -- 172.21.15.89:0/791152004 shutdown_connections 2023-10-26T06:57:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.963+0000 7fd256fff700 1 -- 172.21.15.89:0/791152004 wait complete. 2023-10-26T06:57:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.964+0000 7fd256fff700 1 Processor -- start 2023-10-26T06:57:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.964+0000 7fd256fff700 1 -- start start 2023-10-26T06:57:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.964+0000 7fd256fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 0x7fd250198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd256fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2501010b0 0x7fd2501992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd256fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 0x7fd25019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd256fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019cb40 con 0x7fd2501010b0 2023-10-26T06:57:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd256fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019ccb0 con 0x7fd2501006f0 2023-10-26T06:57:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd256fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019ce70 con 0x7fd250106700 2023-10-26T06:57:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 0x7fd250198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 0x7fd250198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52348/0 (socket says 172.21.15.89:52348) 2023-10-26T06:57:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2501010b0 0x7fd2501992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:21.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 -- 172.21.15.89:0/760282844 learned_addr learned my addr 172.21.15.89:0/760282844 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:21.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd25559c700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 0x7fd25019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:21.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 -- 172.21.15.89:0/760282844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 msgr2=0x7fd25019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:21.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 0x7fd25019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:21.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 -- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2501010b0 msgr2=0x7fd2501992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:21.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2501010b0 0x7fd2501992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:21.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 -- 172.21.15.89:0/760282844 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244009d00 con 0x7fd2501006f0 2023-10-26T06:57:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.965+0000 7fd254d9b700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 0x7fd250198d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b070 tx=0x7fd244007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.966+0000 7fd24dffb700 1 -- 172.21.15.89:0/760282844 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24400e040 con 0x7fd2501006f0 2023-10-26T06:57:21.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.966+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25019d0f0 con 0x7fd2501006f0 2023-10-26T06:57:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.966+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2501a5ab0 con 0x7fd2501006f0 2023-10-26T06:57:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.966+0000 7fd24dffb700 1 -- 172.21.15.89:0/760282844 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd24400be20 con 0x7fd2501006f0 2023-10-26T06:57:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.966+0000 7fd24dffb700 1 -- 172.21.15.89:0/760282844 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24401b460 con 0x7fd2501006f0 2023-10-26T06:57:21.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.967+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd250101e30 con 0x7fd2501006f0 2023-10-26T06:57:21.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.969+0000 7fd24dffb700 1 -- 172.21.15.89:0/760282844 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd24401d470 con 0x7fd2501006f0 2023-10-26T06:57:21.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.970+0000 7fd24dffb700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2400a1670 0x7fd2400a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:21.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.970+0000 7fd24dffb700 1 -- 172.21.15.89:0/760282844 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fd2440cdfe0 con 0x7fd2501006f0 2023-10-26T06:57:21.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.970+0000 7fd24ffff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2400a1670 0x7fd2400a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:21.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.971+0000 7fd24ffff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2400a1670 0x7fd2400a3b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd25019a220 tx=0x7fd23c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:21.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:21.972+0000 7fd24dffb700 1 -- 172.21.15.89:0/760282844 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd244097380 con 0x7fd2501006f0 2023-10-26T06:57:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.183+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd250199fd0 con 0x7fd2501006f0 2023-10-26T06:57:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.185+0000 7fd24dffb700 1 -- 172.21.15.89:0/760282844 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+22978 (secure 0 0 0) 0x7fd24409a9a0 con 0x7fd2501006f0 2023-10-26T06:57:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:57:22.187 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:57:20.318854+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:57:22.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2400a1670 msgr2=0x7fd2400a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.189 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2400a1670 0x7fd2400a3b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd25019a220 tx=0x7fd23c009260 comp rx=0 tx=0).stop 2023-10-26T06:57:22.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 msgr2=0x7fd250198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.190 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 0x7fd250198d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b070 tx=0x7fd244007870 comp rx=0 tx=0).stop 2023-10-26T06:57:22.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 shutdown_connections 2023-10-26T06:57:22.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd2400a1670 0x7fd2400a3b30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.191 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd250106700 0x7fd25019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2501006f0 0x7fd250198d60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 --2- 172.21.15.89:0/760282844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2501010b0 0x7fd2501992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 >> 172.21.15.89:0/760282844 conn(0x7fd2500fc1a0 msgr2=0x7fd2500fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:22.192 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 shutdown_connections 2023-10-26T06:57:22.193 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.188+0000 7fd256fff700 1 -- 172.21.15.89:0/760282844 wait complete. 2023-10-26T06:57:22.209 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:57:22.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:57:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 -- 172.21.15.89:0/3996344386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 msgr2=0x7f82140ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 --2- 172.21.15.89:0/3996344386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f82140ff770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8210009990 tx=0x7f8210009ca0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 -- 172.21.15.89:0/3996344386 shutdown_connections 2023-10-26T06:57:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 --2- 172.21.15.89:0/3996344386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f821410cf40 0x7f821410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 --2- 172.21.15.89:0/3996344386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f82140ff770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 --2- 172.21.15.89:0/3996344386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 0x7f821410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 -- 172.21.15.89:0/3996344386 >> 172.21.15.89:0/3996344386 conn(0x7f82140fafb0 msgr2=0x7f82140fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 -- 172.21.15.89:0/3996344386 shutdown_connections 2023-10-26T06:57:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.453+0000 7f821b39d700 1 -- 172.21.15.89:0/3996344386 wait complete. 2023-10-26T06:57:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.454+0000 7f821b39d700 1 Processor -- start 2023-10-26T06:57:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821b39d700 1 -- start start 2023-10-26T06:57:22.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821b39d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f8214198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821b39d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 0x7f8214199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821b39d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f821410cf40 0x7f821419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821b39d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419cb30 con 0x7f821410cf40 2023-10-26T06:57:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821b39d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419cca0 con 0x7f82140ff390 2023-10-26T06:57:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821b39d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419ce60 con 0x7f82140ffd40 2023-10-26T06:57:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821a39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f8214198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f8219b9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 0x7f8214199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821ab9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f821410cf40 0x7f821419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821a39b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f8214198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52374/0 (socket says 172.21.15.89:52374) 2023-10-26T06:57:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f8219b9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 0x7f8214199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33562/0 (socket says 172.21.15.89:33562) 2023-10-26T06:57:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.455+0000 7f821a39b700 1 -- 172.21.15.89:0/732291193 learned_addr learned my addr 172.21.15.89:0/732291193 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821a39b700 1 -- 172.21.15.89:0/732291193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 msgr2=0x7f8214199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821a39b700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 0x7f8214199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821a39b700 1 -- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f821410cf40 msgr2=0x7f821419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821a39b700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f821410cf40 0x7f821419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821a39b700 1 -- 172.21.15.89:0/732291193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100095f0 con 0x7f82140ff390 2023-10-26T06:57:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f8219b9a700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 0x7f8214199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:57:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821a39b700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f8214198d00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8210000c00 tx=0x7f8210005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f820b7fe700 1 -- 172.21.15.89:0/732291193 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821001d070 con 0x7f82140ff390 2023-10-26T06:57:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f820b7fe700 1 -- 172.21.15.89:0/732291193 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8210022470 con 0x7f82140ff390 2023-10-26T06:57:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821419d0e0 con 0x7f82140ff390 2023-10-26T06:57:22.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f820b7fe700 1 -- 172.21.15.89:0/732291193 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210017650 con 0x7f82140ff390 2023-10-26T06:57:22.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.456+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a5aa0 con 0x7f82140ff390 2023-10-26T06:57:22.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.458+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214108160 con 0x7f82140ff390 2023-10-26T06:57:22.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.459+0000 7f820b7fe700 1 -- 172.21.15.89:0/732291193 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f82100225e0 con 0x7f82140ff390 2023-10-26T06:57:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.460+0000 7f820b7fe700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82000a1670 0x7f82000a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.461+0000 7f820b7fe700 1 -- 172.21.15.89:0/732291193 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f82100ce3c0 con 0x7f82140ff390 2023-10-26T06:57:22.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.461+0000 7f8219b9a700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82000a1670 0x7f82000a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.461+0000 7f8219b9a700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82000a1670 0x7f82000a3b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f821419a210 tx=0x7f8204009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:22.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.464+0000 7f820b7fe700 1 -- 172.21.15.89:0/732291193 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8210097780 con 0x7f82140ff390 2023-10-26T06:57:22.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:22 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/760282844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:57:22.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:22 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/760282844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:57:22.701 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.700+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82140673e0 con 0x7f82140ff390 2023-10-26T06:57:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.700+0000 7f820b7fe700 1 -- 172.21.15.89:0/732291193 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f821009ada0 con 0x7f82140ff390 2023-10-26T06:57:22.702 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:57:22.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82000a1670 msgr2=0x7f82000a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.704 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82000a1670 0x7f82000a3b30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f821419a210 tx=0x7f8204009260 comp rx=0 tx=0).stop 2023-10-26T06:57:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 msgr2=0x7f8214198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f8214198d00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8210000c00 tx=0x7f8210005740 comp rx=0 tx=0).stop 2023-10-26T06:57:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 shutdown_connections 2023-10-26T06:57:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f82000a1670 0x7f82000a3b30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f821410cf40 0x7f821419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82140ff390 0x7f8214198d00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 --2- 172.21.15.89:0/732291193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82140ffd40 0x7f8214199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 >> 172.21.15.89:0/732291193 conn(0x7f82140fafb0 msgr2=0x7f82140fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 shutdown_connections 2023-10-26T06:57:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.703+0000 7f821b39d700 1 -- 172.21.15.89:0/732291193 wait complete. 2023-10-26T06:57:22.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:22 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/760282844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:57:22.724 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:57:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 -- 172.21.15.89:0/1880274830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 msgr2=0x7feab01055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 --2- 172.21.15.89:0/1880274830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab01055f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feaa40099c0 tx=0x7feaa4009cd0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 -- 172.21.15.89:0/1880274830 shutdown_connections 2023-10-26T06:57:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 --2- 172.21.15.89:0/1880274830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab0105b30 0x7feab010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 --2- 172.21.15.89:0/1880274830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feab0067d40 0x7feab0068120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.929 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 --2- 172.21.15.89:0/1880274830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab01055f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 -- 172.21.15.89:0/1880274830 >> 172.21.15.89:0/1880274830 conn(0x7feab00767f0 msgr2=0x7feab0076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 -- 172.21.15.89:0/1880274830 shutdown_connections 2023-10-26T06:57:22.930 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.926+0000 7feab4fff700 1 -- 172.21.15.89:0/1880274830 wait complete. 2023-10-26T06:57:22.932 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.927+0000 7feab4fff700 1 Processor -- start 2023-10-26T06:57:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.927+0000 7feab4fff700 1 -- start start 2023-10-26T06:57:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.927+0000 7feab4fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab0067d40 0x7feab0198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.933 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feab4fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feab4fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feab0105b30 0x7feab019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feab4fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab019dcf0 con 0x7feab0067d40 2023-10-26T06:57:22.934 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feab4fff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab019de60 con 0x7feab0105b30 2023-10-26T06:57:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feab4fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab019e020 con 0x7feab0105170 2023-10-26T06:57:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab0067d40 0x7feab0198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.935 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33586/0 (socket says 172.21.15.89:33586) 2023-10-26T06:57:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaae59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab0067d40 0x7feab0198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48072/0 (socket says 172.21.15.89:48072) 2023-10-26T06:57:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 -- 172.21.15.89:0/962339133 learned_addr learned my addr 172.21.15.89:0/962339133 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:22.936 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaaed9d700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feab0105b30 0x7feab019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 -- 172.21.15.89:0/962339133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feab0105b30 msgr2=0x7feab019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feab0105b30 0x7feab019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 -- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab0067d40 msgr2=0x7feab0198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:22.937 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab0067d40 0x7feab0198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.928+0000 7feaadd9b700 1 -- 172.21.15.89:0/962339133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa40095f0 con 0x7feab0105170 2023-10-26T06:57:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.929+0000 7feaadd9b700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab0199310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005b40 tx=0x7feaa400b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.929+0000 7fea9f7fe700 1 -- 172.21.15.89:0/962339133 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa401d070 con 0x7feab0105170 2023-10-26T06:57:22.938 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.929+0000 7fea9f7fe700 1 -- 172.21.15.89:0/962339133 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa400be00 con 0x7feab0105170 2023-10-26T06:57:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.929+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab019e2a0 con 0x7feab0105170 2023-10-26T06:57:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.929+0000 7fea9f7fe700 1 -- 172.21.15.89:0/962339133 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa400fb00 con 0x7feab0105170 2023-10-26T06:57:22.939 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.929+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a5b40 con 0x7feab0105170 2023-10-26T06:57:22.940 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.930+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0101e40 con 0x7feab0105170 2023-10-26T06:57:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.932+0000 7fea9f7fe700 1 -- 172.21.15.89:0/962339133 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7feaa4003680 con 0x7feab0105170 2023-10-26T06:57:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.933+0000 7fea9f7fe700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea980a1670 0x7fea980a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.933+0000 7fea9f7fe700 1 -- 172.21.15.89:0/962339133 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7feaa40cdfe0 con 0x7feab0105170 2023-10-26T06:57:22.941 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.933+0000 7feaae59c700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea980a1670 0x7fea980a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.934+0000 7feaae59c700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea980a1670 0x7fea980a3b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009c60 tx=0x7feaa0009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:22.942 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:22.936+0000 7fea9f7fe700 1 -- 172.21.15.89:0/962339133 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feaa4097380 con 0x7feab0105170 2023-10-26T06:57:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.144+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feab019a0d0 con 0x7feab0105170 2023-10-26T06:57:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.145+0000 7fea9f7fe700 1 -- 172.21.15.89:0/962339133 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7feaa409a9a0 con 0x7feab0105170 2023-10-26T06:57:23.147 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:57:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea980a1670 msgr2=0x7fea980a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea980a1670 0x7fea980a3b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009c60 tx=0x7feaa0009260 comp rx=0 tx=0).stop 2023-10-26T06:57:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 msgr2=0x7feab0199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab0199310 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005b40 tx=0x7feaa400b9c0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 shutdown_connections 2023-10-26T06:57:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fea980a1670 0x7fea980a3b30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feab0067d40 0x7feab0198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feab0105b30 0x7feab019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 --2- 172.21.15.89:0/962339133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feab0105170 0x7feab0199310 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 >> 172.21.15.89:0/962339133 conn(0x7feab00767f0 msgr2=0x7feab0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 shutdown_connections 2023-10-26T06:57:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.147+0000 7feab4fff700 1 -- 172.21.15.89:0/962339133 wait complete. 2023-10-26T06:57:23.167 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:57:23.372 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.370+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/3122470379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf81010e0 msgr2=0x7f7bf810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.370+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/3122470379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf81010e0 0x7f7bf810d5f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b0a0 tx=0x7f7bf000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.371+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/3122470379 shutdown_connections 2023-10-26T06:57:23.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.371+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/3122470379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf81010e0 0x7f7bf810d5f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.373 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.371+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/3122470379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf8106730 0x7f7bf8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.371+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/3122470379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bf8100720 0x7f7bf8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.371+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/3122470379 >> 172.21.15.89:0/3122470379 conn(0x7f7bf80fc1b0 msgr2=0x7f7bf80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:23.374 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.371+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/3122470379 shutdown_connections 2023-10-26T06:57:23.375 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.371+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/3122470379 wait complete. 2023-10-26T06:57:23.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 Processor -- start 2023-10-26T06:57:23.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 -- start start 2023-10-26T06:57:23.377 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bf8100720 0x7f7bf819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf81010e0 0x7f7bf819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf8106730 0x7f7bf81969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.378 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8197030 con 0x7f7bf8100720 2023-10-26T06:57:23.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81971a0 con 0x7f7bf8106730 2023-10-26T06:57:23.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.372+0000 7f7bfe7ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8197360 con 0x7f7bf81010e0 2023-10-26T06:57:23.379 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf8106730 0x7f7bf81969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf8106730 0x7f7bf81969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:52424/0 (socket says 172.21.15.89:52424) 2023-10-26T06:57:23.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 -- 172.21.15.89:0/4214504118 learned_addr learned my addr 172.21.15.89:0/4214504118 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:23.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bf7fff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bf8100720 0x7f7bf819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.380 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bf77fe700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf81010e0 0x7f7bf819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 -- 172.21.15.89:0/4214504118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf81010e0 msgr2=0x7f7bf819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf81010e0 0x7f7bf819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.381 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 -- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bf8100720 msgr2=0x7f7bf819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bf8100720 0x7f7bf819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.382 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 -- 172.21.15.89:0/4214504118 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0009d00 con 0x7f7bf8106730 2023-10-26T06:57:23.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfcd9c700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf8106730 0x7f7bf81969d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b070 tx=0x7f7bf00077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:23.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bf57fa700 1 -- 172.21.15.89:0/4214504118 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000e040 con 0x7f7bf8106730 2023-10-26T06:57:23.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bf57fa700 1 -- 172.21.15.89:0/4214504118 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf00094c0 con 0x7f7bf8106730 2023-10-26T06:57:23.383 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf81975e0 con 0x7f7bf8106730 2023-10-26T06:57:23.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.373+0000 7f7bf57fa700 1 -- 172.21.15.89:0/4214504118 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00043a0 con 0x7f7bf8106730 2023-10-26T06:57:23.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.374+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81a3890 con 0x7f7bf8106730 2023-10-26T06:57:23.384 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.375+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8108cb0 con 0x7f7bf8106730 2023-10-26T06:57:23.385 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.376+0000 7f7bf57fa700 1 -- 172.21.15.89:0/4214504118 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7bf0004500 con 0x7f7bf8106730 2023-10-26T06:57:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.377+0000 7f7bf57fa700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7be00a1670 0x7f7be00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.378+0000 7f7bf57fa700 1 -- 172.21.15.89:0/4214504118 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f7bf00cdf30 con 0x7f7bf8106730 2023-10-26T06:57:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.378+0000 7f7bf7fff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7be00a1670 0x7f7be00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.386 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.378+0000 7f7bf7fff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7be00a1670 0x7f7be00a3b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009be0 tx=0x7f7be8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:23.387 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.382+0000 7f7bf57fa700 1 -- 172.21.15.89:0/4214504118 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7bf0097290 con 0x7f7bf8106730 2023-10-26T06:57:23.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:23 smithi089 ceph-mon[95031]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:57:23.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:23 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/732291193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:57:23.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:23 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/962339133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:57:23.598 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.597+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7bf8197e90 con 0x7f7bf8106730 2023-10-26T06:57:23.599 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.598+0000 7f7bf57fa700 1 -- 172.21.15.89:0/4214504118 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7bf009a8b0 con 0x7f7bf8106730 2023-10-26T06:57:23.600 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:57:23.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7be00a1670 msgr2=0x7f7be00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.602 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7be00a1670 0x7f7be00a3b30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009be0 tx=0x7f7be8009260 comp rx=0 tx=0).stop 2023-10-26T06:57:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf8106730 msgr2=0x7f7bf81969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf8106730 0x7f7bf81969d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b070 tx=0x7f7bf00077f0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.603 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 shutdown_connections 2023-10-26T06:57:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7be00a1670 0x7f7be00a3b30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bf8100720 0x7f7bf819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bf8106730 0x7f7bf81969d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.604 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 --2- 172.21.15.89:0/4214504118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf81010e0 0x7f7bf819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 >> 172.21.15.89:0/4214504118 conn(0x7f7bf80fc1b0 msgr2=0x7f7bf80fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 shutdown_connections 2023-10-26T06:57:23.605 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.601+0000 7f7bfe7ff700 1 -- 172.21.15.89:0/4214504118 wait complete. 2023-10-26T06:57:23.625 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-26T06:57:23.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:23 smithi192 ceph-mon[131661]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:57:23.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:23 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/732291193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:57:23.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:23 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/962339133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:57:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:23 smithi135 ceph-mon[131403]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:57:23.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:23 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/732291193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:57:23.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:23 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/962339133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:57:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.835+0000 7fd4a0dff700 1 -- 172.21.15.89:0/803833239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 msgr2=0x7fd49c10d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.837 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.835+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/803833239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 0x7fd49c10d570 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd494009f90 tx=0x7fd49400b330 comp rx=0 tx=0).stop 2023-10-26T06:57:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.836+0000 7fd4a0dff700 1 -- 172.21.15.89:0/803833239 shutdown_connections 2023-10-26T06:57:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.836+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/803833239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 0x7fd49c10d570 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.838 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.836+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/803833239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49c1066b0 0x7fd49c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.836+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/803833239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 0x7fd49c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.836+0000 7fd4a0dff700 1 -- 172.21.15.89:0/803833239 >> 172.21.15.89:0/803833239 conn(0x7fd49c0fc170 msgr2=0x7fd49c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.836+0000 7fd4a0dff700 1 -- 172.21.15.89:0/803833239 shutdown_connections 2023-10-26T06:57:23.839 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.836+0000 7fd4a0dff700 1 -- 172.21.15.89:0/803833239 wait complete. 2023-10-26T06:57:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.837+0000 7fd4a0dff700 1 Processor -- start 2023-10-26T06:57:23.842 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.837+0000 7fd4a0dff700 1 -- start start 2023-10-26T06:57:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd4a0dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 0x7fd49c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd4a0dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 0x7fd49c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.843 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd4a0dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49c1066b0 0x7fd49c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd4a0dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c1992a0 con 0x7fd49c101060 2023-10-26T06:57:23.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd4a0dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c199410 con 0x7fd49c1066b0 2023-10-26T06:57:23.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd4a0dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c1995d0 con 0x7fd49c1006a0 2023-10-26T06:57:23.844 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 0x7fd49c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 0x7fd49c19ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33628/0 (socket says 172.21.15.89:33628) 2023-10-26T06:57:23.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 -- 172.21.15.89:0/4049085878 learned_addr learned my addr 172.21.15.89:0/4049085878 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:23.845 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd499d9b700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 0x7fd49c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49ad9d700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49c1066b0 0x7fd49c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 -- 172.21.15.89:0/4049085878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49c1066b0 msgr2=0x7fd49c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49c1066b0 0x7fd49c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.846 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 -- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 msgr2=0x7fd49c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:23.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 0x7fd49c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:23.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.838+0000 7fd49a59c700 1 -- 172.21.15.89:0/4049085878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd494009c40 con 0x7fd49c1006a0 2023-10-26T06:57:23.847 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.839+0000 7fd49a59c700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 0x7fd49c19ea30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e9c0 tx=0x7fd48c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.839+0000 7fd48b7fe700 1 -- 172.21.15.89:0/4049085878 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c004a30 con 0x7fd49c1006a0 2023-10-26T06:57:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.839+0000 7fd48b7fe700 1 -- 172.21.15.89:0/4049085878 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd48c00cc80 con 0x7fd49c1006a0 2023-10-26T06:57:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.839+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c1998b0 con 0x7fd49c1006a0 2023-10-26T06:57:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.839+0000 7fd48b7fe700 1 -- 172.21.15.89:0/4049085878 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0054e0 con 0x7fd49c1006a0 2023-10-26T06:57:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.839+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c1a59b0 con 0x7fd49c1006a0 2023-10-26T06:57:23.849 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.840+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c101de0 con 0x7fd49c1006a0 2023-10-26T06:57:23.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.842+0000 7fd48b7fe700 1 -- 172.21.15.89:0/4049085878 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd48c004b90 con 0x7fd49c1006a0 2023-10-26T06:57:23.850 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.843+0000 7fd48b7fe700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4840a15f0 0x7fd4840a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:23.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.844+0000 7fd48b7fe700 1 -- 172.21.15.89:0/4049085878 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fd48c014070 con 0x7fd49c1006a0 2023-10-26T06:57:23.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.844+0000 7fd499d9b700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4840a15f0 0x7fd4840a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:23.851 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.844+0000 7fd499d9b700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4840a15f0 0x7fd4840a3ab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd49c19a240 tx=0x7fd490009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:23.852 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:23.847+0000 7fd48b7fe700 1 -- 172.21.15.89:0/4049085878 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd48c096480 con 0x7fd49c1006a0 2023-10-26T06:57:24.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.120+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fd49c110670 con 0x7fd49c1006a0 2023-10-26T06:57:24.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.129+0000 7fd48b7fe700 1 -- 172.21.15.89:0/4049085878 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+734408 (secure 0 0 0) 0x7fd48c099aa0 con 0x7fd49c1006a0 2023-10-26T06:57:24.140 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:57:24.141 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:57:24.141 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 24, 2023-10-26T06:57:24.141 INFO:teuthology.orchestra.run.smithi089.stdout: "active_gid": 14217, 2023-10-26T06:57:24.142 INFO:teuthology.orchestra.run.smithi089.stdout: "active_name": "x", 2023-10-26T06:57:24.142 INFO:teuthology.orchestra.run.smithi089.stdout: "active_addrs": { 2023-10-26T06:57:24.142 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-10-26T06:57:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2023-10-26T06:57:24.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6800", 2023-10-26T06:57:24.144 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1106080795 2023-10-26T06:57:24.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.144 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2023-10-26T06:57:24.145 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6801", 2023-10-26T06:57:24.145 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1106080795 2023-10-26T06:57:24.145 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.146 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:24.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.146 INFO:teuthology.orchestra.run.smithi089.stdout: "active_addr": "172.21.15.89:6801/1106080795", 2023-10-26T06:57:24.146 INFO:teuthology.orchestra.run.smithi089.stdout: "active_change": "2023-10-26T06:39:01.640088+0000", 2023-10-26T06:57:24.147 INFO:teuthology.orchestra.run.smithi089.stdout: "active_mgr_features": 4540138322906710015, 2023-10-26T06:57:24.147 INFO:teuthology.orchestra.run.smithi089.stdout: "available": true, 2023-10-26T06:57:24.147 INFO:teuthology.orchestra.run.smithi089.stdout: "standbys": [ 2023-10-26T06:57:24.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.148 INFO:teuthology.orchestra.run.smithi089.stdout: "gid": 24163, 2023-10-26T06:57:24.148 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "z", 2023-10-26T06:57:24.148 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr_features": 4540138322906710015, 2023-10-26T06:57:24.148 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2023-10-26T06:57:24.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.149 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2023-10-26T06:57:24.149 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.150 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.150 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.150 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:24.150 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:24.151 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.151 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.151 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.152 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:24.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.152 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.152 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.153 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2023-10-26T06:57:24.153 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.153 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.154 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.154 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.155 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.155 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.155 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.156 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.156 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.156 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.157 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.157 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.157 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.157 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.158 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.158 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.158 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.159 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.159 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.159 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.159 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.160 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.160 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.160 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.161 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.161 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.162 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.162 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.162 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.162 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.163 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.163 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.163 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.164 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.164 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.165 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.165 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.165 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.166 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.166 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.166 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.166 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.167 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.167 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.167 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.168 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.168 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.168 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.168 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.169 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.169 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.169 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.169 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.170 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.170 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.171 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.171 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.171 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.172 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.172 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.172 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.172 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.173 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.173 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.173 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.174 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2023-10-26T06:57:24.174 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2023-10-26T06:57:24.174 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.175 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.175 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.175 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.175 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.176 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.176 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.176 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to", 2023-10-26T06:57:24.176 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.177 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.177 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.177 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2023-10-26T06:57:24.178 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2023-10-26T06:57:24.178 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.178 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.179 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.179 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2023-10-26T06:57:24.179 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.179 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.180 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.180 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2023-10-26T06:57:24.180 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.181 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.181 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.181 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2023-10-26T06:57:24.182 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2023-10-26T06:57:24.182 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.182 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.183 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.183 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.183 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.184 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.184 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2023-10-26T06:57:24.184 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.185 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.185 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2023-10-26T06:57:24.186 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2023-10-26T06:57:24.186 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.186 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.186 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.187 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.187 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.187 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.187 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.188 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2023-10-26T06:57:24.188 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.189 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.189 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2023-10-26T06:57:24.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2023-10-26T06:57:24.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.190 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.190 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.191 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2023-10-26T06:57:24.191 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.191 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.192 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.192 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2023-10-26T06:57:24.192 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.192 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.193 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.193 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2023-10-26T06:57:24.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2023-10-26T06:57:24.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.194 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.194 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.195 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.195 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.195 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.196 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.196 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2023-10-26T06:57:24.196 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.196 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.197 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.197 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2023-10-26T06:57:24.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2023-10-26T06:57:24.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.198 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.198 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.199 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.199 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.199 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.200 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.200 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-26T06:57:24.200 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.201 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.201 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.201 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2023-10-26T06:57:24.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2023-10-26T06:57:24.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.202 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.202 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.203 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.203 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.203 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.203 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.204 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2023-10-26T06:57:24.204 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.205 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2023-10-26T06:57:24.206 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.206 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.207 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.207 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2023-10-26T06:57:24.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2023-10-26T06:57:24.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.208 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.208 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.208 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.209 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.209 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.209 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.209 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2023-10-26T06:57:24.210 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.210 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.210 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.211 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2023-10-26T06:57:24.211 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2023-10-26T06:57:24.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.212 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.212 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.212 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2023-10-26T06:57:24.212 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.213 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.213 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.213 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2023-10-26T06:57:24.213 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-26T06:57:24.214 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.214 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.215 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2023-10-26T06:57:24.215 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2023-10-26T06:57:24.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:24.215 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.216 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.216 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:24.216 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2023-10-26T06:57:24.216 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2023-10-26T06:57:24.217 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.217 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-26T06:57:24.217 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-26T06:57:24.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.218 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.218 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2023-10-26T06:57:24.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2023-10-26T06:57:24.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:24.219 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.220 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2023-10-26T06:57:24.220 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:24.220 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2023-10-26T06:57:24.221 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.221 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-26T06:57:24.221 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.222 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.222 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2023-10-26T06:57:24.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2023-10-26T06:57:24.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.223 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.223 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.224 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2023-10-26T06:57:24.224 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.224 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.224 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.225 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-26T06:57:24.225 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-26T06:57:24.225 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.225 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2023-10-26T06:57:24.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2023-10-26T06:57:24.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:24.227 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.227 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.227 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2023-10-26T06:57:24.228 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2023-10-26T06:57:24.228 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2023-10-26T06:57:24.228 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.228 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2023-10-26T06:57:24.229 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-26T06:57:24.229 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.229 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.230 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2023-10-26T06:57:24.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2023-10-26T06:57:24.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.231 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.231 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.231 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2023-10-26T06:57:24.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.232 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.232 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.232 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2023-10-26T06:57:24.233 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-26T06:57:24.233 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.233 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.234 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2023-10-26T06:57:24.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2023-10-26T06:57:24.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:24.234 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.235 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.235 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:24.235 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2023-10-26T06:57:24.236 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2023-10-26T06:57:24.236 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.236 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-26T06:57:24.236 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-26T06:57:24.237 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.237 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.237 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.238 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.238 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.239 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.239 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.239 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.239 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.240 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.240 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.240 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.240 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.241 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.241 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.241 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.241 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.242 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.242 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.242 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.243 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.243 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.244 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.244 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.245 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.245 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.245 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.246 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.246 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.246 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.246 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.247 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.247 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.247 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.248 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.248 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.248 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.249 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.249 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.249 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.249 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.250 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.250 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.250 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.250 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.251 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.251 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.251 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.252 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.252 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.252 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.253 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.253 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.253 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.254 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.254 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.254 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.254 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.255 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.255 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.255 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.256 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.256 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.256 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.256 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.257 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2023-10-26T06:57:24.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2023-10-26T06:57:24.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:24.258 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.258 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.258 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:24.258 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.259 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.259 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.259 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-26T06:57:24.260 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.260 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.260 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.261 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2023-10-26T06:57:24.261 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2023-10-26T06:57:24.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.261 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.262 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.262 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2023-10-26T06:57:24.262 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.265 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.265 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.266 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2023-10-26T06:57:24.266 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2023-10-26T06:57:24.266 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap" 2023-10-26T06:57:24.266 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.267 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2023-10-26T06:57:24.267 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.267 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.268 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.268 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2023-10-26T06:57:24.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2023-10-26T06:57:24.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.269 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.269 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.269 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.270 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.270 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.270 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.271 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-26T06:57:24.271 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.271 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.271 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.272 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:24.272 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:24.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.273 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.273 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.273 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:24.273 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.274 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.274 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.274 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-26T06:57:24.275 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.275 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.275 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.276 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2023-10-26T06:57:24.276 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2023-10-26T06:57:24.276 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.276 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.277 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.277 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:24.277 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:24.278 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.278 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.278 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-26T06:57:24.278 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-26T06:57:24.279 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.279 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.279 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2023-10-26T06:57:24.280 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2023-10-26T06:57:24.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:24.280 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.280 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.281 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:24.281 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.281 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.282 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.282 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-26T06:57:24.282 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.282 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.283 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.283 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.283 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.284 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2023-10-26T06:57:24.284 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.284 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.285 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.285 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2023-10-26T06:57:24.285 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2023-10-26T06:57:24.286 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:24.286 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.286 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.286 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2023-10-26T06:57:24.287 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.287 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.287 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.287 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-26T06:57:24.288 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.288 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.288 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.289 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2023-10-26T06:57:24.289 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2023-10-26T06:57:24.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.290 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.290 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.290 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2023-10-26T06:57:24.290 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.291 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.291 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.291 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-26T06:57:24.292 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.292 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.292 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.293 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2023-10-26T06:57:24.293 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2023-10-26T06:57:24.293 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.293 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.294 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.294 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2023-10-26T06:57:24.294 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.295 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.295 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.295 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-26T06:57:24.295 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.296 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.296 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.297 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2023-10-26T06:57:24.297 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2023-10-26T06:57:24.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.297 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.298 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.298 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.298 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.299 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.299 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.299 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-26T06:57:24.299 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-26T06:57:24.300 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.300 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.300 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2023-10-26T06:57:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2023-10-26T06:57:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.302 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.302 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:24.302 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.302 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.303 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.303 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-26T06:57:24.303 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.303 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.304 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.304 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2023-10-26T06:57:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2023-10-26T06:57:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2023-10-26T06:57:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-26T06:57:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2023-10-26T06:57:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2023-10-26T06:57:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2023-10-26T06:57:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2023-10-26T06:57:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2023-10-26T06:57:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-26T06:57:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2023-10-26T06:57:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2023-10-26T06:57:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-26T06:57:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2023-10-26T06:57:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2023-10-26T06:57:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-26T06:57:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2023-10-26T06:57:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2023-10-26T06:57:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-26T06:57:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2023-10-26T06:57:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2023-10-26T06:57:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-26T06:57:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2023-10-26T06:57:24.329 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2023-10-26T06:57:24.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.329 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.329 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.330 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-26T06:57:24.330 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.330 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.331 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.331 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2023-10-26T06:57:24.331 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.331 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.332 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.332 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2023-10-26T06:57:24.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2023-10-26T06:57:24.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.333 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.333 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.333 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-26T06:57:24.334 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.334 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.334 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.335 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2023-10-26T06:57:24.335 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.335 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.335 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.336 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2023-10-26T06:57:24.336 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2023-10-26T06:57:24.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.337 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.337 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.337 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-26T06:57:24.337 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.338 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.338 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.338 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:24.339 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.339 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.339 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.340 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2023-10-26T06:57:24.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2023-10-26T06:57:24.340 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.340 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.341 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.341 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-26T06:57:24.341 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.341 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.342 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.342 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2023-10-26T06:57:24.342 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.343 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.343 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.344 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2023-10-26T06:57:24.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2023-10-26T06:57:24.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.344 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.345 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.345 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-26T06:57:24.345 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.345 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.346 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.346 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2023-10-26T06:57:24.346 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.347 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.347 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.348 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2023-10-26T06:57:24.348 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2023-10-26T06:57:24.348 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.349 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.349 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.349 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-26T06:57:24.349 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.350 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.350 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.350 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2023-10-26T06:57:24.351 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.351 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.351 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.352 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2023-10-26T06:57:24.352 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2023-10-26T06:57:24.352 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.353 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.353 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.353 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-26T06:57:24.353 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.354 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.354 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.354 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2023-10-26T06:57:24.355 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.355 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.355 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.356 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2023-10-26T06:57:24.356 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2023-10-26T06:57:24.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.356 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.357 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.357 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-26T06:57:24.357 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.358 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.358 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.358 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2023-10-26T06:57:24.358 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.359 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.359 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.359 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2023-10-26T06:57:24.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2023-10-26T06:57:24.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.360 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.361 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.361 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-26T06:57:24.361 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.361 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.362 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.362 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2023-10-26T06:57:24.362 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.362 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.363 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.363 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2023-10-26T06:57:24.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2023-10-26T06:57:24.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.364 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.364 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.365 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-26T06:57:24.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.365 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.365 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.366 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:24.366 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.366 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.366 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.367 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2023-10-26T06:57:24.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2023-10-26T06:57:24.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.368 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.368 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.369 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-26T06:57:24.369 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.369 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.369 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.370 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2023-10-26T06:57:24.370 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.370 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.370 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.371 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2023-10-26T06:57:24.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2023-10-26T06:57:24.372 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.372 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.372 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.373 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-26T06:57:24.373 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.373 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.373 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.374 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:24.374 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.374 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.374 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.375 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2023-10-26T06:57:24.375 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2023-10-26T06:57:24.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.376 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.376 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.376 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-26T06:57:24.377 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.377 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.377 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.377 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2023-10-26T06:57:24.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.378 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.378 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.379 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2023-10-26T06:57:24.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2023-10-26T06:57:24.379 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.380 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.380 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.380 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-26T06:57:24.380 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.381 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2023-10-26T06:57:24.381 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.382 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.382 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.383 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2023-10-26T06:57:24.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2023-10-26T06:57:24.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.384 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.384 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.384 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.384 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.385 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.385 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.385 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2023-10-26T06:57:24.386 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.386 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.386 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.387 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2023-10-26T06:57:24.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2023-10-26T06:57:24.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.387 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.388 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:24.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.388 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.389 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.389 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-26T06:57:24.389 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.390 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.390 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2023-10-26T06:57:24.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2023-10-26T06:57:24.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.391 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2023-10-26T06:57:24.392 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.392 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.393 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.393 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-26T06:57:24.393 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.394 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2023-10-26T06:57:24.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2023-10-26T06:57:24.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.395 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.395 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.396 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2023-10-26T06:57:24.396 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.396 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.397 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.397 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-26T06:57:24.397 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.397 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.398 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.398 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2023-10-26T06:57:24.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2023-10-26T06:57:24.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.399 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.399 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.399 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2023-10-26T06:57:24.400 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.400 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.400 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.401 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2023-10-26T06:57:24.401 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.401 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.401 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.402 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2023-10-26T06:57:24.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2023-10-26T06:57:24.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.403 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.403 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.403 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.404 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.404 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.404 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.404 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-26T06:57:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.406 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2023-10-26T06:57:24.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2023-10-26T06:57:24.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.407 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.407 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.407 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:24.407 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.408 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.408 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.408 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2023-10-26T06:57:24.408 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.409 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.410 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2023-10-26T06:57:24.410 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2023-10-26T06:57:24.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.410 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.411 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.411 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:24.411 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.411 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.412 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.412 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2023-10-26T06:57:24.412 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.413 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.413 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.413 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2023-10-26T06:57:24.414 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2023-10-26T06:57:24.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.414 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.414 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.415 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.415 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.415 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.416 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.416 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-26T06:57:24.416 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.416 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.417 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.417 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.418 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.418 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.419 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.419 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.419 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.419 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.420 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.420 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.420 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.421 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.421 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.421 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.421 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.422 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.422 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.422 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.423 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2023-10-26T06:57:24.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2023-10-26T06:57:24.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.425 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.425 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.426 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-26T06:57:24.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.427 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.429 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.429 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.429 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.429 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.430 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-26T06:57:24.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.430 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.431 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.432 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.432 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.432 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.432 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.433 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.433 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.434 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.435 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.437 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.437 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.438 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.438 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.439 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.439 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.439 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.440 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.440 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.441 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-26T06:57:24.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-26T06:57:24.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.441 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.442 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.442 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.442 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.442 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.443 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.443 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-26T06:57:24.443 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.444 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.444 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-26T06:57:24.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-26T06:57:24.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.445 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.446 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.446 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2023-10-26T06:57:24.446 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.446 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.447 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.447 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-26T06:57:24.447 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.447 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.448 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.448 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2023-10-26T06:57:24.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2023-10-26T06:57:24.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.449 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.449 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.450 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:24.450 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.450 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.451 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.451 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2023-10-26T06:57:24.451 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.451 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.452 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2023-10-26T06:57:24.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2023-10-26T06:57:24.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.453 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.453 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.453 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:24.454 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.454 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.454 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.455 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-26T06:57:24.455 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.455 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.455 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.456 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2023-10-26T06:57:24.456 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2023-10-26T06:57:24.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.457 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.457 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.457 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.457 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.458 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.458 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.458 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2023-10-26T06:57:24.459 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.459 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.459 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.460 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2023-10-26T06:57:24.460 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2023-10-26T06:57:24.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.460 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.461 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.461 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2023-10-26T06:57:24.461 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.462 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.462 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.462 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2023-10-26T06:57:24.462 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2023-10-26T06:57:24.463 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.463 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2023-10-26T06:57:24.463 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.464 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.464 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2023-10-26T06:57:24.465 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2023-10-26T06:57:24.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.465 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.466 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-26T06:57:24.466 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.466 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.469 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.469 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-26T06:57:24.470 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.470 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.470 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.471 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2023-10-26T06:57:24.471 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2023-10-26T06:57:24.471 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.471 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.472 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.472 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.472 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.472 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.473 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.473 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-26T06:57:24.473 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.474 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.474 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.474 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2023-10-26T06:57:24.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2023-10-26T06:57:24.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.475 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.476 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.476 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.477 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.477 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-26T06:57:24.477 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.478 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.478 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.478 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2023-10-26T06:57:24.479 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2023-10-26T06:57:24.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.479 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.480 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.480 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.480 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.481 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.481 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-26T06:57:24.481 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.481 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.482 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.482 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2023-10-26T06:57:24.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2023-10-26T06:57:24.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.483 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.483 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.484 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.484 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.484 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.484 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-26T06:57:24.485 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.485 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.486 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2023-10-26T06:57:24.486 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2023-10-26T06:57:24.487 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.487 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.487 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.487 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.488 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.488 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.488 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.489 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-26T06:57:24.489 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.489 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.490 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2023-10-26T06:57:24.490 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2023-10-26T06:57:24.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.491 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.491 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.491 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2023-10-26T06:57:24.492 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.492 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.492 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.492 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2023-10-26T06:57:24.493 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.493 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.493 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.494 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2023-10-26T06:57:24.494 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2023-10-26T06:57:24.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.495 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.495 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.496 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.496 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.496 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-26T06:57:24.496 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.497 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.497 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.498 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2023-10-26T06:57:24.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2023-10-26T06:57:24.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.498 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.499 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.499 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.499 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.499 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.500 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.500 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-26T06:57:24.500 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.501 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.501 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.501 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2023-10-26T06:57:24.502 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2023-10-26T06:57:24.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.502 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.503 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.503 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.503 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.504 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.504 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-26T06:57:24.504 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.504 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.505 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.505 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2023-10-26T06:57:24.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2023-10-26T06:57:24.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.506 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.506 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.507 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.507 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.507 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.507 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.508 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2023-10-26T06:57:24.508 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.508 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.508 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.509 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2023-10-26T06:57:24.509 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2023-10-26T06:57:24.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.510 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.510 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.510 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.511 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.511 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.511 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.511 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-26T06:57:24.512 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.512 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.512 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.513 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2023-10-26T06:57:24.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2023-10-26T06:57:24.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.514 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.514 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.514 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.514 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.515 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.515 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.515 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-26T06:57:24.515 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.516 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.516 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.516 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2023-10-26T06:57:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.519 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.519 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.519 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.520 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.520 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.520 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.520 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.521 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.521 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.521 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.522 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.522 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.522 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.522 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.523 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.523 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.523 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.523 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.524 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.524 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.525 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.525 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.526 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.526 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.526 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.526 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.527 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.527 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.527 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.528 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.528 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.528 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.529 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.529 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.529 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.529 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.530 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.530 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.530 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.530 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.531 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.531 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.531 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.532 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.532 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.532 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.532 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.533 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.533 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.533 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.534 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.534 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.535 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.535 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.535 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.535 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.536 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.536 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.536 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.537 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.537 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.537 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2023-10-26T06:57:24.538 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2023-10-26T06:57:24.538 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.538 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.539 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.539 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2023-10-26T06:57:24.539 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.539 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.540 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.540 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-26T06:57:24.540 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.540 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.541 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.541 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2023-10-26T06:57:24.541 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2023-10-26T06:57:24.542 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.542 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.542 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.543 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2023-10-26T06:57:24.543 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.543 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.543 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.544 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-26T06:57:24.544 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.544 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.544 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.546 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2023-10-26T06:57:24.546 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.546 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.547 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.547 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-26T06:57:24.547 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-26T06:57:24.547 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.548 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.548 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.548 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:24.548 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.549 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.549 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.549 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.550 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.550 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.550 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.551 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2023-10-26T06:57:24.551 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-26T06:57:24.551 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.552 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.552 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.552 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.553 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.553 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.553 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.554 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.554 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.554 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.555 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-26T06:57:24.555 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-26T06:57:24.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.555 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.556 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.556 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.556 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.557 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.557 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.557 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.557 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.558 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.558 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.558 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2023-10-26T06:57:24.559 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2023-10-26T06:57:24.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.559 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.560 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.560 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.560 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.560 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.561 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.561 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.561 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.561 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.562 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.562 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-26T06:57:24.563 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-26T06:57:24.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.563 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.563 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.564 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.564 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.564 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.564 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.565 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.565 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.565 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.565 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.566 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2023-10-26T06:57:24.566 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-26T06:57:24.567 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.567 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.567 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.568 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.568 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.568 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.568 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.569 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.569 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.569 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.570 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.570 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-26T06:57:24.570 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-26T06:57:24.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.571 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.571 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.572 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.572 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.572 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.572 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.573 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.573 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.573 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.573 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.574 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-26T06:57:24.574 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-26T06:57:24.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.575 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.575 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.575 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.576 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.576 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.576 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.577 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.577 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.577 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.577 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.578 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-26T06:57:24.578 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-26T06:57:24.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.579 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.579 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.580 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.580 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.580 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.580 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.581 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.581 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.581 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.582 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.582 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-26T06:57:24.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-26T06:57:24.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.583 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.583 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.584 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.584 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.584 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.585 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.585 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.585 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.586 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.586 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-26T06:57:24.586 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-26T06:57:24.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.587 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.587 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.588 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.588 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.588 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.588 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.589 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.589 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.589 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.590 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.590 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-26T06:57:24.590 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-26T06:57:24.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.591 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.591 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.591 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.592 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.592 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.592 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.593 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.593 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.593 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.594 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-26T06:57:24.594 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-26T06:57:24.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.595 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.595 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.595 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.596 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.596 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.596 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.597 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.597 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.597 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.598 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-26T06:57:24.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-26T06:57:24.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.598 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.599 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.599 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.599 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.599 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.600 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.600 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.600 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.601 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.601 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.601 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2023-10-26T06:57:24.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-26T06:57:24.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.602 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.602 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.603 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:24.603 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.603 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.604 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.604 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.604 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.604 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.605 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.605 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-26T06:57:24.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-26T06:57:24.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.606 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.607 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.607 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.607 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.608 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.608 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.608 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.608 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.609 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.609 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.609 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2023-10-26T06:57:24.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2023-10-26T06:57:24.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.610 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.611 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.611 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.611 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.612 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.612 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.612 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.612 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.613 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.613 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2023-10-26T06:57:24.614 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-26T06:57:24.614 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.614 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.614 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.615 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:24.615 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.615 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.616 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.616 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.616 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.616 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.617 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.617 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-26T06:57:24.618 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-26T06:57:24.618 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.618 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.618 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.619 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.619 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.619 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.619 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.620 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.620 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.620 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.620 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.621 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-26T06:57:24.621 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-26T06:57:24.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.622 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.622 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.622 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.623 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.623 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.623 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.623 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.624 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.624 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.624 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.625 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-26T06:57:24.625 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-26T06:57:24.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.626 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.626 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.626 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.627 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.627 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.627 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.627 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.628 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.628 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.628 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.629 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-26T06:57:24.629 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-26T06:57:24.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.629 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.630 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.630 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.631 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.631 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.631 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.631 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.632 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.632 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.632 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2023-10-26T06:57:24.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-26T06:57:24.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.633 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.633 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.634 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.634 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.634 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.635 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.635 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.635 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.635 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.636 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.636 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-26T06:57:24.636 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-26T06:57:24.637 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.637 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.637 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.637 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.638 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.638 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.638 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.639 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.639 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.639 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.640 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.640 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-26T06:57:24.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-26T06:57:24.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.641 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.641 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.642 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.642 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.642 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.643 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.643 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.643 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.643 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.644 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-26T06:57:24.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-26T06:57:24.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.645 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.645 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.645 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.646 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.646 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.646 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.646 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.647 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.647 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.647 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.648 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-26T06:57:24.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-26T06:57:24.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.649 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.649 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.649 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.649 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.650 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.650 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.650 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.650 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.651 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.651 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.651 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-26T06:57:24.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-26T06:57:24.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.652 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.653 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.653 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.653 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.654 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.654 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.654 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.654 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.655 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.655 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-26T06:57:24.655 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-26T06:57:24.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.656 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.656 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.657 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.657 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.657 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.658 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.658 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.658 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.659 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.659 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-26T06:57:24.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-26T06:57:24.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.660 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.660 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.661 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.661 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.661 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.662 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.662 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.662 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.662 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.663 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.663 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-26T06:57:24.664 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-26T06:57:24.664 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.664 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.664 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.665 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.665 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.665 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.665 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.666 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.666 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.666 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.667 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.667 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2023-10-26T06:57:24.667 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-26T06:57:24.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.668 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.668 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.668 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.669 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.669 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.669 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.669 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.670 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.670 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.670 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.671 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-26T06:57:24.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-26T06:57:24.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.673 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-26T06:57:24.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.673 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.674 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.674 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.674 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.674 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.678 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-26T06:57:24.678 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-26T06:57:24.678 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.679 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.679 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.679 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:24.680 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.680 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.680 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.681 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.681 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.681 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.682 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.682 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-26T06:57:24.683 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-26T06:57:24.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.683 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.683 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.684 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2023-10-26T06:57:24.684 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.684 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.685 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.685 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.685 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.686 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.686 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.687 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-26T06:57:24.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-26T06:57:24.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.688 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.688 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.688 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2023-10-26T06:57:24.688 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.689 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.689 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.689 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.690 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.690 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.691 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2023-10-26T06:57:24.691 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-26T06:57:24.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.692 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.692 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.693 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.693 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.694 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.694 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.694 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.695 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.695 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.695 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.696 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-26T06:57:24.696 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-26T06:57:24.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.697 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.697 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.698 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:24.698 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.698 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.699 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.699 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.699 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.700 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.700 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.700 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2023-10-26T06:57:24.701 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-26T06:57:24.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.701 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.702 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.702 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.703 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.703 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.703 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.704 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.704 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.705 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2023-10-26T06:57:24.705 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-26T06:57:24.705 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.706 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.706 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.706 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.706 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.707 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.707 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.707 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.708 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.708 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.708 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.709 INFO:teuthology.orchestra.run.smithi089.stdout: "UNSAFE_TLS_v1_2": { 2023-10-26T06:57:24.709 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-26T06:57:24.709 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.710 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.710 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.711 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.711 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.711 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.711 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.712 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.712 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.712 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.713 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-26T06:57:24.713 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-26T06:57:24.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.714 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.714 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:24.715 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.715 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.715 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.716 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.716 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.716 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.716 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.717 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-26T06:57:24.717 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-26T06:57:24.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.718 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.718 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.719 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:24.719 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.719 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.719 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.720 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.720 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.720 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.721 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.721 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-26T06:57:24.722 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-26T06:57:24.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.722 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.722 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.723 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:24.723 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.723 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.724 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.724 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.724 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.725 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.725 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.725 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2023-10-26T06:57:24.726 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2023-10-26T06:57:24.726 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.726 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.727 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.727 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.727 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.728 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.728 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.728 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.729 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.729 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.729 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.730 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2023-10-26T06:57:24.730 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2023-10-26T06:57:24.730 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.731 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.731 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.731 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.732 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.732 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.732 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.733 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.733 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.733 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.733 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.734 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2023-10-26T06:57:24.734 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2023-10-26T06:57:24.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.735 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.735 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.736 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.736 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.736 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.736 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.737 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2023-10-26T06:57:24.737 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.737 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.738 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.738 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2023-10-26T06:57:24.739 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2023-10-26T06:57:24.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.739 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.740 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.740 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2023-10-26T06:57:24.740 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.740 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.741 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.741 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.741 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.742 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.742 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2023-10-26T06:57:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2023-10-26T06:57:24.743 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.744 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.745 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.748 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.749 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.750 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.751 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.752 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.753 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.754 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.754 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.755 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.755 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.755 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.755 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.756 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.756 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.756 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.757 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.757 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.758 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.758 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.758 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.759 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.759 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.759 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.760 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.760 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.760 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.761 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.761 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.761 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.761 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.762 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.762 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.762 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.763 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.763 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.763 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.764 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.764 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.765 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.765 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.765 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.765 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.766 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.766 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.767 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.767 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.767 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.768 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2023-10-26T06:57:24.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2023-10-26T06:57:24.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.769 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.769 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.769 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.770 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.770 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.770 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.771 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2023-10-26T06:57:24.771 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.771 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.771 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.772 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2023-10-26T06:57:24.772 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2023-10-26T06:57:24.773 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.773 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.773 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.775 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.775 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.775 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.776 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.776 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:24.776 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:24.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.777 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.777 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.778 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2023-10-26T06:57:24.778 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.778 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.779 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.779 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.779 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.780 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.780 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.780 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:24.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:24.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.781 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.782 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.782 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2023-10-26T06:57:24.782 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.782 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.783 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.783 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.784 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.784 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.784 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.785 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2023-10-26T06:57:24.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2023-10-26T06:57:24.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.786 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.786 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.786 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.787 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.787 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.787 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.788 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.788 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.788 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.788 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.789 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2023-10-26T06:57:24.789 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2023-10-26T06:57:24.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.790 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.790 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.791 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2023-10-26T06:57:24.791 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.791 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.792 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.792 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.792 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.792 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.793 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.793 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2023-10-26T06:57:24.794 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2023-10-26T06:57:24.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.794 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.795 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.795 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2023-10-26T06:57:24.795 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.795 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.796 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.796 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.796 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2023-10-26T06:57:24.797 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.797 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.797 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.797 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.798 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.798 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2023-10-26T06:57:24.799 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2023-10-26T06:57:24.799 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.799 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.800 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.800 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2023-10-26T06:57:24.800 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2023-10-26T06:57:24.800 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2023-10-26T06:57:24.801 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.801 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.801 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.802 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.802 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.802 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2023-10-26T06:57:24.803 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2023-10-26T06:57:24.803 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.803 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.804 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.804 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.804 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.804 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.805 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.805 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.805 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.806 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.806 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.806 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.807 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.807 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2023-10-26T06:57:24.808 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.808 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.808 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.809 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2023-10-26T06:57:24.809 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2023-10-26T06:57:24.809 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.810 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.810 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.810 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.810 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.811 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.811 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.811 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2023-10-26T06:57:24.812 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.812 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.812 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.813 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.813 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.814 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.814 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.815 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.815 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.815 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.816 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.816 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.816 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.817 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.817 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.817 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.817 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.818 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.818 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.818 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.819 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.819 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.820 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.820 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.821 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.821 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.821 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.822 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.822 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.822 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.823 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.823 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.823 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.824 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.825 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.826 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.826 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.826 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.827 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.827 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.827 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.828 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.828 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.828 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.829 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.829 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.829 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.829 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.830 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.830 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.830 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.831 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.831 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.832 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.832 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.832 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.833 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.833 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.833 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.834 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.834 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.834 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.835 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.835 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2023-10-26T06:57:24.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2023-10-26T06:57:24.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.836 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.836 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.837 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2023-10-26T06:57:24.837 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.837 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.837 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.838 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-26T06:57:24.838 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.838 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.839 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.839 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2023-10-26T06:57:24.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2023-10-26T06:57:24.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.840 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.841 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.841 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2023-10-26T06:57:24.841 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.841 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.842 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.842 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-26T06:57:24.842 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.843 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.843 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.844 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2023-10-26T06:57:24.844 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2023-10-26T06:57:24.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.844 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.845 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.845 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2023-10-26T06:57:24.845 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.846 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.846 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.846 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2023-10-26T06:57:24.847 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.847 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.847 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.848 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2023-10-26T06:57:24.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2023-10-26T06:57:24.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.849 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.849 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.850 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2023-10-26T06:57:24.850 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.850 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.851 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.851 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-26T06:57:24.852 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.852 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.852 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.853 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2023-10-26T06:57:24.853 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2023-10-26T06:57:24.854 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.854 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.854 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.855 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:24.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.855 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.856 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.856 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-26T06:57:24.857 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.857 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.857 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.858 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:24.858 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:24.858 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.859 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:24.860 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.860 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.860 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.860 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2023-10-26T06:57:24.861 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.861 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.861 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2023-10-26T06:57:24.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2023-10-26T06:57:24.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.863 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.863 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.863 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2023-10-26T06:57:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.864 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-26T06:57:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.865 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.867 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2023-10-26T06:57:24.867 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.868 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.868 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.868 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.869 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.870 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.870 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.870 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.871 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.871 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.871 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.871 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.872 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.872 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.872 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.873 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.873 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.873 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.873 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.874 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.874 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.875 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.877 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.877 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.878 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.878 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.878 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.879 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.879 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.880 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.880 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.880 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.880 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.881 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.881 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.882 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.882 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.882 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.883 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.883 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.883 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.884 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.884 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.884 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.885 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.885 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.885 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.886 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.886 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.886 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.887 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.887 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.887 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.888 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.888 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.888 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.889 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.889 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.889 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.890 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.890 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.890 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.891 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.891 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.891 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.892 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.892 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.892 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2023-10-26T06:57:24.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2023-10-26T06:57:24.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.893 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.894 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.894 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2023-10-26T06:57:24.894 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.895 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.895 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.895 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.895 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.896 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.896 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.897 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2023-10-26T06:57:24.897 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2023-10-26T06:57:24.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.897 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.898 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.898 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2023-10-26T06:57:24.898 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.899 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.899 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.899 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.900 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.900 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.900 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.901 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:24.901 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:24.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.902 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.902 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.902 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:24.903 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.903 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.903 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.903 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.904 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.904 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.904 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.906 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2023-10-26T06:57:24.906 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2023-10-26T06:57:24.906 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2023-10-26T06:57:24.907 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.907 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2023-10-26T06:57:24.907 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2023-10-26T06:57:24.908 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.908 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.908 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.908 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2023-10-26T06:57:24.909 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.909 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.909 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.910 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-26T06:57:24.910 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.910 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.911 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.911 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2023-10-26T06:57:24.911 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2023-10-26T06:57:24.912 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.912 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.912 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.913 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2023-10-26T06:57:24.913 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.913 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.914 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.914 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-26T06:57:24.914 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.914 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.915 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.915 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2023-10-26T06:57:24.916 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2023-10-26T06:57:24.916 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.916 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.917 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.917 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.917 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.917 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2023-10-26T06:57:24.918 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.919 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.919 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.919 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:24.920 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:24.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:24.920 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.921 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.921 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2023-10-26T06:57:24.921 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2023-10-26T06:57:24.922 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.922 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.922 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-26T06:57:24.922 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.923 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.923 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.924 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.924 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.924 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.925 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.925 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.925 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.926 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.926 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.927 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.927 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.927 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.928 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.928 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.928 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.928 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.929 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.929 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.930 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.930 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.930 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.931 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.931 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.931 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.932 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.932 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.932 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.933 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.933 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.933 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.933 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.934 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.934 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.935 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.935 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.936 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.936 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.936 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.937 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.937 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.937 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.938 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.938 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.938 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.939 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.939 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.939 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.939 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.940 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.940 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.940 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.941 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.941 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.942 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.942 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.942 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.943 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.943 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.943 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.944 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.944 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.944 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.945 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.945 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2023-10-26T06:57:24.945 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2023-10-26T06:57:24.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.946 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.946 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.947 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.947 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.947 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.947 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.948 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2023-10-26T06:57:24.948 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.948 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.949 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.949 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2023-10-26T06:57:24.950 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2023-10-26T06:57:24.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.950 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.950 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.951 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2023-10-26T06:57:24.951 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.951 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.952 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.952 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2023-10-26T06:57:24.952 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.952 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.953 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.953 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2023-10-26T06:57:24.954 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2023-10-26T06:57:24.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.954 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.955 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.955 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2023-10-26T06:57:24.955 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.956 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.956 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.956 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-26T06:57:24.956 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.957 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.957 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.958 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2023-10-26T06:57:24.958 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2023-10-26T06:57:24.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:24.959 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.959 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.959 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:24.960 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:24.960 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2023-10-26T06:57:24.960 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.960 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-26T06:57:24.961 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.961 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.961 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.962 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2023-10-26T06:57:24.962 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2023-10-26T06:57:24.962 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.963 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.963 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.963 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.964 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.964 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.965 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.965 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2023-10-26T06:57:24.965 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.966 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.966 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.966 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2023-10-26T06:57:24.967 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2023-10-26T06:57:24.967 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.967 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.968 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:24.968 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2023-10-26T06:57:24.968 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.969 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.969 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.969 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-26T06:57:24.970 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.970 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.970 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.972 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2023-10-26T06:57:24.972 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.972 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.973 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.973 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.973 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.974 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.974 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.974 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.975 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.975 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.975 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.975 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.976 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.976 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.976 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.977 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.977 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.977 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.978 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.978 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.978 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.978 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.979 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.979 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:24.980 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:24.980 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.980 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.981 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.981 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.981 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.981 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.982 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.982 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.982 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.983 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.983 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.984 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:24.984 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:24.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.985 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.985 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.985 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:24.986 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.986 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.986 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.986 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:24.987 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:24.987 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:24.987 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:24.988 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:24.988 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:24.988 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:24.989 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.989 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.989 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.989 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.990 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:24.990 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:24.991 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:24.991 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.991 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.992 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:24.992 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.992 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.993 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:24.993 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:24.993 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:24.993 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:24.994 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:24.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:24.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:24.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:24.995 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2023-10-26T06:57:24.996 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:24.996 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:24.996 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:24.996 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:24.997 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:24.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:24.997 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:24.998 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:24.998 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:24.998 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:24.999 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:24.999 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:24.999 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.000 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.000 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.000 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.000 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.001 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.001 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.001 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.002 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.002 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.002 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.003 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.003 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.004 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.004 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.004 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.004 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.005 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.005 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.005 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.006 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.006 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.006 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.007 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.007 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.008 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.008 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.008 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.009 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.009 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.009 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.010 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.010 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.010 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.011 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.011 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.011 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.012 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.012 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.012 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.012 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.013 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.013 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.014 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.015 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.015 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.015 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.016 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.016 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.016 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.016 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.017 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.017 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.017 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.019 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2023-10-26T06:57:25.019 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.020 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.020 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.020 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2023-10-26T06:57:25.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2023-10-26T06:57:25.021 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.021 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.021 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.022 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2023-10-26T06:57:25.022 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.022 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.023 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.023 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-26T06:57:25.023 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.024 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.024 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.025 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2023-10-26T06:57:25.025 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2023-10-26T06:57:25.025 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.025 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.026 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.026 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:25.026 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2023-10-26T06:57:25.027 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.027 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.027 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-26T06:57:25.028 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.028 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.028 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.029 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.029 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.030 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.030 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.030 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.031 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.031 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.031 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.032 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.032 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.032 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.032 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.033 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.033 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.033 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.034 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.034 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.034 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.035 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.035 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.036 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.036 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.037 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.037 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.037 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.038 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.038 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.038 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.039 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.039 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.040 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.041 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.041 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.041 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.041 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.042 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.042 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.043 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.043 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.043 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.043 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.045 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.045 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.045 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.045 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.046 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.046 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.047 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.047 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.048 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.048 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.048 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.048 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.049 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.049 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.049 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.050 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.050 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.050 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2023-10-26T06:57:25.051 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.052 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.052 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.052 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2023-10-26T06:57:25.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2023-10-26T06:57:25.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.053 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.054 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.054 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2023-10-26T06:57:25.054 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.054 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.055 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.055 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2023-10-26T06:57:25.055 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-26T06:57:25.056 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.056 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.057 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.057 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.058 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.058 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.058 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.059 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.059 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.059 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.060 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.060 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.060 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.061 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.061 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.061 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.062 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.062 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.062 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.062 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.063 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.063 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.064 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.064 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.064 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.065 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.065 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.065 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.065 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.066 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.066 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.066 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.067 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.067 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.068 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.068 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.068 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.068 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.069 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.069 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.070 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.070 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.070 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.070 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.071 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.071 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.071 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.072 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.072 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.072 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.073 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.073 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.073 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.074 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.074 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.074 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.075 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.075 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.075 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.075 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.076 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.076 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.076 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.077 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.077 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.077 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.078 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2023-10-26T06:57:25.078 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2023-10-26T06:57:25.079 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.079 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.079 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.079 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.081 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.081 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.082 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.082 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2023-10-26T06:57:25.082 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-26T06:57:25.083 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.083 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.083 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2023-10-26T06:57:25.084 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2023-10-26T06:57:25.084 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.084 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.085 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.085 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2023-10-26T06:57:25.085 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.086 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.086 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.086 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2023-10-26T06:57:25.087 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.087 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.087 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.088 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2023-10-26T06:57:25.088 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2023-10-26T06:57:25.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.089 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.089 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.089 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2023-10-26T06:57:25.090 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.090 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.090 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.091 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2023-10-26T06:57:25.091 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.091 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.092 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.092 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2023-10-26T06:57:25.093 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2023-10-26T06:57:25.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.093 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.094 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.094 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.094 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.095 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.095 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.095 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2023-10-26T06:57:25.095 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.096 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.096 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.097 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2023-10-26T06:57:25.097 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2023-10-26T06:57:25.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.098 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.098 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.098 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2023-10-26T06:57:25.099 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.099 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.099 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.100 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-26T06:57:25.100 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-26T06:57:25.100 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.101 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.102 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2023-10-26T06:57:25.103 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.103 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.104 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.104 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.104 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.105 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.105 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.105 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.106 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.106 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.106 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.107 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.107 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.107 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.107 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.108 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.108 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.108 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.109 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.109 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.109 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.110 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.110 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.110 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.111 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.111 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.111 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.112 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.112 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.112 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.113 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.113 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.113 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.114 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.114 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.114 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.115 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.116 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.116 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.116 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.116 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.117 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.117 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.117 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.118 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.118 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.118 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.119 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.119 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.119 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.119 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.120 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.120 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.120 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.121 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.121 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.122 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.122 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.123 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.123 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.123 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.124 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.124 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.124 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.124 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.125 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2023-10-26T06:57:25.126 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.127 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.127 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.127 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.128 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.128 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.129 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.129 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.129 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.130 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.130 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.130 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.131 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.131 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.131 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.131 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.132 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.132 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.132 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.133 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.133 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.134 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.134 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.134 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.135 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.135 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.135 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.136 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.136 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.136 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.137 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.137 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.137 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.138 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.138 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.139 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.139 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.139 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.139 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.140 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.140 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.140 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.141 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.141 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.141 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.142 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.142 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.142 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.142 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.143 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.143 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.143 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.144 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.144 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.144 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.145 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.145 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.145 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.146 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.146 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.146 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.147 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.147 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.147 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.147 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.148 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.148 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.149 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2023-10-26T06:57:25.149 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.149 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.150 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.150 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.150 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.151 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.151 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.151 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.152 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.152 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.153 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.153 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.153 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.154 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.154 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.154 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.155 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.155 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.155 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.155 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.156 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.156 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.157 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.157 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.157 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.158 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.158 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.158 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.159 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.159 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.159 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.160 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.160 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.160 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.161 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.161 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.162 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.162 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.162 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.162 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.163 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.163 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.163 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.164 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.164 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.164 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.165 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.165 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.165 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.166 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.166 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.166 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.167 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.167 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.167 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.168 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.168 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.169 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.169 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.169 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.170 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.170 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.170 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.170 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.171 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.171 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.172 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.172 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2023-10-26T06:57:25.173 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.173 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.173 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.174 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2023-10-26T06:57:25.174 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2023-10-26T06:57:25.174 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.175 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.175 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.175 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.176 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.176 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.176 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.176 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-26T06:57:25.177 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.177 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.177 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.178 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.179 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.179 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.179 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.179 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.180 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.180 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.180 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.181 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.181 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.181 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.182 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.182 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.182 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.182 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.183 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.183 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.183 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.184 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.184 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.185 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.185 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.185 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.186 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.186 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.187 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.187 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.187 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.187 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.188 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.189 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.189 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.189 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.190 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.190 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.190 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.190 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.191 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.191 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.191 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.192 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.192 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.192 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.193 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.193 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.193 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.194 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.194 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.194 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.194 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.195 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.195 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.196 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.196 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.197 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.197 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.197 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.198 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.198 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.198 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.199 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.199 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.200 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2023-10-26T06:57:25.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2023-10-26T06:57:25.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.201 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.201 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.201 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.201 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.202 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.202 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.202 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2023-10-26T06:57:25.203 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2023-10-26T06:57:25.203 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2023-10-26T06:57:25.203 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.204 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2023-10-26T06:57:25.204 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.205 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2023-10-26T06:57:25.207 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.207 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.207 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.207 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.208 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.208 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.209 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.209 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.210 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.210 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.210 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.211 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.211 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.211 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.212 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.212 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.212 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.213 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.213 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.213 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.214 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.214 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.215 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.215 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.216 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.216 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.216 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.217 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.217 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.217 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.218 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.218 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.219 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.220 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.220 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.221 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.221 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.222 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.222 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.222 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.223 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.223 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.223 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.224 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.224 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.224 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.225 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.225 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.225 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.226 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.227 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.227 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.227 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.228 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.228 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.228 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.228 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.229 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.229 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.229 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.230 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.230 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2023-10-26T06:57:25.231 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.231 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.232 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.232 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.232 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.233 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.233 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.234 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.234 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.234 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.234 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.235 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.235 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.235 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.236 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.236 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.236 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.237 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.237 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.237 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.237 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.238 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.238 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.239 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.240 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.240 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.240 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.240 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.241 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.241 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.241 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.242 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.242 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.243 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.243 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.244 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.244 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.244 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.245 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.245 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.245 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.246 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.246 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.247 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.247 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.247 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.247 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.248 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.248 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.249 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.249 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.250 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.250 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.250 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.251 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.251 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.251 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.252 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.252 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.252 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.252 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.253 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.253 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.253 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.254 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.255 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2023-10-26T06:57:25.255 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.255 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.256 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.256 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.256 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.256 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.257 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.257 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.257 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.258 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.258 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.258 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.259 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.259 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.259 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.259 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.260 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.260 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.260 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.261 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.261 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.261 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.262 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.263 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.263 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.264 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.264 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.264 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.265 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.265 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.265 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.265 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.266 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.266 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.267 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.268 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.268 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.268 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.269 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.269 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.269 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.270 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.270 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.270 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.270 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.271 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.271 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.271 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.272 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.272 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.272 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.273 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.273 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.273 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.274 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.274 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.274 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.275 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.275 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.275 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.276 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.276 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.276 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.277 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.277 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:25.278 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:25.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:25.278 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.279 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:25.279 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.279 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.280 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.280 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.280 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.281 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.281 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.282 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2023-10-26T06:57:25.282 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2023-10-26T06:57:25.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:25.283 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.283 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.283 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2023-10-26T06:57:25.284 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2023-10-26T06:57:25.284 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.284 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.284 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2023-10-26T06:57:25.285 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-26T06:57:25.285 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.285 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.286 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.286 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2023-10-26T06:57:25.287 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.289 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.289 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.290 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2023-10-26T06:57:25.290 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2023-10-26T06:57:25.290 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.291 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.291 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.291 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.291 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.292 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.292 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.292 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-26T06:57:25.293 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.293 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.293 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.294 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2023-10-26T06:57:25.294 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2023-10-26T06:57:25.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.295 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.295 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.295 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.296 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.296 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.296 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.297 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.297 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.297 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.297 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.298 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.298 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.299 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.299 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.299 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.300 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.300 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.300 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.301 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.301 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.301 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.302 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.302 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.302 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.302 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.303 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.303 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.303 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.304 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.304 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.305 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.306 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.306 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.306 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.307 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.307 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.307 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.307 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.308 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.308 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.309 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.309 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.310 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.310 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.310 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.310 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.311 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.311 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.311 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.312 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.312 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.312 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.313 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.313 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.313 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.314 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.314 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.314 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.315 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.315 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.315 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.316 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.316 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.316 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.317 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.317 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.317 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.318 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.318 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.318 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.319 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.319 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.319 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2023-10-26T06:57:25.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2023-10-26T06:57:25.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.320 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2023-10-26T06:57:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2023-10-26T06:57:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.325 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.325 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:25.325 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.326 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.326 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.326 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2023-10-26T06:57:25.326 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.327 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.327 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.329 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2023-10-26T06:57:25.329 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.329 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.329 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.330 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2023-10-26T06:57:25.330 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2023-10-26T06:57:25.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.331 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.331 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.331 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.331 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.332 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.332 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.332 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.333 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.333 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.333 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.334 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2023-10-26T06:57:25.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2023-10-26T06:57:25.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.335 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.335 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.335 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.336 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.336 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-26T06:57:25.337 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-26T06:57:25.337 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.338 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.339 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.339 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.339 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.340 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.340 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.341 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.341 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.341 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.342 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.342 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.342 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.342 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.343 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.343 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.343 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.344 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.344 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.345 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.345 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.345 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.345 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.346 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.346 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.346 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.347 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.347 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.347 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.348 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.348 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.349 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.349 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.350 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.350 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.350 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.351 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.351 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.351 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.351 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.352 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.352 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.352 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.353 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.353 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.353 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.354 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.354 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.354 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.354 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.355 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.356 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.356 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.357 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.357 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.357 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.358 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.358 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.358 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.358 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.359 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.359 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2023-10-26T06:57:25.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2023-10-26T06:57:25.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.360 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.361 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.361 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.361 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.362 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.362 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.362 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.363 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.363 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.363 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.364 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-26T06:57:25.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-26T06:57:25.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.365 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.365 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.365 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2023-10-26T06:57:25.366 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.366 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.366 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.367 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.367 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.367 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.368 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.368 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2023-10-26T06:57:25.368 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2023-10-26T06:57:25.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:25.369 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.369 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.370 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2023-10-26T06:57:25.370 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.370 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.371 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.371 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.371 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.373 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2023-10-26T06:57:25.374 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.375 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-26T06:57:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.376 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:25.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.378 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2023-10-26T06:57:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.379 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-26T06:57:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.380 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.381 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2023-10-26T06:57:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2023-10-26T06:57:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.382 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2023-10-26T06:57:25.383 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.384 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2023-10-26T06:57:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2023-10-26T06:57:25.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.387 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2023-10-26T06:57:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.388 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2023-10-26T06:57:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2023-10-26T06:57:25.389 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.390 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2023-10-26T06:57:25.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2023-10-26T06:57:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2023-10-26T06:57:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2023-10-26T06:57:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2023-10-26T06:57:25.393 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2023-10-26T06:57:25.396 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.397 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.398 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.399 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.400 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.401 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.402 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.403 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.405 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.406 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.407 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.408 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.409 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.410 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.411 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.412 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.413 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.415 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.416 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2023-10-26T06:57:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2023-10-26T06:57:25.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:25.420 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.421 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2023-10-26T06:57:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2023-10-26T06:57:25.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.424 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.425 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2023-10-26T06:57:25.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2023-10-26T06:57:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.429 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.431 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.432 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2023-10-26T06:57:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2023-10-26T06:57:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2023-10-26T06:57:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.434 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.435 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.436 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2023-10-26T06:57:25.438 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2023-10-26T06:57:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.439 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.440 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.441 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.442 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.443 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.444 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.445 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.446 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.447 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.450 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.451 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.454 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.455 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.456 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.457 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.458 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.460 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.461 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.462 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:25.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.465 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.466 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.467 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:25.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.469 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.470 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.471 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2023-10-26T06:57:25.473 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.474 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.477 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.478 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.479 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.482 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.483 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.484 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.486 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.486 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.487 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.487 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.488 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.488 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.488 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.488 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.489 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.489 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.490 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.490 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.490 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.493 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.493 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.494 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.494 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.496 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.496 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.497 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.497 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.497 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.498 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.498 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.499 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2023-10-26T06:57:25.500 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.500 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.500 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.501 INFO:teuthology.orchestra.run.smithi089.stdout: "drive_group_interval": { 2023-10-26T06:57:25.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "drive_group_interval", 2023-10-26T06:57:25.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:25.502 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.502 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.503 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300.0", 2023-10-26T06:57:25.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.503 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.503 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.504 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-26T06:57:25.504 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.504 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.505 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.505 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.506 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.507 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.507 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.508 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.508 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.508 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.509 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.509 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.509 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.509 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.510 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.510 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.510 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.511 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.511 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.511 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.512 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.512 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.513 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.513 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.514 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.514 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.514 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.514 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.515 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.515 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.515 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.515 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.516 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.516 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.517 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.517 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.517 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.518 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.518 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.518 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.519 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.519 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.519 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.520 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.520 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.520 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.521 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.521 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.521 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.521 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.522 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.522 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.523 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.523 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.523 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.524 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.524 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.524 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.525 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.525 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.525 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.526 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.526 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.526 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.527 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2023-10-26T06:57:25.527 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2023-10-26T06:57:25.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.528 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.528 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.528 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2023-10-26T06:57:25.528 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.529 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.529 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.529 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-26T06:57:25.530 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.530 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.530 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2023-10-26T06:57:25.532 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.532 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.533 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.533 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.534 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.534 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.535 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.535 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.535 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.536 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.536 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.536 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.536 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.537 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.537 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.537 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.537 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.538 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.538 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.538 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.539 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.539 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.540 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.540 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.541 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.541 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.542 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.542 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.542 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.542 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.543 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.543 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.544 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.545 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.545 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.545 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.545 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.546 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.546 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.546 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.547 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.547 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.547 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.548 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.548 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.548 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.549 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.549 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.549 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.549 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.550 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.550 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.551 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.551 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.551 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.552 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.552 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.552 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.552 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.553 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.553 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.553 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.554 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.554 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2023-10-26T06:57:25.554 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2023-10-26T06:57:25.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.555 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.556 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.556 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.556 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.556 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.557 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.557 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.557 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.558 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.558 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2023-10-26T06:57:25.559 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2023-10-26T06:57:25.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.559 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.560 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.560 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2023-10-26T06:57:25.560 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.561 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.561 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.561 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.562 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.562 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.562 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.563 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2023-10-26T06:57:25.563 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2023-10-26T06:57:25.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.564 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.564 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.564 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.565 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.565 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.565 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.565 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.566 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.566 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.566 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.567 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2023-10-26T06:57:25.567 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2023-10-26T06:57:25.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.568 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.568 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.569 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.569 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.569 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.569 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.570 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.570 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.570 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.571 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.571 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2023-10-26T06:57:25.572 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2023-10-26T06:57:25.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:25.572 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.573 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.573 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.574 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.574 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.574 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.575 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.575 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.575 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.576 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2023-10-26T06:57:25.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2023-10-26T06:57:25.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.577 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.577 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.577 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.578 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.578 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.578 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.579 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.579 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.579 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.580 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.580 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2023-10-26T06:57:25.581 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2023-10-26T06:57:25.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.581 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.581 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.582 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.582 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.582 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.583 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.583 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.583 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.584 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.585 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2023-10-26T06:57:25.585 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2023-10-26T06:57:25.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.585 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.586 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.586 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.586 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.587 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.587 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.587 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.588 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.588 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.589 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2023-10-26T06:57:25.589 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2023-10-26T06:57:25.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.590 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.590 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.590 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.591 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:25.591 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2023-10-26T06:57:25.591 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.592 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.592 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.593 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.593 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2023-10-26T06:57:25.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2023-10-26T06:57:25.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.594 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.594 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.595 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.595 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.596 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.596 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.596 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.597 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.597 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.597 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2023-10-26T06:57:25.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2023-10-26T06:57:25.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.598 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.599 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.599 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.599 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.600 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.600 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.600 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.600 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.601 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.601 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.602 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2023-10-26T06:57:25.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2023-10-26T06:57:25.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.603 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.603 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.603 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.603 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.604 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.604 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.604 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.605 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.605 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.605 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.607 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2023-10-26T06:57:25.607 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.607 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.608 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.608 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2023-10-26T06:57:25.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2023-10-26T06:57:25.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.609 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.609 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.609 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.610 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.610 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.610 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.611 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2023-10-26T06:57:25.611 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.611 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.611 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.612 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2023-10-26T06:57:25.612 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2023-10-26T06:57:25.613 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.613 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.613 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.613 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.614 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.614 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.614 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.615 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2023-10-26T06:57:25.615 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.615 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.616 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.616 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.616 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.617 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.618 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.618 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.618 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.619 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.619 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.619 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.620 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.620 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.620 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.620 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.621 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.621 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.621 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.622 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.622 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.623 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.623 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.623 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.623 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.624 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.624 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.625 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.625 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.625 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.626 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.626 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.626 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.627 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.628 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.628 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.628 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.629 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.629 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.629 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.630 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.630 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.630 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.630 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.631 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.631 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.631 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.632 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.633 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.633 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.634 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.634 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.635 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.635 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.635 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.636 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.636 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.636 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.636 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.637 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.637 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.637 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.639 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2023-10-26T06:57:25.639 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.639 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.640 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.640 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.641 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.642 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.642 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.642 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.642 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.643 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.643 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.643 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.644 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.644 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.644 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.645 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.645 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.645 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.645 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.646 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.646 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.647 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.647 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.647 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.648 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.648 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.649 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.649 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.649 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.649 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.650 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.650 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.651 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.651 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.651 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.652 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.652 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.652 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.653 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.653 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.653 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.654 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.654 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.654 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.654 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.655 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.655 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.655 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.656 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.656 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.657 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.657 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.657 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.658 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.658 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.659 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.659 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.659 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.660 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.660 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.660 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.661 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.661 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.663 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2023-10-26T06:57:25.663 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.664 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.664 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.664 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.665 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.665 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.666 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.666 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.667 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.667 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.667 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.667 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.668 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.668 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.668 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.669 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.669 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.669 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.670 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.670 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.670 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.671 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.672 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.673 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.673 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.674 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.674 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.674 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.675 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.675 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.676 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.676 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.676 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.677 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.677 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.677 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.678 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.678 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.678 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.679 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.679 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.679 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.679 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.680 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.680 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.680 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.681 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.681 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.682 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.683 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.683 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.683 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.684 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.684 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.684 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.684 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.685 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.685 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2023-10-26T06:57:25.687 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.687 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.688 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.688 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2023-10-26T06:57:25.688 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2023-10-26T06:57:25.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.689 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.689 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.689 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-26T06:57:25.690 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.690 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.690 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.691 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.691 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.691 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.691 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.692 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:25.692 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:25.693 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:25.693 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.693 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.694 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2023-10-26T06:57:25.694 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.694 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.694 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.695 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.695 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.695 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.696 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.696 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.697 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.697 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.697 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.697 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.698 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.698 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.698 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.701 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.701 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.702 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.702 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.702 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.703 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.703 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.703 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.704 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.704 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.705 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.705 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.705 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.706 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.706 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.707 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.707 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.707 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.708 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.708 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.708 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.709 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.709 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.710 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.710 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.711 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.711 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.711 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.711 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.712 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.712 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.712 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.713 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.713 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.713 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.714 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.714 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.714 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.715 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.715 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.715 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.715 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.716 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.716 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.717 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.717 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.718 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.718 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.718 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.718 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.719 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.719 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.719 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.720 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.720 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.720 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.721 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2023-10-26T06:57:25.722 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.722 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.722 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.723 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2023-10-26T06:57:25.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2023-10-26T06:57:25.723 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.723 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.724 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.724 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.724 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.725 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.725 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-26T06:57:25.726 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.726 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.726 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.727 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2023-10-26T06:57:25.728 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2023-10-26T06:57:25.728 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.728 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.728 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.729 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.729 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.729 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.730 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.730 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-26T06:57:25.730 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.731 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.731 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.732 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2023-10-26T06:57:25.732 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2023-10-26T06:57:25.732 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.732 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.733 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.733 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.733 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.734 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.734 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.734 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-26T06:57:25.735 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.735 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.735 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.736 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2023-10-26T06:57:25.736 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2023-10-26T06:57:25.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.737 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.737 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.737 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.738 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.738 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.738 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.738 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-26T06:57:25.739 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.739 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.739 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.740 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2023-10-26T06:57:25.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2023-10-26T06:57:25.741 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.741 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.741 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.741 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.742 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.742 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.742 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.743 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-26T06:57:25.743 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.743 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.744 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.744 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2023-10-26T06:57:25.744 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2023-10-26T06:57:25.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.745 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.745 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.746 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.746 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.746 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.747 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.747 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.747 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.747 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.748 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.748 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2023-10-26T06:57:25.749 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2023-10-26T06:57:25.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.749 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.750 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.750 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.750 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.751 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.751 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.751 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.751 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.752 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.752 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.753 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2023-10-26T06:57:25.753 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2023-10-26T06:57:25.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.753 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.754 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.754 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-26T06:57:25.754 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.755 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.755 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.755 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.756 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.756 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.756 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.757 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2023-10-26T06:57:25.757 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2023-10-26T06:57:25.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.758 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.758 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.758 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.759 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.759 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.759 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.760 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.760 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.760 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.761 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.761 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:25.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:25.762 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.762 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.762 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.763 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2023-10-26T06:57:25.763 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2023-10-26T06:57:25.763 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.764 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.764 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.764 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.764 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.765 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.765 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2023-10-26T06:57:25.766 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2023-10-26T06:57:25.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.766 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.767 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.767 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2023-10-26T06:57:25.767 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.767 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.768 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.768 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.768 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.768 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.769 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.769 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2023-10-26T06:57:25.770 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2023-10-26T06:57:25.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.770 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.771 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.771 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.771 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.772 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.772 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.772 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.772 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.773 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.773 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.774 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2023-10-26T06:57:25.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2023-10-26T06:57:25.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.775 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.775 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.775 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.775 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.776 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.776 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.776 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.777 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.777 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.777 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.778 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.778 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.779 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.779 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.779 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.780 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.780 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.780 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.781 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.781 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.781 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.782 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.782 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.782 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.783 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.783 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.783 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.784 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.784 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.785 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.785 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.786 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.786 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.786 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.787 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.787 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.787 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.788 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.788 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.788 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.789 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.789 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.790 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.790 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.790 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.791 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.791 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.791 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.791 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.792 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.792 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.792 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.793 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.793 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.793 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.793 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.794 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.794 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.794 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.796 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.796 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.796 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.798 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.798 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.798 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.799 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2023-10-26T06:57:25.799 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2023-10-26T06:57:25.800 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.800 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.800 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.800 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.801 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.801 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.801 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.802 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.802 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.802 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.802 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.803 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2023-10-26T06:57:25.803 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2023-10-26T06:57:25.804 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.804 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.804 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.805 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.805 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.805 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.805 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.806 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.806 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.806 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.807 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.807 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2023-10-26T06:57:25.808 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2023-10-26T06:57:25.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.808 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.808 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.809 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-26T06:57:25.809 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.809 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.810 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.810 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.810 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.811 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.811 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.812 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.812 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2023-10-26T06:57:25.813 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.813 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.813 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.814 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.814 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.815 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.815 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.815 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.816 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.816 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.816 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.817 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.817 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.817 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.818 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.818 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.818 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.819 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.819 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.819 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.820 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.820 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.821 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.821 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.822 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.822 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.822 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.823 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.823 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.824 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.824 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.824 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.825 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.825 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.825 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.826 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.826 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.827 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.827 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.827 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.828 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.828 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.828 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.829 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.829 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.829 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.829 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.830 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.830 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.830 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.831 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.831 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.832 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.833 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.833 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.833 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.833 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.834 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.834 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.834 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.835 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.835 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.835 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.836 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.836 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.837 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2023-10-26T06:57:25.837 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.837 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.838 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.838 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.839 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.839 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.840 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.840 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.840 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.840 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.841 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.841 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.841 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.842 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.842 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.842 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.843 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.843 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.843 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.843 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.844 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.844 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.845 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.846 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.846 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.846 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.846 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.847 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.847 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.847 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.848 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.848 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.849 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.849 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.850 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.850 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.850 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.851 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.851 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.851 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.851 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.852 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.852 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.852 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.853 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.853 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.853 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.853 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.854 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.854 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.854 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.855 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.855 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.855 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.856 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.856 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.856 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.857 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.857 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.857 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.858 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.858 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.858 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.859 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.859 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2023-10-26T06:57:25.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2023-10-26T06:57:25.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.860 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.860 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.861 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2023-10-26T06:57:25.861 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.861 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.862 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.862 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-26T06:57:25.862 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.863 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.863 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.863 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2023-10-26T06:57:25.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2023-10-26T06:57:25.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.864 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.864 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.865 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.865 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.865 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.866 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.866 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2023-10-26T06:57:25.866 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.867 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.867 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.867 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2023-10-26T06:57:25.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2023-10-26T06:57:25.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.868 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.869 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.869 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:25.869 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.870 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.870 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.870 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-26T06:57:25.871 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.871 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.871 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.873 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2023-10-26T06:57:25.873 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.873 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.874 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.874 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2023-10-26T06:57:25.874 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2023-10-26T06:57:25.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:25.875 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.875 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.875 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2023-10-26T06:57:25.876 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.876 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.876 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.877 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.877 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.877 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.878 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.878 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2023-10-26T06:57:25.879 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2023-10-26T06:57:25.879 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.879 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.879 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.880 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.880 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.880 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.881 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.881 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.881 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.882 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.882 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.882 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:25.883 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:25.883 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:25.883 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.884 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.884 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:25.884 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.884 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.885 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.885 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.885 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.886 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.886 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.887 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.887 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.887 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.888 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.888 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.888 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.888 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.889 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.889 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.889 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.890 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.890 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.890 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.890 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.891 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.891 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.891 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.892 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.892 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.892 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.893 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.894 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.894 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.894 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.895 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.895 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.895 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.896 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.896 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.896 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.897 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.897 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.897 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.898 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.898 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.899 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.899 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.899 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.900 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.900 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.900 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.901 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.901 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.901 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.901 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.902 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.902 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.902 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.903 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.903 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.904 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.904 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.904 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.905 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.905 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.905 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.906 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.906 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.906 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.907 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.907 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.907 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.910 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2023-10-26T06:57:25.910 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2023-10-26T06:57:25.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.911 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.911 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.911 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.911 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.912 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.912 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.912 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.913 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.913 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.913 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.914 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2023-10-26T06:57:25.914 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2023-10-26T06:57:25.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.915 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.915 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2023-10-26T06:57:25.915 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.916 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.916 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.916 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.917 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.917 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.917 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.918 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2023-10-26T06:57:25.918 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2023-10-26T06:57:25.918 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.919 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.919 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:25.919 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-26T06:57:25.920 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.920 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.920 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.920 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.921 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.921 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.921 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.922 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.922 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.922 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.923 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:25.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.923 INFO:teuthology.orchestra.run.smithi089.stdout: "gid": 24167, 2023-10-26T06:57:25.924 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "y", 2023-10-26T06:57:25.924 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr_features": 4540138322906710015, 2023-10-26T06:57:25.924 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2023-10-26T06:57:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2023-10-26T06:57:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.925 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:25.926 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.927 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.928 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2023-10-26T06:57:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.930 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.931 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.932 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.933 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.934 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.935 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.936 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:25.937 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:25.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.937 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.938 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.938 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.938 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.939 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.939 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.939 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.939 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.940 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.940 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.941 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:25.941 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:25.941 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.942 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.942 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.942 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:25.942 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.943 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.943 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:25.943 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:25.944 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:25.944 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:25.944 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:25.945 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:25.945 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:25.945 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:25.945 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.946 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.946 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.946 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.947 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:25.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:25.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.948 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.948 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.948 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:25.949 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.949 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.949 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.950 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:25.950 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.950 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.951 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.951 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2023-10-26T06:57:25.951 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2023-10-26T06:57:25.952 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.952 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.952 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.953 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.953 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.953 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.953 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.954 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to", 2023-10-26T06:57:25.954 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.954 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.955 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.955 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2023-10-26T06:57:25.955 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2023-10-26T06:57:25.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.956 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.956 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.957 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2023-10-26T06:57:25.957 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.957 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.957 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.958 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2023-10-26T06:57:25.958 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.958 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.959 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.959 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2023-10-26T06:57:25.960 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2023-10-26T06:57:25.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.960 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.960 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.961 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.961 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.961 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.962 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.962 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2023-10-26T06:57:25.962 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.962 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.963 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.963 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2023-10-26T06:57:25.964 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2023-10-26T06:57:25.964 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.964 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.965 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.965 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.965 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.965 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.966 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.966 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2023-10-26T06:57:25.966 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.967 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.967 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.968 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2023-10-26T06:57:25.968 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2023-10-26T06:57:25.968 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:25.968 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.969 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.969 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2023-10-26T06:57:25.969 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.970 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.970 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.970 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2023-10-26T06:57:25.971 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.971 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.971 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.972 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2023-10-26T06:57:25.972 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2023-10-26T06:57:25.972 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.973 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.973 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.973 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.974 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.974 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.974 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.975 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2023-10-26T06:57:25.975 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.975 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.976 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.976 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2023-10-26T06:57:25.976 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2023-10-26T06:57:25.977 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.977 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.977 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.978 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.978 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.978 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.979 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.979 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-26T06:57:25.979 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.979 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.980 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.980 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2023-10-26T06:57:25.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2023-10-26T06:57:25.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.981 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.982 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.982 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:25.982 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.982 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.983 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.983 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2023-10-26T06:57:25.983 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.984 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.984 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.985 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:25.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:25.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2023-10-26T06:57:25.986 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:25.986 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:25.986 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:25.987 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2023-10-26T06:57:25.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2023-10-26T06:57:25.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:25.987 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.988 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.988 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:25.988 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.989 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.989 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.989 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2023-10-26T06:57:25.989 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:25.990 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.990 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.991 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2023-10-26T06:57:25.991 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2023-10-26T06:57:25.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:25.992 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.992 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.993 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2023-10-26T06:57:25.993 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:25.993 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:25.994 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.994 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2023-10-26T06:57:25.994 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-26T06:57:25.995 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.995 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:25.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:25.996 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2023-10-26T06:57:25.996 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2023-10-26T06:57:25.996 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:25.996 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:25.997 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:25.997 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:25.997 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2023-10-26T06:57:25.998 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2023-10-26T06:57:25.998 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:25.998 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-26T06:57:25.999 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-26T06:57:25.999 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:25.999 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.000 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2023-10-26T06:57:26.000 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2023-10-26T06:57:26.001 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:26.001 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.001 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.001 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2023-10-26T06:57:26.002 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:26.002 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2023-10-26T06:57:26.002 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.003 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-26T06:57:26.003 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.003 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.004 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.004 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2023-10-26T06:57:26.005 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2023-10-26T06:57:26.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.005 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.006 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.006 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2023-10-26T06:57:26.006 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.006 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.007 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.007 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-26T06:57:26.007 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-26T06:57:26.008 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.008 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.009 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2023-10-26T06:57:26.009 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2023-10-26T06:57:26.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:26.009 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.010 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.010 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2023-10-26T06:57:26.010 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2023-10-26T06:57:26.011 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2023-10-26T06:57:26.011 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.011 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2023-10-26T06:57:26.012 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-26T06:57:26.012 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.012 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.013 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2023-10-26T06:57:26.013 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2023-10-26T06:57:26.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.014 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.014 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.014 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2023-10-26T06:57:26.015 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.015 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.015 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.015 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2023-10-26T06:57:26.016 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-26T06:57:26.016 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.016 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.017 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2023-10-26T06:57:26.017 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2023-10-26T06:57:26.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:26.018 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.018 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.018 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:26.019 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2023-10-26T06:57:26.019 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2023-10-26T06:57:26.019 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.020 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-26T06:57:26.020 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-26T06:57:26.020 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.021 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.021 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.022 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.022 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.022 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.023 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.023 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.023 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.024 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.024 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.024 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.025 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.025 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.025 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.025 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.026 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.026 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.026 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.027 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.027 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.028 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.028 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.028 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.029 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.029 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.029 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.030 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.030 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.030 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.031 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.031 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.031 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.032 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.032 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.033 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.033 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.033 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.034 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.034 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.034 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.035 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.035 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.036 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.036 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.036 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.036 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.037 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.037 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.037 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.038 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.038 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.039 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.039 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.040 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.040 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.040 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.041 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.041 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.041 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.042 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.042 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.042 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.043 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2023-10-26T06:57:26.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2023-10-26T06:57:26.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:26.044 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.044 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.044 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:26.045 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.045 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.045 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.046 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-26T06:57:26.046 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.046 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.047 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.047 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2023-10-26T06:57:26.048 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2023-10-26T06:57:26.048 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.048 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.048 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.049 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2023-10-26T06:57:26.049 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.049 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.050 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.050 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2023-10-26T06:57:26.050 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2023-10-26T06:57:26.051 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap" 2023-10-26T06:57:26.051 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.051 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2023-10-26T06:57:26.051 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.052 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.052 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.053 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2023-10-26T06:57:26.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2023-10-26T06:57:26.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.054 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.054 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.054 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.055 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.055 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.055 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.055 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-26T06:57:26.056 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.056 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.056 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.057 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:26.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:26.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.058 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.058 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.058 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:26.059 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.059 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.059 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.059 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-26T06:57:26.060 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.060 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.060 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.061 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2023-10-26T06:57:26.061 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2023-10-26T06:57:26.062 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.062 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.062 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.062 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:26.063 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:26.063 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.063 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.064 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-26T06:57:26.064 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-26T06:57:26.064 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.065 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.065 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2023-10-26T06:57:26.065 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2023-10-26T06:57:26.066 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:26.066 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.066 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.067 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:26.067 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.067 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.068 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.068 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-26T06:57:26.068 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.068 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.069 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.069 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.069 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.070 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.070 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.070 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2023-10-26T06:57:26.071 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.071 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.071 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.071 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2023-10-26T06:57:26.072 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2023-10-26T06:57:26.072 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:26.072 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.073 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.073 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2023-10-26T06:57:26.073 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.074 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.074 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.074 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-26T06:57:26.075 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.075 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.075 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.076 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2023-10-26T06:57:26.076 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2023-10-26T06:57:26.076 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.077 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.077 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.077 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2023-10-26T06:57:26.078 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.078 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.078 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.078 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-26T06:57:26.079 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.079 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.079 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.080 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2023-10-26T06:57:26.080 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2023-10-26T06:57:26.080 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.081 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.081 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.081 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2023-10-26T06:57:26.082 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.082 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.082 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.083 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-26T06:57:26.083 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.083 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.083 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.084 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2023-10-26T06:57:26.084 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2023-10-26T06:57:26.085 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.085 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.085 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.086 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.086 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.086 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.086 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.087 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-26T06:57:26.087 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-26T06:57:26.087 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.088 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.088 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2023-10-26T06:57:26.088 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2023-10-26T06:57:26.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.089 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.089 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.090 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:26.090 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.090 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.091 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.091 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-26T06:57:26.091 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.091 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.092 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.092 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2023-10-26T06:57:26.093 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2023-10-26T06:57:26.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:26.093 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.094 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.094 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2023-10-26T06:57:26.094 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.095 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.095 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.095 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-26T06:57:26.095 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.096 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.096 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.097 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2023-10-26T06:57:26.097 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2023-10-26T06:57:26.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.098 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.098 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.098 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.099 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.099 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.099 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.100 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2023-10-26T06:57:26.100 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2023-10-26T06:57:26.100 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2023-10-26T06:57:26.101 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.101 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-26T06:57:26.101 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.101 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.102 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.102 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2023-10-26T06:57:26.103 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2023-10-26T06:57:26.103 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.103 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.104 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.104 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.104 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.104 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.105 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.105 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-26T06:57:26.105 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.106 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.106 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.107 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2023-10-26T06:57:26.107 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2023-10-26T06:57:26.107 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.108 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.108 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.108 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.108 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.109 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.109 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.109 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-26T06:57:26.110 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.110 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.110 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.111 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2023-10-26T06:57:26.111 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2023-10-26T06:57:26.111 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.112 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.112 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.112 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.113 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.113 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.113 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.114 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-26T06:57:26.114 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.114 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.115 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.115 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2023-10-26T06:57:26.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2023-10-26T06:57:26.116 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.116 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.117 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.117 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-26T06:57:26.117 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.118 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.118 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.118 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:26.118 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.119 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.119 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.120 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2023-10-26T06:57:26.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2023-10-26T06:57:26.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.121 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.121 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.121 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-26T06:57:26.121 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.122 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.122 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.122 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2023-10-26T06:57:26.123 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.123 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.123 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.124 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2023-10-26T06:57:26.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2023-10-26T06:57:26.124 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.125 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.125 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.125 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-26T06:57:26.126 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.126 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.126 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.127 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2023-10-26T06:57:26.127 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.127 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.127 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.128 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2023-10-26T06:57:26.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2023-10-26T06:57:26.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.129 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.129 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.129 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-26T06:57:26.130 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.130 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.130 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.131 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:26.131 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.131 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.132 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.132 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2023-10-26T06:57:26.132 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2023-10-26T06:57:26.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.133 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.133 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.134 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-26T06:57:26.134 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.134 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.134 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.135 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2023-10-26T06:57:26.135 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.135 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.136 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.136 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2023-10-26T06:57:26.137 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2023-10-26T06:57:26.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.137 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.137 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.138 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-26T06:57:26.138 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.138 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.139 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.139 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2023-10-26T06:57:26.139 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.140 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.140 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.141 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2023-10-26T06:57:26.141 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2023-10-26T06:57:26.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.142 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.142 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.142 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-26T06:57:26.142 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.143 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.143 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.143 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2023-10-26T06:57:26.144 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.144 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.144 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.145 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2023-10-26T06:57:26.145 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2023-10-26T06:57:26.145 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.146 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.146 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.146 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-26T06:57:26.147 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.147 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.147 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.147 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2023-10-26T06:57:26.148 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.148 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.148 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.149 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2023-10-26T06:57:26.149 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2023-10-26T06:57:26.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.150 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.150 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.150 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-26T06:57:26.151 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.151 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.151 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.152 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2023-10-26T06:57:26.152 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.152 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.153 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.153 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2023-10-26T06:57:26.154 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2023-10-26T06:57:26.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.154 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.154 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.155 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-26T06:57:26.155 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.155 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.156 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.156 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2023-10-26T06:57:26.156 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.156 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.157 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.157 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2023-10-26T06:57:26.158 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2023-10-26T06:57:26.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.158 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.159 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.159 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-26T06:57:26.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.160 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.160 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.160 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:26.160 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.161 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.162 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2023-10-26T06:57:26.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2023-10-26T06:57:26.162 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.163 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.163 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.163 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-26T06:57:26.163 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.164 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.164 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.164 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2023-10-26T06:57:26.165 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.165 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.165 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.166 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2023-10-26T06:57:26.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2023-10-26T06:57:26.167 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.167 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.167 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.167 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-26T06:57:26.168 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.168 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.168 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.169 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:26.169 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.169 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.169 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.170 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2023-10-26T06:57:26.170 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2023-10-26T06:57:26.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.171 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.171 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.172 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-26T06:57:26.172 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.172 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.173 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.173 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2023-10-26T06:57:26.173 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.173 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.174 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.174 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2023-10-26T06:57:26.175 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2023-10-26T06:57:26.175 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.175 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.175 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.176 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-26T06:57:26.176 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.176 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.176 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.177 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2023-10-26T06:57:26.177 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.177 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.178 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.178 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2023-10-26T06:57:26.178 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2023-10-26T06:57:26.179 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.179 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.179 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.180 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.180 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.180 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.180 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.181 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2023-10-26T06:57:26.181 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.181 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.182 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.182 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2023-10-26T06:57:26.182 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2023-10-26T06:57:26.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.183 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.184 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:26.184 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.184 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.184 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.185 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-26T06:57:26.185 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.185 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.186 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.186 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2023-10-26T06:57:26.186 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2023-10-26T06:57:26.187 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.187 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.187 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.188 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2023-10-26T06:57:26.188 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.188 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.188 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.189 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-26T06:57:26.189 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.189 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.189 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.190 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2023-10-26T06:57:26.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2023-10-26T06:57:26.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.191 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.191 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.191 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2023-10-26T06:57:26.192 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.192 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.192 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.192 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-26T06:57:26.193 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.193 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.193 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.194 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2023-10-26T06:57:26.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2023-10-26T06:57:26.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.195 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.195 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.195 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2023-10-26T06:57:26.195 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.196 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.196 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.196 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2023-10-26T06:57:26.196 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.197 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.197 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.198 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2023-10-26T06:57:26.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2023-10-26T06:57:26.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.198 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.199 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.199 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.199 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.199 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.200 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.200 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-26T06:57:26.200 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.201 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.201 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.201 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2023-10-26T06:57:26.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2023-10-26T06:57:26.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.202 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.203 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.203 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:26.203 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.203 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.204 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.204 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2023-10-26T06:57:26.204 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.205 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.205 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2023-10-26T06:57:26.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2023-10-26T06:57:26.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.206 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.206 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.207 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:26.207 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.207 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.207 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.208 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2023-10-26T06:57:26.208 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.208 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.209 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.209 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2023-10-26T06:57:26.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2023-10-26T06:57:26.210 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.210 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.210 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.210 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.211 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.211 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.211 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.211 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-26T06:57:26.212 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.212 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.212 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.213 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.214 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.214 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.214 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.214 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.215 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.215 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.215 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.216 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.216 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.216 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.216 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.217 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.217 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.217 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.217 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.218 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.219 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2023-10-26T06:57:26.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2023-10-26T06:57:26.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.219 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.220 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.220 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.220 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.221 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.221 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-26T06:57:26.221 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.222 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.222 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.223 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.224 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.224 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.224 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.224 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.225 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.225 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-26T06:57:26.225 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.225 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.226 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.226 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.227 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.227 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.228 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.228 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.228 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.228 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.229 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.229 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.229 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.230 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.230 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.230 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.230 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.231 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.231 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.231 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.231 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.232 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.232 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.233 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.233 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.233 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.234 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.234 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.234 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.235 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.235 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.235 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.235 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.236 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-26T06:57:26.236 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-26T06:57:26.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.237 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.237 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.237 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.238 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.238 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.238 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.239 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-26T06:57:26.239 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.239 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.239 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.240 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-26T06:57:26.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-26T06:57:26.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.241 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.241 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.241 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2023-10-26T06:57:26.242 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.242 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.243 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.243 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-26T06:57:26.243 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.243 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.244 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.244 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2023-10-26T06:57:26.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2023-10-26T06:57:26.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.245 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.246 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:26.246 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.247 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.247 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.247 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2023-10-26T06:57:26.247 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.248 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.249 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2023-10-26T06:57:26.249 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2023-10-26T06:57:26.249 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.250 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.250 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.250 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:26.250 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.251 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.251 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.251 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-26T06:57:26.252 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.252 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.253 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2023-10-26T06:57:26.253 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2023-10-26T06:57:26.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.254 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.254 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.254 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.255 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.255 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.255 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.255 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2023-10-26T06:57:26.256 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.256 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.256 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.257 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2023-10-26T06:57:26.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2023-10-26T06:57:26.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.258 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.258 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.258 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2023-10-26T06:57:26.259 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.259 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.259 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.259 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2023-10-26T06:57:26.260 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2023-10-26T06:57:26.260 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.260 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2023-10-26T06:57:26.261 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.261 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.262 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2023-10-26T06:57:26.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2023-10-26T06:57:26.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.263 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.263 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.264 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-26T06:57:26.264 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.264 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.265 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.265 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-26T06:57:26.265 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.265 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.266 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.266 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2023-10-26T06:57:26.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2023-10-26T06:57:26.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.267 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.268 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.268 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.268 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.268 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.269 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.269 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-26T06:57:26.269 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.270 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.270 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.270 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2023-10-26T06:57:26.271 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2023-10-26T06:57:26.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.271 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.272 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.272 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.272 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.272 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.273 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.273 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-26T06:57:26.273 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.274 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.274 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2023-10-26T06:57:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2023-10-26T06:57:26.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.276 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.276 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.276 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.277 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.277 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.277 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-26T06:57:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.278 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2023-10-26T06:57:26.279 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2023-10-26T06:57:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.281 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-26T06:57:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.282 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2023-10-26T06:57:26.283 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2023-10-26T06:57:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.284 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.285 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-26T06:57:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.286 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2023-10-26T06:57:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2023-10-26T06:57:26.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.288 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2023-10-26T06:57:26.289 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.289 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.289 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.289 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2023-10-26T06:57:26.290 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.290 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.290 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.291 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2023-10-26T06:57:26.291 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2023-10-26T06:57:26.291 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.292 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.292 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.292 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.292 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.293 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.293 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.293 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-26T06:57:26.294 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.294 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.294 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.295 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2023-10-26T06:57:26.295 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2023-10-26T06:57:26.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.295 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.296 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.296 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.296 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.297 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.297 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.297 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-26T06:57:26.297 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.298 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.298 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.299 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2023-10-26T06:57:26.299 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2023-10-26T06:57:26.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.299 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.300 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.300 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.300 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.300 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.301 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.301 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-26T06:57:26.301 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.302 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.302 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.302 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2023-10-26T06:57:26.303 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2023-10-26T06:57:26.303 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.303 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.304 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.304 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.304 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.304 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.305 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.305 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2023-10-26T06:57:26.305 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.305 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.306 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.306 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2023-10-26T06:57:26.307 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2023-10-26T06:57:26.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.307 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.307 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.308 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.308 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.308 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.308 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.309 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-26T06:57:26.309 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.309 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.310 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.310 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2023-10-26T06:57:26.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2023-10-26T06:57:26.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.311 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.311 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.311 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.312 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.312 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.312 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.313 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-26T06:57:26.313 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.313 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.313 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.314 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.314 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.315 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2023-10-26T06:57:26.315 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.315 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.317 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.317 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.318 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.318 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.318 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.318 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.319 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.319 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.319 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.320 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.320 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.320 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.320 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.321 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.321 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.321 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.322 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.322 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.322 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.322 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.324 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.325 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.326 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.328 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.328 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.329 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.329 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.329 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.329 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.330 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.330 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.330 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.331 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.331 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.331 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.331 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.332 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.332 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.332 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.332 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.333 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.334 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.334 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.334 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.335 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.335 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.336 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.336 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.336 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.337 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2023-10-26T06:57:26.337 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2023-10-26T06:57:26.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.338 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.338 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.338 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2023-10-26T06:57:26.339 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.339 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.339 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.340 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-26T06:57:26.340 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.340 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.341 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2023-10-26T06:57:26.341 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2023-10-26T06:57:26.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.342 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.342 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.342 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2023-10-26T06:57:26.343 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.343 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.343 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.344 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-26T06:57:26.344 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.344 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.344 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.345 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.345 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.346 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.346 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2023-10-26T06:57:26.346 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.347 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.347 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.347 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-26T06:57:26.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-26T06:57:26.348 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.348 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.348 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.348 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:26.349 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.349 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.349 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.350 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.350 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.350 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.350 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.351 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2023-10-26T06:57:26.351 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-26T06:57:26.352 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.352 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.352 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.352 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.353 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.353 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.353 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.354 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.354 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.354 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.355 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.355 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-26T06:57:26.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-26T06:57:26.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.356 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.356 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.357 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.357 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.357 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.357 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.358 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.358 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.358 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.359 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.359 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2023-10-26T06:57:26.359 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2023-10-26T06:57:26.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.360 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.360 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.361 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.361 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.361 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.362 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.362 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.362 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.362 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.363 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.363 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-26T06:57:26.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-26T06:57:26.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.364 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.364 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.365 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.365 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.366 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.366 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.366 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.366 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.367 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.367 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2023-10-26T06:57:26.368 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-26T06:57:26.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.368 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.368 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.369 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.369 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.369 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.369 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.370 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.370 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.370 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.371 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.371 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-26T06:57:26.372 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-26T06:57:26.372 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.372 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.372 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.373 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.373 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.373 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.373 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.374 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.374 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.374 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.375 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.375 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-26T06:57:26.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-26T06:57:26.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.376 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.376 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.377 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.377 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.377 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.378 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.378 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.378 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.379 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.379 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-26T06:57:26.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-26T06:57:26.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.380 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.380 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.381 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.382 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.382 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.382 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.383 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.383 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-26T06:57:26.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-26T06:57:26.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.384 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.384 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.385 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.385 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.385 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.385 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.386 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.386 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.386 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.386 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.387 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-26T06:57:26.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-26T06:57:26.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.388 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.388 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.389 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.389 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.389 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.390 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.390 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.390 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.391 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-26T06:57:26.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-26T06:57:26.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.393 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.393 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.393 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.394 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.394 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.395 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-26T06:57:26.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-26T06:57:26.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.396 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.396 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.396 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.397 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.397 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.397 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.398 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.398 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.398 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.399 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-26T06:57:26.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-26T06:57:26.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.400 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.400 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.400 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.401 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.401 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.401 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.401 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.402 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.402 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.402 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.403 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2023-10-26T06:57:26.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-26T06:57:26.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.404 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.405 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:26.405 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.405 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.405 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.406 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.407 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.407 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-26T06:57:26.408 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-26T06:57:26.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.408 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.408 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.409 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.409 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.409 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.409 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.410 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.410 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.410 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.411 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2023-10-26T06:57:26.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2023-10-26T06:57:26.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.412 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.412 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.413 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.413 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.413 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.414 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.414 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.414 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.415 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2023-10-26T06:57:26.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-26T06:57:26.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.416 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.416 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:26.417 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.417 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.418 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.418 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.419 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-26T06:57:26.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-26T06:57:26.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.420 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.420 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.420 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.421 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.421 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.421 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.422 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.422 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.423 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-26T06:57:26.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-26T06:57:26.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.424 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.425 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.425 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.426 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.427 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-26T06:57:26.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-26T06:57:26.427 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.428 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.428 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.429 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.429 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.429 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.430 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.431 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-26T06:57:26.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-26T06:57:26.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.432 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.432 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.432 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.432 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.433 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.433 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.433 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.434 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.435 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2023-10-26T06:57:26.435 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-26T06:57:26.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.435 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.436 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.436 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.436 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.437 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.437 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.437 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.437 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.438 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.439 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-26T06:57:26.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-26T06:57:26.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.439 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.440 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.440 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.440 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.440 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.441 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.441 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.441 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.442 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.442 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-26T06:57:26.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-26T06:57:26.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.443 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.444 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.444 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.444 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.445 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.445 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.445 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.446 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.446 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.446 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-26T06:57:26.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-26T06:57:26.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.447 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.448 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.448 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.448 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.448 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.449 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.449 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.449 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.449 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.450 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.450 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-26T06:57:26.451 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-26T06:57:26.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.451 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.451 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.452 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.452 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.452 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.453 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.453 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.453 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.454 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.454 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.454 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-26T06:57:26.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-26T06:57:26.455 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.455 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.455 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.456 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.456 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.456 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.456 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.457 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.457 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.457 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.458 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.458 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-26T06:57:26.458 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-26T06:57:26.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.459 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.459 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.460 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.460 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.460 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.460 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.461 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.461 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.461 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.462 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.462 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-26T06:57:26.462 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-26T06:57:26.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.463 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.464 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.464 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.464 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.464 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.465 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.465 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.465 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.465 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.466 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-26T06:57:26.466 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-26T06:57:26.467 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.467 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.467 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.467 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.468 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.468 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.468 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.469 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.469 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.469 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.470 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2023-10-26T06:57:26.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-26T06:57:26.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.471 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.471 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.471 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.472 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.472 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.472 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.473 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.473 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.473 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.473 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.474 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-26T06:57:26.474 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-26T06:57:26.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.475 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.475 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.475 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-26T06:57:26.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.476 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.476 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.477 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.477 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.477 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.478 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-26T06:57:26.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-26T06:57:26.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.479 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.479 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:26.480 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.480 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.480 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.480 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.481 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.481 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.481 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.482 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-26T06:57:26.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-26T06:57:26.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.483 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.483 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2023-10-26T06:57:26.484 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.484 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.484 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.484 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.485 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.485 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.486 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-26T06:57:26.486 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-26T06:57:26.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.487 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.487 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.487 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2023-10-26T06:57:26.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.488 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.488 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.488 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.489 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.489 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.490 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2023-10-26T06:57:26.490 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-26T06:57:26.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.491 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.491 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.491 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.491 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.492 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.492 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.492 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.493 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.493 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.493 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.494 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-26T06:57:26.494 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-26T06:57:26.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.495 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:26.495 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.496 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.496 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.496 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.497 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.497 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.497 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.498 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2023-10-26T06:57:26.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-26T06:57:26.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.499 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.499 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.499 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.499 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.500 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.500 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.500 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.500 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.501 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.501 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.502 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2023-10-26T06:57:26.502 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-26T06:57:26.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.503 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.503 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.503 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.504 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.504 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.504 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.504 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.505 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.505 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.506 INFO:teuthology.orchestra.run.smithi089.stdout: "UNSAFE_TLS_v1_2": { 2023-10-26T06:57:26.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-26T06:57:26.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.506 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.507 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.507 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.508 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.508 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.508 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.508 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.509 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.509 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.510 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-26T06:57:26.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-26T06:57:26.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.511 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.511 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.511 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:26.511 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.512 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.512 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.512 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.513 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.513 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.513 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.514 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-26T06:57:26.514 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-26T06:57:26.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.515 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.515 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.515 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:26.515 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.516 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.516 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.516 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.517 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.517 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.518 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-26T06:57:26.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-26T06:57:26.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.519 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.519 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.519 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:26.519 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.520 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.520 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.520 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.521 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.521 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.521 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.522 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2023-10-26T06:57:26.522 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2023-10-26T06:57:26.522 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.525 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.525 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.526 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.526 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.526 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.526 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.527 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.527 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.527 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.528 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2023-10-26T06:57:26.528 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2023-10-26T06:57:26.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.529 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.529 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.529 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.530 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.530 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.530 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.530 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.531 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.531 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.532 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2023-10-26T06:57:26.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2023-10-26T06:57:26.532 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.533 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.533 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.533 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.534 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.534 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.534 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.534 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2023-10-26T06:57:26.535 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.535 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.535 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.536 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2023-10-26T06:57:26.536 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2023-10-26T06:57:26.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.537 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.537 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.537 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2023-10-26T06:57:26.538 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.538 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.538 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.538 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.539 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.539 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.539 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.540 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2023-10-26T06:57:26.540 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2023-10-26T06:57:26.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.541 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.541 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.541 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.542 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.542 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.542 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.543 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.543 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.543 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.544 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.545 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.545 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.545 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.545 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.546 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.546 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.546 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.547 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.547 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.547 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.548 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.548 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.548 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.548 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.549 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.549 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.549 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.550 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.550 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.550 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.551 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.551 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.551 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.551 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.552 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.552 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.552 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.553 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.553 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.553 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.554 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.554 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.554 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.555 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.555 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.555 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.556 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.556 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.556 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.556 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.557 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.557 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.557 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.558 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.558 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.558 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.558 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.559 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.559 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.560 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.560 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.560 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.560 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.561 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.561 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.561 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.562 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.562 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.562 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.563 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.563 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.563 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.564 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2023-10-26T06:57:26.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2023-10-26T06:57:26.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.565 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.565 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.565 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.565 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.566 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.566 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.566 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2023-10-26T06:57:26.567 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.567 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.568 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2023-10-26T06:57:26.568 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2023-10-26T06:57:26.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.569 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.569 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.569 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.569 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.570 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.570 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.570 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.571 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.571 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.571 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.572 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:26.572 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:26.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.573 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.573 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2023-10-26T06:57:26.574 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.574 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.574 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.574 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.575 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.575 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.575 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.576 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:26.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:26.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.577 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.577 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.577 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2023-10-26T06:57:26.578 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.578 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.578 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.578 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.579 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.579 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.579 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.580 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2023-10-26T06:57:26.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2023-10-26T06:57:26.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.581 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.581 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.581 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.581 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.582 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.582 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.582 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.583 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.583 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.583 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.584 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2023-10-26T06:57:26.584 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2023-10-26T06:57:26.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.585 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.585 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.585 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2023-10-26T06:57:26.585 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.586 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.586 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.586 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.587 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.587 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.587 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.588 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2023-10-26T06:57:26.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2023-10-26T06:57:26.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.588 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.589 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.589 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2023-10-26T06:57:26.589 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.590 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.590 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.590 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.590 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2023-10-26T06:57:26.591 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.591 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.591 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.592 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.592 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2023-10-26T06:57:26.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2023-10-26T06:57:26.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.593 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.593 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.594 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2023-10-26T06:57:26.594 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2023-10-26T06:57:26.594 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2023-10-26T06:57:26.595 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.595 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.595 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.595 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.596 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.596 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2023-10-26T06:57:26.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2023-10-26T06:57:26.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.597 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.597 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.598 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.598 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.598 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.598 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.599 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.599 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.599 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.600 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2023-10-26T06:57:26.601 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.601 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.602 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.602 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2023-10-26T06:57:26.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2023-10-26T06:57:26.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.603 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.603 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.603 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.604 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.604 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.604 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.605 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2023-10-26T06:57:26.605 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.605 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.605 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.606 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.607 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.607 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.607 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.608 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.608 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.608 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.609 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.609 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.609 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.609 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.610 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.610 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.610 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.611 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.611 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.611 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.611 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.612 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.612 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.613 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.613 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.613 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.613 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.614 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.614 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.614 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.615 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.615 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.615 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.615 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.616 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.616 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.617 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.617 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.618 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.618 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.618 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.618 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.619 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.619 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.619 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.620 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.620 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.620 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.620 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.621 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.621 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.621 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.622 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.622 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.623 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.623 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.623 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.624 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.624 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.624 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.624 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.625 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.625 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.625 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.626 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2023-10-26T06:57:26.626 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2023-10-26T06:57:26.626 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.627 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.627 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.627 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2023-10-26T06:57:26.627 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.628 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.628 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.628 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-26T06:57:26.629 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.629 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.630 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2023-10-26T06:57:26.630 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2023-10-26T06:57:26.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.630 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.631 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.631 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2023-10-26T06:57:26.631 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.632 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.632 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.632 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-26T06:57:26.632 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.633 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.633 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.634 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2023-10-26T06:57:26.634 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2023-10-26T06:57:26.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.634 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.635 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.635 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2023-10-26T06:57:26.635 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.636 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.636 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.636 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2023-10-26T06:57:26.636 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.637 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.637 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.638 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2023-10-26T06:57:26.638 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2023-10-26T06:57:26.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.638 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.639 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.639 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2023-10-26T06:57:26.639 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.640 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.640 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.640 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-26T06:57:26.640 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.641 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.641 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2023-10-26T06:57:26.642 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2023-10-26T06:57:26.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.642 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.643 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.643 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:26.643 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.643 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.644 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.644 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-26T06:57:26.644 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.644 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.645 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.645 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:26.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:26.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.646 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.646 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.647 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:26.647 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.647 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.648 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.648 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2023-10-26T06:57:26.648 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.648 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.649 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.649 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2023-10-26T06:57:26.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2023-10-26T06:57:26.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.650 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.650 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.651 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2023-10-26T06:57:26.651 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.651 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.652 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.652 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-26T06:57:26.652 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.652 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.653 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.654 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2023-10-26T06:57:26.654 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.655 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.655 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.655 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.655 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.656 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.656 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.657 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.657 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.657 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.658 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.658 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.658 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.659 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.659 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.659 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.659 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.660 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.660 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.660 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.661 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.661 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.661 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.662 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.662 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.663 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.663 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.663 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.663 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.664 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.664 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.664 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.664 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.665 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.665 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.666 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.667 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.667 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.667 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.667 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.668 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.668 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.668 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.669 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.669 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.669 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.669 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.670 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.670 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.670 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.671 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.671 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.672 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.673 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.674 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.674 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.674 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.674 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.675 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.675 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2023-10-26T06:57:26.676 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2023-10-26T06:57:26.676 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.676 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.676 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.677 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2023-10-26T06:57:26.677 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.677 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.678 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.678 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.678 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.678 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.679 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.679 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2023-10-26T06:57:26.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2023-10-26T06:57:26.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.680 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.680 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.681 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2023-10-26T06:57:26.681 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.681 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.682 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.682 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.682 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.682 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.683 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.683 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:26.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:26.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.684 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.684 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.685 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:26.685 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.685 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.686 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.686 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.686 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.686 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.687 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.688 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2023-10-26T06:57:26.688 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2023-10-26T06:57:26.689 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2023-10-26T06:57:26.689 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.689 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2023-10-26T06:57:26.690 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2023-10-26T06:57:26.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.690 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.690 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.691 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2023-10-26T06:57:26.691 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.691 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.692 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.692 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-26T06:57:26.692 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.693 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.693 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.693 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2023-10-26T06:57:26.694 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2023-10-26T06:57:26.694 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.694 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.695 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.695 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2023-10-26T06:57:26.695 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.695 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.696 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.696 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-26T06:57:26.696 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.697 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.697 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.697 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2023-10-26T06:57:26.698 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2023-10-26T06:57:26.698 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.698 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.699 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.699 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.699 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.699 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.700 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.700 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2023-10-26T06:57:26.700 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.701 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.701 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.701 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:26.702 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:26.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:26.702 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.703 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2023-10-26T06:57:26.703 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2023-10-26T06:57:26.703 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.704 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.704 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-26T06:57:26.704 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.705 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.705 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.706 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.706 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.706 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.707 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.707 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.707 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.708 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.708 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.708 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.708 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.709 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.709 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.709 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.710 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.710 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.710 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.710 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.711 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.711 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.712 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.712 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.712 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.713 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.713 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.713 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.713 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.714 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.714 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.714 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.715 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.715 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.715 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.716 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.716 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.716 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.716 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.717 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.717 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.717 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.718 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.718 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.718 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.718 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.719 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.719 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.719 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.720 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.720 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.720 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.720 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.721 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.721 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.722 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.722 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.722 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.723 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.725 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.726 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.726 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.726 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.727 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.727 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2023-10-26T06:57:26.728 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2023-10-26T06:57:26.728 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.728 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.728 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.729 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.729 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.729 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.730 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.730 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2023-10-26T06:57:26.730 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.731 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.731 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.731 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2023-10-26T06:57:26.732 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2023-10-26T06:57:26.732 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.732 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.733 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.733 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2023-10-26T06:57:26.733 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.734 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.734 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.734 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2023-10-26T06:57:26.734 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.735 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.735 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.736 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2023-10-26T06:57:26.736 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2023-10-26T06:57:26.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.736 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.737 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.737 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2023-10-26T06:57:26.737 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.738 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.738 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.738 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-26T06:57:26.738 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.739 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.739 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.739 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2023-10-26T06:57:26.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2023-10-26T06:57:26.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.740 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.741 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.741 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:26.741 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:26.741 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2023-10-26T06:57:26.742 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.742 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-26T06:57:26.742 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.743 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.743 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.744 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2023-10-26T06:57:26.744 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2023-10-26T06:57:26.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.744 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.745 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.745 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.745 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.745 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.746 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.746 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2023-10-26T06:57:26.746 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.747 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.747 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.747 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2023-10-26T06:57:26.748 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2023-10-26T06:57:26.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.748 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.749 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2023-10-26T06:57:26.749 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.749 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.750 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.750 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-26T06:57:26.750 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.750 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.751 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.751 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.751 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2023-10-26T06:57:26.752 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.753 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.753 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.753 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.753 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.754 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.754 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.755 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.755 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.755 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.755 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.756 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.756 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.756 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.757 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.757 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.757 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.757 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.758 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.758 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.758 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.759 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.759 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.759 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.760 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.760 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.760 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.761 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.761 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.761 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.762 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.762 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.762 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.762 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.763 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.763 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.763 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.764 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.764 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.764 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.764 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.765 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.765 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.765 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.765 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.766 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.766 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.766 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.767 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.767 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.767 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.767 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.768 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.768 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.769 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.769 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.769 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.769 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.770 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.770 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.770 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.771 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.771 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.771 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.771 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.772 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.772 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.772 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.773 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.773 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2023-10-26T06:57:26.774 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.774 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.774 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.774 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.775 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.775 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.775 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.776 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.776 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.776 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.776 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.777 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.777 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.777 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.777 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.778 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.778 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.778 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.779 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.779 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.779 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.779 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.780 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.780 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.781 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.782 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.782 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.782 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.782 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.783 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.783 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.783 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.784 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.784 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.784 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.785 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.785 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.786 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.786 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.786 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.787 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.788 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.788 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.788 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.789 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.790 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.791 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.792 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.793 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.794 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.794 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2023-10-26T06:57:26.795 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2023-10-26T06:57:26.796 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2023-10-26T06:57:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.797 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2023-10-26T06:57:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.798 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-26T06:57:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.799 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2023-10-26T06:57:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2023-10-26T06:57:26.800 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.801 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:26.802 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2023-10-26T06:57:26.802 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.802 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-26T06:57:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.803 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.804 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.804 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.805 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.805 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.805 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.805 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.806 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.807 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.808 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.809 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.811 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.812 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.812 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.812 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.812 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.813 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.813 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.813 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.814 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.814 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.815 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.816 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.817 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.818 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.820 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.821 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.822 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.824 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2023-10-26T06:57:26.825 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.825 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.825 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.825 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2023-10-26T06:57:26.826 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2023-10-26T06:57:26.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.826 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.826 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.827 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2023-10-26T06:57:26.827 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.827 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.828 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.828 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2023-10-26T06:57:26.828 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-26T06:57:26.828 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.829 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.829 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.830 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.830 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.830 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.831 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.831 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.831 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.832 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.832 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.832 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.832 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.833 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.833 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.833 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.834 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.834 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.834 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.835 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.835 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.836 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.836 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.836 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.837 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.837 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.837 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.838 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.838 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.838 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.838 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.839 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.840 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.840 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.840 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.841 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.841 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.841 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.841 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.842 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.842 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.842 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.843 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.843 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.843 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.843 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.844 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.844 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.844 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.845 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.846 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.846 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.846 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.846 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.847 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.847 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.847 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.848 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.848 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.848 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.849 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2023-10-26T06:57:26.849 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2023-10-26T06:57:26.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.850 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.850 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.850 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.850 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.851 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.851 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.851 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2023-10-26T06:57:26.852 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-26T06:57:26.852 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.852 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.853 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2023-10-26T06:57:26.853 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2023-10-26T06:57:26.853 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.854 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.854 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.854 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2023-10-26T06:57:26.854 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.855 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.855 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.855 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2023-10-26T06:57:26.856 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.856 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.856 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.857 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2023-10-26T06:57:26.857 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2023-10-26T06:57:26.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:26.857 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.858 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.858 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2023-10-26T06:57:26.858 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.859 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.859 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.859 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2023-10-26T06:57:26.859 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.860 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.860 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.860 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2023-10-26T06:57:26.861 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2023-10-26T06:57:26.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.861 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.862 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.862 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.862 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.862 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.863 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.863 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2023-10-26T06:57:26.863 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.864 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.864 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.864 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2023-10-26T06:57:26.865 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2023-10-26T06:57:26.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.865 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.865 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.866 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2023-10-26T06:57:26.866 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.866 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.867 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.867 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-26T06:57:26.867 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-26T06:57:26.867 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.868 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2023-10-26T06:57:26.869 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.870 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.870 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.870 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.871 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.871 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.871 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.872 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.872 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.872 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.872 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.873 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.873 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.873 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.874 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.874 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.874 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.875 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.875 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.875 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.875 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.876 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.876 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.876 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.877 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.877 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.877 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.877 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.878 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.878 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.878 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.879 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.879 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.879 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.879 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.880 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.881 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.882 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.883 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.884 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.885 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.886 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.888 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.889 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2023-10-26T06:57:26.891 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.892 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.893 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.894 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.895 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.896 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.897 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.899 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.900 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.901 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.903 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.904 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.905 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.906 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.907 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.908 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.909 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.910 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.911 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2023-10-26T06:57:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.913 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.915 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.916 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.917 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.918 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.919 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.921 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.922 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.925 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.926 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.927 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.928 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.930 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.931 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.932 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.933 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2023-10-26T06:57:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.935 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2023-10-26T06:57:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2023-10-26T06:57:26.936 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.937 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.938 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-26T06:57:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.940 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.941 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.942 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.943 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.944 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.946 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.947 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.948 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.949 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.950 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.951 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.952 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.953 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.954 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.955 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.956 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.957 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.958 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.959 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2023-10-26T06:57:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2023-10-26T06:57:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.960 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.961 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2023-10-26T06:57:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2023-10-26T06:57:26.962 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2023-10-26T06:57:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2023-10-26T06:57:26.963 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2023-10-26T06:57:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.968 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.969 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.970 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.971 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.972 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.973 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.974 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.975 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.976 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.977 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:26.978 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.979 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.980 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.981 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:26.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.984 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.985 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:26.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2023-10-26T06:57:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:26.988 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:26.989 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:26.990 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:26.991 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:26.992 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.993 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.995 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:26.995 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:26.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:26.995 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:26.996 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:26.996 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:26.996 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:26.997 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:26.997 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:26.997 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:26.997 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:26.998 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:26.998 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:26.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:26.999 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:26.999 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:26.999 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.000 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.000 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.000 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.000 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.001 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.001 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.001 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.002 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.003 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.006 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.007 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.008 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.009 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2023-10-26T06:57:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.010 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.012 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.013 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.014 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.015 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.017 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.018 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.019 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.021 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.022 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.023 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.024 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.025 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.026 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.027 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.028 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.029 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.030 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.031 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:27.032 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.033 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2023-10-26T06:57:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2023-10-26T06:57:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.035 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2023-10-26T06:57:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2023-10-26T06:57:27.036 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2023-10-26T06:57:27.037 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-26T06:57:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2023-10-26T06:57:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.040 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2023-10-26T06:57:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2023-10-26T06:57:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.041 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.042 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-26T06:57:27.043 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.044 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2023-10-26T06:57:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2023-10-26T06:57:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.045 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.046 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.047 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.048 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.049 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.050 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.051 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.052 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.054 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.055 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.056 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.057 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.059 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.060 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.061 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.062 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.064 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.065 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.066 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.067 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2023-10-26T06:57:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2023-10-26T06:57:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.068 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2023-10-26T06:57:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.069 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2023-10-26T06:57:27.070 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.071 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.072 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:27.073 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2023-10-26T06:57:27.074 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2023-10-26T06:57:27.076 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2023-10-26T06:57:27.077 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2023-10-26T06:57:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.078 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.079 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.080 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2023-10-26T06:57:27.081 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2023-10-26T06:57:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.082 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.083 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-26T06:57:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-26T06:57:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.084 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.085 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.086 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.087 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.088 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.089 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.090 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.092 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.093 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.094 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.096 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.097 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.098 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.099 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.100 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.101 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.103 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.104 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2023-10-26T06:57:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2023-10-26T06:57:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.105 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.106 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.107 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-26T06:57:27.108 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-26T06:57:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.109 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2023-10-26T06:57:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.110 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.111 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2023-10-26T06:57:27.112 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2023-10-26T06:57:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.113 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2023-10-26T06:57:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.114 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.115 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:27.116 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.117 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2023-10-26T06:57:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.118 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-26T06:57:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.119 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:27.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.121 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2023-10-26T06:57:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.122 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-26T06:57:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.123 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2023-10-26T06:57:27.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2023-10-26T06:57:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.125 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2023-10-26T06:57:27.126 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.126 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.126 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.127 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.127 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.127 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.128 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.128 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2023-10-26T06:57:27.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2023-10-26T06:57:27.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.129 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.129 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.130 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2023-10-26T06:57:27.130 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.130 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.131 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.131 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2023-10-26T06:57:27.131 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2023-10-26T06:57:27.131 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.132 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.132 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.132 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.133 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.133 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2023-10-26T06:57:27.133 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2023-10-26T06:57:27.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.134 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.137 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.137 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2023-10-26T06:57:27.138 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2023-10-26T06:57:27.138 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2023-10-26T06:57:27.138 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.139 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.139 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.139 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.139 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.141 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2023-10-26T06:57:27.141 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.141 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.142 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.142 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.142 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.143 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.143 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.143 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.144 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.144 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.144 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.144 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.145 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.145 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.145 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.145 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.146 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.146 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.146 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.147 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.147 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.147 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.148 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.148 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.149 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.149 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.149 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.149 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.150 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.150 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.150 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.151 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.151 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.151 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.152 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.152 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.153 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.153 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.153 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.153 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.154 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.154 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.154 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.155 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.155 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.155 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.155 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.156 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.156 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.156 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.156 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.157 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.157 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.157 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.158 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.158 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.159 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.159 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.159 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.160 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.160 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.160 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.160 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.161 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.162 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2023-10-26T06:57:27.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2023-10-26T06:57:27.162 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.163 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.163 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.163 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:27.163 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.164 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.164 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.164 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.165 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.165 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.165 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.166 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2023-10-26T06:57:27.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2023-10-26T06:57:27.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.167 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.167 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.167 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.167 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.168 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.168 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.168 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.168 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.169 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.169 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.170 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2023-10-26T06:57:27.170 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2023-10-26T06:57:27.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.170 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.171 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.171 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.171 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.172 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.172 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.172 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.173 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.173 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.173 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.173 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.175 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2023-10-26T06:57:27.175 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.175 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.175 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.176 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2023-10-26T06:57:27.176 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2023-10-26T06:57:27.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.176 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.177 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.177 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.177 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.178 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.178 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.178 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.179 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.179 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.179 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.180 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2023-10-26T06:57:27.180 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2023-10-26T06:57:27.180 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.180 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.181 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.181 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.181 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.182 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.182 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.182 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.182 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.183 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.183 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.183 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.184 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.184 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.184 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.185 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.185 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.185 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.185 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.186 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.186 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.186 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.186 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.187 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.187 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.187 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.187 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.188 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.188 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.189 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.189 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.189 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.190 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.190 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.191 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.191 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.191 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.192 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.192 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.192 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.192 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.193 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.193 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2023-10-26T06:57:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.218 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.218 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.219 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.220 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.220 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.220 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.221 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.221 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.221 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.222 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.222 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.222 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.223 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.224 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.224 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.224 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.224 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.225 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.225 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.225 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.226 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.226 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.226 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.227 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.227 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.227 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.227 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.228 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.228 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.228 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.229 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.229 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.230 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.230 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.230 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.231 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.231 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.231 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.232 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.232 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.232 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2023-10-26T06:57:27.234 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.234 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.235 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.235 INFO:teuthology.orchestra.run.smithi089.stdout: "drive_group_interval": { 2023-10-26T06:57:27.235 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "drive_group_interval", 2023-10-26T06:57:27.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.236 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.236 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.236 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300.0", 2023-10-26T06:57:27.237 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.237 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.237 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.237 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-26T06:57:27.238 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.238 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.238 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.239 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.240 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.240 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.240 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.241 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.241 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.241 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.241 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.242 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.242 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.242 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.243 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.243 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.243 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.244 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.246 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.246 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.247 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.247 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.247 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.247 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.248 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.248 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.249 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.249 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.249 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.250 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.250 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.250 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.251 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.251 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.251 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.251 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.252 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.252 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.252 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.253 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.253 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.253 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.253 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.254 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.254 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.254 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.255 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.255 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.256 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.256 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.256 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.256 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.257 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.257 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.257 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.258 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.258 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.258 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.259 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2023-10-26T06:57:27.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2023-10-26T06:57:27.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.260 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.260 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.260 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2023-10-26T06:57:27.261 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.261 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.261 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.261 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-26T06:57:27.262 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.262 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.262 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.262 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.263 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.263 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2023-10-26T06:57:27.264 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.264 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.264 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.265 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.265 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.265 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.266 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.266 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.266 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.266 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.267 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.267 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.267 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.267 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.268 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.268 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.268 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.268 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.269 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.269 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.269 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.270 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.270 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.270 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.271 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.271 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.273 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.273 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.273 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.273 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.274 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.274 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.275 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.275 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.276 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.276 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.276 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.277 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.277 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.277 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.277 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.278 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.278 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.278 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.279 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.279 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.279 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.279 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.280 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.280 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.281 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.281 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.281 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.281 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.282 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.282 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.282 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.283 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.283 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.283 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.283 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.284 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.284 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2023-10-26T06:57:27.284 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2023-10-26T06:57:27.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.285 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.285 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.286 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.286 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.286 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.286 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.287 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.287 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.287 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.288 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.288 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2023-10-26T06:57:27.288 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2023-10-26T06:57:27.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.289 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.289 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.289 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2023-10-26T06:57:27.290 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.290 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.290 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.291 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.291 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.291 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.291 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.292 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2023-10-26T06:57:27.292 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2023-10-26T06:57:27.293 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.293 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.293 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.293 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.294 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.294 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.294 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.295 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.295 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.295 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.295 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.296 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2023-10-26T06:57:27.296 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2023-10-26T06:57:27.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.297 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.297 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.297 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.298 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.298 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.298 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.299 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.299 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.299 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.299 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.300 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2023-10-26T06:57:27.300 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2023-10-26T06:57:27.300 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.301 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.301 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.301 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.302 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.302 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.302 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.303 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.303 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.303 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.303 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.304 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2023-10-26T06:57:27.304 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2023-10-26T06:57:27.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.305 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.305 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.305 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.306 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.306 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.306 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.307 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.307 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.307 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.307 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.308 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2023-10-26T06:57:27.308 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2023-10-26T06:57:27.308 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.309 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.309 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.309 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.309 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.310 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.310 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.310 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.311 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.311 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.311 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.312 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2023-10-26T06:57:27.312 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2023-10-26T06:57:27.312 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.313 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.313 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.313 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.313 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.314 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.314 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.314 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.315 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.315 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.315 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.316 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2023-10-26T06:57:27.316 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2023-10-26T06:57:27.316 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.316 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.317 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.317 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.317 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:27.318 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2023-10-26T06:57:27.318 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.318 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.319 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.319 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.319 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.320 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2023-10-26T06:57:27.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2023-10-26T06:57:27.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.320 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.321 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.321 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.321 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.322 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.322 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.322 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.322 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.323 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.323 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2023-10-26T06:57:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2023-10-26T06:57:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.324 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.325 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.325 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.325 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.326 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.326 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.326 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.326 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.327 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.327 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.328 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2023-10-26T06:57:27.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2023-10-26T06:57:27.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.328 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.329 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.329 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.329 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.329 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.330 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.330 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.330 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.330 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.331 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2023-10-26T06:57:27.333 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.333 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.333 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.333 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2023-10-26T06:57:27.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2023-10-26T06:57:27.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.334 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.334 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.335 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.335 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.336 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2023-10-26T06:57:27.336 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.336 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.337 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2023-10-26T06:57:27.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2023-10-26T06:57:27.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.338 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.339 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.339 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.339 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.339 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.340 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2023-10-26T06:57:27.340 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.341 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.341 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.342 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.342 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.345 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.345 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.345 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.345 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.346 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.346 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.346 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.347 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.347 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.348 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.348 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.348 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.349 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.349 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.349 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.349 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.350 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.350 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.350 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.350 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.351 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.351 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.352 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.352 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.352 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.352 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.353 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.353 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.353 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.354 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.354 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.354 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.354 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.355 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.355 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.355 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.356 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.356 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.356 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.356 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.357 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.357 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.358 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.358 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.358 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.358 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.359 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.359 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.359 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.359 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.360 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.360 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.360 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.361 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.361 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.361 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.362 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2023-10-26T06:57:27.362 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.362 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.363 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.363 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.363 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.364 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.364 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.364 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.365 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.365 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.366 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.366 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.366 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.366 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.367 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.367 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.367 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.367 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.368 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.368 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.368 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.369 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.369 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.370 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.370 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.370 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.371 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.371 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.371 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.371 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.372 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.372 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.372 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.373 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.373 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.374 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.374 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.374 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.375 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.375 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.375 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.375 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.376 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.376 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.376 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2023-10-26T06:57:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.402 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.402 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.402 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.403 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.403 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.403 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.404 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.404 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2023-10-26T06:57:27.406 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.406 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.407 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.407 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2023-10-26T06:57:27.407 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2023-10-26T06:57:27.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.408 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.408 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.408 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-26T06:57:27.408 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.409 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.409 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.409 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.410 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.410 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.411 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:27.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:27.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.412 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.412 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2023-10-26T06:57:27.412 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.413 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.413 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.413 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.414 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.414 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.415 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.416 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.416 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.416 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.417 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.418 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.418 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.418 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.418 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.419 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.419 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.419 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.420 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.420 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.420 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.421 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.421 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.422 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.422 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.422 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.422 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.423 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.423 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.423 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.424 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.424 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.424 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.425 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.426 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.426 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.427 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.427 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.427 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.427 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.428 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.428 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.428 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.429 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.429 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.429 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.429 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.430 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.431 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.432 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.432 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.432 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.433 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.433 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.434 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.434 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.435 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.435 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2023-10-26T06:57:27.436 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.436 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.436 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.437 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2023-10-26T06:57:27.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2023-10-26T06:57:27.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-26T06:57:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2023-10-26T06:57:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2023-10-26T06:57:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-26T06:57:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2023-10-26T06:57:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2023-10-26T06:57:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-26T06:57:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2023-10-26T06:57:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2023-10-26T06:57:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-26T06:57:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2023-10-26T06:57:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2023-10-26T06:57:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-26T06:57:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2023-10-26T06:57:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2023-10-26T06:57:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2023-10-26T06:57:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2023-10-26T06:57:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2023-10-26T06:57:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2023-10-26T06:57:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-26T06:57:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2023-10-26T06:57:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2023-10-26T06:57:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2023-10-26T06:57:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2023-10-26T06:57:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2023-10-26T06:57:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2023-10-26T06:57:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2023-10-26T06:57:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2023-10-26T06:57:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2023-10-26T06:57:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2023-10-26T06:57:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2023-10-26T06:57:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2023-10-26T06:57:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2023-10-26T06:57:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2023-10-26T06:57:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2023-10-26T06:57:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2023-10-26T06:57:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2023-10-26T06:57:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-26T06:57:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2023-10-26T06:57:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2023-10-26T06:57:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.544 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.545 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.545 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.560 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.560 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.560 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.561 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.561 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.561 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.562 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.563 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.563 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.563 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.563 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.564 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.564 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.564 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.564 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.565 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.565 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2023-10-26T06:57:27.566 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2023-10-26T06:57:27.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.566 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.566 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.567 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2023-10-26T06:57:27.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.567 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.567 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.568 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-26T06:57:27.568 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.568 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.568 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.569 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2023-10-26T06:57:27.569 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2023-10-26T06:57:27.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.570 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.570 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.570 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.571 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.571 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.571 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.572 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2023-10-26T06:57:27.572 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.572 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.572 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.573 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2023-10-26T06:57:27.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2023-10-26T06:57:27.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.574 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.574 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.574 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:27.575 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.575 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.575 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.575 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-26T06:57:27.576 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.576 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.576 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.576 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.578 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2023-10-26T06:57:27.578 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.578 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.578 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.579 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2023-10-26T06:57:27.579 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2023-10-26T06:57:27.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:27.579 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.580 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.580 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2023-10-26T06:57:27.580 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.581 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.581 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.581 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.581 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.582 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.582 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.583 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2023-10-26T06:57:27.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2023-10-26T06:57:27.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.583 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.584 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.585 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.585 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.585 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.585 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.586 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.587 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:27.587 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:27.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.587 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.588 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:27.588 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.589 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.589 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.589 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.589 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.590 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.590 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.590 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.591 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.591 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.592 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.592 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.592 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.593 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.593 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.593 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.593 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.594 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.594 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.594 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.595 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.595 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.595 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.595 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.596 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.596 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.597 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.597 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.598 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.598 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.598 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.599 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.599 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.599 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.599 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.600 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.600 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.601 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.601 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.602 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.602 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.602 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.602 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.603 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.603 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.603 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.604 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.604 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.604 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.605 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.605 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.605 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.605 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.606 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.606 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.607 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.607 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.608 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.608 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.608 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.608 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.609 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.609 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.609 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.609 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.610 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2023-10-26T06:57:27.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2023-10-26T06:57:27.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.611 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.611 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.612 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.612 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.612 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.613 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.613 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.613 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.613 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.614 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2023-10-26T06:57:27.614 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2023-10-26T06:57:27.615 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.615 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.615 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.615 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2023-10-26T06:57:27.616 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.616 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.616 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.617 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.617 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.617 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.617 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.618 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2023-10-26T06:57:27.618 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2023-10-26T06:57:27.619 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.619 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.619 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.619 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-26T06:57:27.620 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.620 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.620 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.621 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.621 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.621 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.621 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.623 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:27.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.623 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.624 INFO:teuthology.orchestra.run.smithi089.stdout: "modules": [ 2023-10-26T06:57:27.624 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2023-10-26T06:57:27.624 INFO:teuthology.orchestra.run.smithi089.stdout: "dashboard", 2023-10-26T06:57:27.624 INFO:teuthology.orchestra.run.smithi089.stdout: "iostat", 2023-10-26T06:57:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "nfs", 2023-10-26T06:57:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "restful", 2023-10-26T06:57:27.625 INFO:teuthology.orchestra.run.smithi089.stdout: "snap_schedule" 2023-10-26T06:57:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2023-10-26T06:57:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.626 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2023-10-26T06:57:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.627 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:27.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:27.629 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.630 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2023-10-26T06:57:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.633 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.634 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.635 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.636 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.639 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.640 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.643 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.644 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.645 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.646 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.648 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.649 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.650 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2023-10-26T06:57:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2023-10-26T06:57:27.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.653 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to", 2023-10-26T06:57:27.654 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2023-10-26T06:57:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2023-10-26T06:57:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.656 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2023-10-26T06:57:27.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2023-10-26T06:57:27.658 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2023-10-26T06:57:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2023-10-26T06:57:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.660 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.661 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2023-10-26T06:57:27.662 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.663 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2023-10-26T06:57:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2023-10-26T06:57:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.664 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2023-10-26T06:57:27.666 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2023-10-26T06:57:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2023-10-26T06:57:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.668 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2023-10-26T06:57:27.669 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2023-10-26T06:57:27.670 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2023-10-26T06:57:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2023-10-26T06:57:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2023-10-26T06:57:27.674 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2023-10-26T06:57:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2023-10-26T06:57:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.676 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.677 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-26T06:57:27.678 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.679 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2023-10-26T06:57:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2023-10-26T06:57:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.680 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.681 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2023-10-26T06:57:27.682 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.683 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.683 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2023-10-26T06:57:27.684 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.685 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.685 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.685 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2023-10-26T06:57:27.686 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2023-10-26T06:57:27.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.686 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.686 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.687 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.687 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.687 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.687 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.688 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2023-10-26T06:57:27.688 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.688 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.689 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.689 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2023-10-26T06:57:27.689 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2023-10-26T06:57:27.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.690 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.690 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.691 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2023-10-26T06:57:27.691 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.691 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.691 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.692 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2023-10-26T06:57:27.692 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-26T06:57:27.692 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.693 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.693 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2023-10-26T06:57:27.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2023-10-26T06:57:27.694 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:27.694 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.694 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.695 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:27.695 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2023-10-26T06:57:27.695 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2023-10-26T06:57:27.695 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.696 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-26T06:57:27.696 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-26T06:57:27.696 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.697 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.697 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2023-10-26T06:57:27.697 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2023-10-26T06:57:27.698 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:27.698 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.698 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.699 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2023-10-26T06:57:27.699 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:27.699 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2023-10-26T06:57:27.699 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.700 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-26T06:57:27.700 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.700 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.700 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.701 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2023-10-26T06:57:27.701 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2023-10-26T06:57:27.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.702 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.703 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2023-10-26T06:57:27.703 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.703 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.703 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.704 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-26T06:57:27.704 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-26T06:57:27.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.704 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.705 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2023-10-26T06:57:27.705 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2023-10-26T06:57:27.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.706 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.706 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.706 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2023-10-26T06:57:27.707 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2023-10-26T06:57:27.707 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2023-10-26T06:57:27.707 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.708 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2023-10-26T06:57:27.708 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-26T06:57:27.708 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.708 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.709 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2023-10-26T06:57:27.709 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2023-10-26T06:57:27.709 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.710 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.710 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2023-10-26T06:57:27.711 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.711 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.711 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.712 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2023-10-26T06:57:27.712 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-26T06:57:27.712 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.712 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.713 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2023-10-26T06:57:27.713 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2023-10-26T06:57:27.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:27.714 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.714 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:27.715 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2023-10-26T06:57:27.715 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2023-10-26T06:57:27.715 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.715 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-26T06:57:27.716 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-26T06:57:27.716 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.716 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.718 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.718 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.718 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.719 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.720 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.721 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.721 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.721 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.722 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.722 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.722 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.723 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.723 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.724 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.724 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.724 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.725 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.725 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.725 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.726 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.726 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.726 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.727 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.727 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.728 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.728 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.728 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.728 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.729 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.729 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.729 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.729 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.730 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.730 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.730 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.731 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.731 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.731 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.731 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.732 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.732 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.733 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.733 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.734 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.734 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.734 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.734 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.735 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.735 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.735 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.736 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.736 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.736 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.737 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2023-10-26T06:57:27.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2023-10-26T06:57:27.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.737 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.738 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.738 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:27.738 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.739 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.739 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.739 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-26T06:57:27.739 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.740 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.740 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.740 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2023-10-26T06:57:27.741 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2023-10-26T06:57:27.741 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.741 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.742 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.742 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2023-10-26T06:57:27.742 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.742 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.743 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.743 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2023-10-26T06:57:27.743 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2023-10-26T06:57:27.744 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap" 2023-10-26T06:57:27.744 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.744 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2023-10-26T06:57:27.744 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.745 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.745 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.745 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2023-10-26T06:57:27.746 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2023-10-26T06:57:27.746 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.746 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.749 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.749 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.750 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.750 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.750 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.750 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-26T06:57:27.751 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.751 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.751 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.752 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:27.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:27.752 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.753 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.753 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.753 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:27.754 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.754 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.754 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.754 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-26T06:57:27.755 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.755 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.755 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.756 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2023-10-26T06:57:27.756 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2023-10-26T06:57:27.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.756 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.757 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.757 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:27.757 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:27.757 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-26T06:57:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-26T06:57:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2023-10-26T06:57:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2023-10-26T06:57:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-26T06:57:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2023-10-26T06:57:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2023-10-26T06:57:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2023-10-26T06:57:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2023-10-26T06:57:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-26T06:57:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2023-10-26T06:57:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2023-10-26T06:57:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2023-10-26T06:57:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-26T06:57:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2023-10-26T06:57:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2023-10-26T06:57:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2023-10-26T06:57:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-26T06:57:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2023-10-26T06:57:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2023-10-26T06:57:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-26T06:57:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-26T06:57:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2023-10-26T06:57:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2023-10-26T06:57:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.784 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-26T06:57:27.784 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.784 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.785 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.785 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2023-10-26T06:57:27.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2023-10-26T06:57:27.786 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:27.786 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.786 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.787 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2023-10-26T06:57:27.787 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.787 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.787 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.788 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-26T06:57:27.788 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.788 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.788 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.789 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2023-10-26T06:57:27.789 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2023-10-26T06:57:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.790 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.791 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.791 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.791 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.792 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2023-10-26T06:57:27.792 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2023-10-26T06:57:27.792 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2023-10-26T06:57:27.792 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.793 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-26T06:57:27.793 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.793 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.794 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.794 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2023-10-26T06:57:27.794 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2023-10-26T06:57:27.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.795 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.795 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.796 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.796 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.796 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.796 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.797 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-26T06:57:27.797 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.797 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.797 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.798 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2023-10-26T06:57:27.798 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2023-10-26T06:57:27.798 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.799 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.799 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.799 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.800 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.800 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.800 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.800 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-26T06:57:27.801 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.801 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.801 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.802 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2023-10-26T06:57:27.802 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2023-10-26T06:57:27.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.803 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.803 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.803 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.804 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.804 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.804 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.804 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-26T06:57:27.805 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.805 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.805 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.806 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2023-10-26T06:57:27.806 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2023-10-26T06:57:27.806 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.806 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.807 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.807 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-26T06:57:27.807 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.807 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.808 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.808 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:27.808 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.808 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.809 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.809 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2023-10-26T06:57:27.810 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2023-10-26T06:57:27.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.810 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.810 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.811 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-26T06:57:27.811 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.811 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.812 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.812 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2023-10-26T06:57:27.812 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.812 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.813 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.813 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2023-10-26T06:57:27.814 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2023-10-26T06:57:27.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.814 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.814 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-26T06:57:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.815 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2023-10-26T06:57:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.816 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2023-10-26T06:57:27.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2023-10-26T06:57:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.818 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-26T06:57:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.819 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.820 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2023-10-26T06:57:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2023-10-26T06:57:27.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.822 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-26T06:57:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.823 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2023-10-26T06:57:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.824 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2023-10-26T06:57:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2023-10-26T06:57:27.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.826 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-26T06:57:27.827 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.827 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.827 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.827 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2023-10-26T06:57:27.828 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.828 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.828 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.829 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2023-10-26T06:57:27.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2023-10-26T06:57:27.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.830 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.830 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.830 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-26T06:57:27.830 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.831 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.831 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.831 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2023-10-26T06:57:27.832 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.832 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.832 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.833 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2023-10-26T06:57:27.833 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2023-10-26T06:57:27.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.833 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.834 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.834 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-26T06:57:27.834 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.834 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.835 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.835 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2023-10-26T06:57:27.835 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.836 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.836 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.836 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2023-10-26T06:57:27.837 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2023-10-26T06:57:27.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.837 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.838 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.838 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-26T06:57:27.838 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.838 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.839 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.839 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2023-10-26T06:57:27.839 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.840 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.840 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.841 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2023-10-26T06:57:27.841 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2023-10-26T06:57:27.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.841 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.842 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.842 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-26T06:57:27.842 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.843 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.843 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.843 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2023-10-26T06:57:27.843 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.844 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.844 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.844 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2023-10-26T06:57:27.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2023-10-26T06:57:27.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.845 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.846 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.846 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-26T06:57:27.846 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.846 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.847 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.847 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:27.847 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.848 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.848 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.848 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2023-10-26T06:57:27.849 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2023-10-26T06:57:27.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.849 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.849 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.850 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-26T06:57:27.850 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.850 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.851 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.851 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2023-10-26T06:57:27.851 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.851 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.852 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.852 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2023-10-26T06:57:27.853 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2023-10-26T06:57:27.853 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.853 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.853 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.854 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-26T06:57:27.854 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.854 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.855 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.855 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2023-10-26T06:57:27.855 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.855 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.856 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.856 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2023-10-26T06:57:27.856 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2023-10-26T06:57:27.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.857 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.857 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.857 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-26T06:57:27.858 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.858 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.858 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.859 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2023-10-26T06:57:27.859 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.859 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.860 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.860 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2023-10-26T06:57:27.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2023-10-26T06:57:27.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.861 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.861 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.861 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-26T06:57:27.862 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.862 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.862 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.862 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2023-10-26T06:57:27.863 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.863 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.863 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.864 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2023-10-26T06:57:27.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2023-10-26T06:57:27.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.865 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.865 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.865 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.866 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.866 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.866 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.866 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2023-10-26T06:57:27.867 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.867 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.867 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.868 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2023-10-26T06:57:27.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2023-10-26T06:57:27.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.869 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.869 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.869 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:27.869 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.870 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.870 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.870 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-26T06:57:27.870 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.871 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.871 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.872 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2023-10-26T06:57:27.872 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2023-10-26T06:57:27.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.872 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.873 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.873 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2023-10-26T06:57:27.873 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.874 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.874 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.874 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-26T06:57:27.874 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.875 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.875 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.876 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2023-10-26T06:57:27.876 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2023-10-26T06:57:27.876 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.876 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.877 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.877 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2023-10-26T06:57:27.877 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.878 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.878 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.878 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-26T06:57:27.878 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.879 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.879 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.879 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2023-10-26T06:57:27.880 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2023-10-26T06:57:27.880 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.880 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.881 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.881 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2023-10-26T06:57:27.881 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.881 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.882 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.882 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2023-10-26T06:57:27.882 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.883 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.883 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.883 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2023-10-26T06:57:27.884 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2023-10-26T06:57:27.884 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.884 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.885 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.885 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.885 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.885 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.886 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.886 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-26T06:57:27.886 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.887 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.887 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.887 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2023-10-26T06:57:27.888 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2023-10-26T06:57:27.888 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.888 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.889 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.889 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:27.889 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.889 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2023-10-26T06:57:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.890 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.891 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2023-10-26T06:57:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2023-10-26T06:57:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.892 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.893 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2023-10-26T06:57:27.894 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.895 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2023-10-26T06:57:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2023-10-26T06:57:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.896 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.897 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-26T06:57:27.898 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.899 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.900 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.901 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.902 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.903 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.904 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.905 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2023-10-26T06:57:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2023-10-26T06:57:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.906 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.907 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-26T06:57:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.908 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.909 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.910 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.911 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-26T06:57:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.912 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.913 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.914 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.915 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:27.916 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:27.917 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.918 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.919 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.920 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.921 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.922 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.923 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.924 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.925 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-26T06:57:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-26T06:57:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.926 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.927 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.928 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-26T06:57:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-26T06:57:27.930 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-26T06:57:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.931 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.932 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2023-10-26T06:57:27.932 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.932 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.932 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.933 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-26T06:57:27.933 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.933 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.934 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.934 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2023-10-26T06:57:27.935 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2023-10-26T06:57:27.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.935 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.935 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.936 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:27.936 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.936 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.936 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.937 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2023-10-26T06:57:27.937 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.937 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.938 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.938 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2023-10-26T06:57:27.938 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2023-10-26T06:57:27.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.939 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.939 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.940 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:27.940 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.940 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.940 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.941 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-26T06:57:27.941 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.941 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.942 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.942 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2023-10-26T06:57:27.942 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2023-10-26T06:57:27.943 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.943 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.943 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.944 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.944 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.944 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.945 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.945 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2023-10-26T06:57:27.945 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.945 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.946 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.946 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2023-10-26T06:57:27.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2023-10-26T06:57:27.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.947 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.947 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.948 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2023-10-26T06:57:27.948 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.948 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.949 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:27.949 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2023-10-26T06:57:27.949 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2023-10-26T06:57:27.949 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:27.950 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2023-10-26T06:57:27.950 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.950 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.951 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.951 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2023-10-26T06:57:27.952 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2023-10-26T06:57:27.952 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.952 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.952 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.953 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-26T06:57:27.953 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.953 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.954 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.954 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-26T06:57:27.954 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.958 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.958 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.958 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2023-10-26T06:57:27.959 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2023-10-26T06:57:27.959 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.959 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.960 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.960 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.960 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.960 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.961 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.961 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-26T06:57:27.961 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.962 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.962 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.962 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2023-10-26T06:57:27.963 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2023-10-26T06:57:27.963 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.963 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.964 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.964 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.964 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.964 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.965 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.965 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-26T06:57:27.965 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.966 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.966 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.967 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2023-10-26T06:57:27.967 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2023-10-26T06:57:27.967 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.967 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.968 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.968 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.968 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.968 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.969 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.969 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-26T06:57:27.969 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.970 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.970 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.970 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2023-10-26T06:57:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2023-10-26T06:57:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-26T06:57:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2023-10-26T06:57:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2023-10-26T06:57:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-26T06:57:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2023-10-26T06:57:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2023-10-26T06:57:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2023-10-26T06:57:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2023-10-26T06:57:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2023-10-26T06:57:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2023-10-26T06:57:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-26T06:57:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2023-10-26T06:57:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2023-10-26T06:57:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-26T06:57:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2023-10-26T06:57:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2023-10-26T06:57:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-26T06:57:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2023-10-26T06:57:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2023-10-26T06:57:27.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.995 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.995 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.996 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:27.996 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:27.996 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:27.996 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:27.997 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2023-10-26T06:57:27.997 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:27.997 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:27.997 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:27.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:27.998 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2023-10-26T06:57:27.998 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2023-10-26T06:57:27.999 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:27.999 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:27.999 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:27.999 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.000 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.000 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.000 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.001 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-26T06:57:28.001 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.001 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.001 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.002 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2023-10-26T06:57:28.002 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2023-10-26T06:57:28.003 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.003 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.003 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.003 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.004 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.004 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.004 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.005 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-26T06:57:28.005 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.005 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.005 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.006 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.006 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2023-10-26T06:57:28.007 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.007 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.008 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.008 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.008 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.009 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.009 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.009 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.010 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.010 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.010 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.011 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.011 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.011 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.012 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.012 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.012 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.013 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.013 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.013 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.013 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.014 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.014 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.015 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.015 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.015 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.016 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.016 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.016 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.017 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.017 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.017 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.017 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.018 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.018 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.019 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.019 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.020 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.020 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.020 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.020 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.021 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.021 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.021 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.022 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.022 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.022 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.022 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.023 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.023 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.023 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.024 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.024 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.024 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.025 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.025 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.025 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.026 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.026 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.026 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.026 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.027 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.027 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.027 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.028 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.028 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.028 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2023-10-26T06:57:28.029 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2023-10-26T06:57:28.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.029 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.030 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.030 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2023-10-26T06:57:28.030 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.030 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.031 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.031 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-26T06:57:28.031 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.032 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.032 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.032 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2023-10-26T06:57:28.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2023-10-26T06:57:28.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.033 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.034 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.034 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2023-10-26T06:57:28.034 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.035 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.035 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.035 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-26T06:57:28.035 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.036 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.036 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.036 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.036 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2023-10-26T06:57:28.038 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.038 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.038 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.038 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-26T06:57:28.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-26T06:57:28.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.039 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.040 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.040 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:28.040 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.040 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.041 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.041 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.041 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.041 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.042 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.042 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2023-10-26T06:57:28.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-26T06:57:28.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.043 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.044 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.044 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.044 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.044 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.045 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.045 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.045 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.046 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.046 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.046 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-26T06:57:28.047 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-26T06:57:28.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.047 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.048 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.048 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.048 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.048 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.049 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.049 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.049 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.050 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.050 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.050 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2023-10-26T06:57:28.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2023-10-26T06:57:28.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.051 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.052 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.052 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.052 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.052 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.053 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.053 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.053 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.054 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.054 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.054 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-26T06:57:28.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-26T06:57:28.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.055 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.056 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.056 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.056 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.056 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.057 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.057 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.057 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.058 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.058 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.058 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2023-10-26T06:57:28.059 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-26T06:57:28.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.059 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.060 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.060 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.060 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.060 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.061 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.061 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.061 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.062 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.062 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.062 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-26T06:57:28.063 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-26T06:57:28.063 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.063 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.063 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.064 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.064 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.064 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.065 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.065 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.065 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.065 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.066 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.066 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-26T06:57:28.067 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-26T06:57:28.067 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.067 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.067 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.068 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.068 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.068 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.069 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.069 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.069 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.069 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.070 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.070 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.070 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-26T06:57:28.070 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-26T06:57:28.071 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.071 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.071 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.072 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.072 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.072 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.072 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.073 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.073 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.073 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.073 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.074 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-26T06:57:28.074 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-26T06:57:28.075 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.075 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.075 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.075 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.076 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.076 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.076 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.077 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.077 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.077 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.077 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.078 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-26T06:57:28.078 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-26T06:57:28.079 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.079 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.079 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.079 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.080 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.080 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.080 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.081 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.081 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.081 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.081 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.082 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-26T06:57:28.082 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-26T06:57:28.083 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.083 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.083 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.084 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.084 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.084 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.084 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.085 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.085 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.085 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.086 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.086 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-26T06:57:28.086 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-26T06:57:28.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.087 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.087 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.088 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.088 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.088 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.088 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.089 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.089 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.089 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.089 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.090 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-26T06:57:28.090 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-26T06:57:28.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.091 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.091 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.091 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.092 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.092 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.092 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.093 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.093 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.093 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.093 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.094 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2023-10-26T06:57:28.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-26T06:57:28.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.095 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.095 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.095 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:28.096 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.096 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.096 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.097 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.097 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.097 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.097 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.098 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-26T06:57:28.098 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-26T06:57:28.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.099 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.099 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.099 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.100 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.100 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.100 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.100 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.101 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.101 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.101 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.102 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2023-10-26T06:57:28.102 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2023-10-26T06:57:28.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.103 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.103 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.103 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.104 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.104 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.104 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.104 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.105 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.105 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.105 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.106 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2023-10-26T06:57:28.106 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-26T06:57:28.106 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.107 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.107 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.107 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:28.108 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.108 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.108 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.108 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.109 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.109 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.109 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.110 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-26T06:57:28.110 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-26T06:57:28.110 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.111 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.111 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.111 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.112 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.112 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.112 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.112 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.113 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.113 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.113 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.114 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-26T06:57:28.114 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-26T06:57:28.114 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.115 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.115 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.115 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.116 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.116 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.116 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.116 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.117 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.117 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.117 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.118 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-26T06:57:28.118 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-26T06:57:28.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.119 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.119 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.119 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.120 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.120 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.120 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.120 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.121 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.121 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.121 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.122 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-26T06:57:28.122 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-26T06:57:28.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.123 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.123 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.123 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.123 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.124 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.124 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.124 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.125 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.125 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.125 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.126 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2023-10-26T06:57:28.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-26T06:57:28.126 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.127 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.127 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.127 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.127 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.128 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.128 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.128 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.129 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.129 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.129 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.130 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-26T06:57:28.130 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-26T06:57:28.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.131 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.131 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.131 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.132 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.132 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.132 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.132 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.133 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.133 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.133 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.134 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-26T06:57:28.134 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-26T06:57:28.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.135 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.135 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.135 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.136 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.136 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.136 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.136 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.137 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.137 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.137 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.138 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-26T06:57:28.138 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-26T06:57:28.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.139 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.139 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.139 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.140 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.140 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.140 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.140 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.141 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.141 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.141 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.142 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-26T06:57:28.142 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-26T06:57:28.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.143 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.143 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.143 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.143 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.144 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.144 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.144 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.144 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.145 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.145 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.145 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-26T06:57:28.146 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-26T06:57:28.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.146 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.147 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.147 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.147 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.147 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.148 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.148 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.148 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.148 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.149 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.149 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-26T06:57:28.150 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-26T06:57:28.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.150 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.150 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-26T06:57:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-26T06:57:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-26T06:57:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-26T06:57:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2023-10-26T06:57:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-26T06:57:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-26T06:57:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-26T06:57:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-26T06:57:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-26T06:57:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-26T06:57:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-26T06:57:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-26T06:57:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.174 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.174 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.174 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2023-10-26T06:57:28.174 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.175 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.175 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.175 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.176 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.176 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.176 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.177 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-26T06:57:28.177 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-26T06:57:28.177 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.178 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.178 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.178 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2023-10-26T06:57:28.178 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.179 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.179 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.179 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.180 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.180 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.180 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.181 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2023-10-26T06:57:28.181 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-26T06:57:28.181 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.182 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.182 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.182 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.183 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.183 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.183 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.183 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.184 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.184 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.185 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-26T06:57:28.185 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-26T06:57:28.185 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.186 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.186 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.186 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2023-10-26T06:57:28.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.187 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.187 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.187 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.188 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.188 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.189 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2023-10-26T06:57:28.189 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-26T06:57:28.189 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.190 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.190 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.190 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.190 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.191 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.191 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.191 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.192 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.192 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.192 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.193 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2023-10-26T06:57:28.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-26T06:57:28.193 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.194 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.194 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.194 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.194 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.195 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.195 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.195 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.196 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.196 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.196 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.197 INFO:teuthology.orchestra.run.smithi089.stdout: "UNSAFE_TLS_v1_2": { 2023-10-26T06:57:28.197 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-26T06:57:28.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.197 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.198 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.198 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.198 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.199 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.199 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.199 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.200 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.200 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.200 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.201 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-26T06:57:28.201 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-26T06:57:28.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.201 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.202 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.202 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:28.202 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.203 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.203 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.203 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.203 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.204 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.205 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-26T06:57:28.205 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-26T06:57:28.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.205 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.206 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.206 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:28.206 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.206 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.207 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.207 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.207 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.207 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.208 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.208 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-26T06:57:28.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-26T06:57:28.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.209 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.210 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:28.210 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.210 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.210 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.211 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.211 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.211 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.212 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.212 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2023-10-26T06:57:28.212 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2023-10-26T06:57:28.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.213 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.213 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.213 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.214 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.214 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.214 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.215 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.215 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.215 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.216 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.216 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2023-10-26T06:57:28.216 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2023-10-26T06:57:28.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.217 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.217 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.218 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.218 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.218 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.219 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.219 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.219 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.219 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.220 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.220 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2023-10-26T06:57:28.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2023-10-26T06:57:28.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.221 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.221 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.222 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.222 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.222 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.223 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.223 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2023-10-26T06:57:28.223 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.223 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.224 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.224 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2023-10-26T06:57:28.225 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2023-10-26T06:57:28.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.225 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.225 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.226 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2023-10-26T06:57:28.226 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.226 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.227 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.227 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.227 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.227 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.228 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.228 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2023-10-26T06:57:28.229 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2023-10-26T06:57:28.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.229 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.229 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.230 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.230 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.230 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.231 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.231 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.231 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.231 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.232 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.232 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.233 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.234 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.234 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.234 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.234 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.235 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.235 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.235 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.236 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.236 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.236 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.236 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.237 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.237 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.237 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.238 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.238 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.238 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.239 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.240 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.240 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.240 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.240 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.241 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.241 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.241 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.242 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.242 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.242 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.243 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.244 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.244 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.244 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.244 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.245 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.245 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.245 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.245 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.246 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.246 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.246 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.246 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.247 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.247 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.247 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.248 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.248 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.249 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.249 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.249 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.249 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.250 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.250 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.250 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.251 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.251 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.251 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.251 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.252 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.252 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2023-10-26T06:57:28.253 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2023-10-26T06:57:28.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.253 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.254 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.254 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.254 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.254 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.255 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.255 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2023-10-26T06:57:28.255 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.255 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.256 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.256 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2023-10-26T06:57:28.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2023-10-26T06:57:28.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.257 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.257 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.258 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.258 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.258 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.258 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.259 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.259 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.259 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.260 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.260 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:28.261 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:28.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.261 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.261 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.262 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2023-10-26T06:57:28.262 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.262 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.263 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.263 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.263 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.263 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.264 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.264 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:28.265 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:28.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.265 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.265 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.266 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2023-10-26T06:57:28.266 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.266 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.267 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.267 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.267 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.267 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.268 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.268 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2023-10-26T06:57:28.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2023-10-26T06:57:28.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.269 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.269 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.269 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.270 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.270 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.270 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.271 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.271 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.271 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.272 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.272 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2023-10-26T06:57:28.272 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2023-10-26T06:57:28.273 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.273 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.273 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.273 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2023-10-26T06:57:28.274 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.274 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.274 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.275 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.275 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.275 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.275 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.276 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2023-10-26T06:57:28.276 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2023-10-26T06:57:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.277 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2023-10-26T06:57:28.278 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.278 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.278 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.278 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.279 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2023-10-26T06:57:28.279 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.279 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.280 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.280 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.280 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.281 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2023-10-26T06:57:28.281 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2023-10-26T06:57:28.281 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.281 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.282 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.282 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2023-10-26T06:57:28.282 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2023-10-26T06:57:28.283 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2023-10-26T06:57:28.283 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.283 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.283 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.284 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.284 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.285 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2023-10-26T06:57:28.285 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2023-10-26T06:57:28.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.285 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.286 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.286 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.286 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.287 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.287 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.287 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.287 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.288 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.288 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.288 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.289 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.289 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2023-10-26T06:57:28.290 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.290 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.290 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.291 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2023-10-26T06:57:28.291 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2023-10-26T06:57:28.291 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.291 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.292 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.292 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.292 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.293 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.293 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.293 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2023-10-26T06:57:28.293 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.294 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.294 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.295 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.295 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.295 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.296 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.296 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.296 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.297 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.297 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.297 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.297 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.298 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.298 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.298 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.299 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.299 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.299 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.299 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.300 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.300 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.300 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.301 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.301 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.302 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.302 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.302 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.302 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.303 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.303 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.303 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.304 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.304 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.304 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.305 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.305 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.306 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.306 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.306 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.307 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.307 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.307 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.307 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.308 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.308 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.308 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.308 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.309 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.309 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.309 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.310 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.310 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.311 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.311 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.312 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.312 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.312 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.312 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.313 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.313 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.313 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.313 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.314 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2023-10-26T06:57:28.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2023-10-26T06:57:28.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.315 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.315 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.315 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2023-10-26T06:57:28.316 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.316 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.316 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.316 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-26T06:57:28.317 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.317 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.317 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.318 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2023-10-26T06:57:28.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2023-10-26T06:57:28.318 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.319 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.319 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.319 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2023-10-26T06:57:28.319 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.320 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.320 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.320 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-26T06:57:28.320 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.321 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.321 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.322 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2023-10-26T06:57:28.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2023-10-26T06:57:28.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.322 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.323 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.323 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2023-10-26T06:57:28.323 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.324 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.324 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.324 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2023-10-26T06:57:28.324 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.325 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.325 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.325 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2023-10-26T06:57:28.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2023-10-26T06:57:28.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.326 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.327 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.327 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2023-10-26T06:57:28.327 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.327 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-26T06:57:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2023-10-26T06:57:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2023-10-26T06:57:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-26T06:57:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2023-10-26T06:57:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2023-10-26T06:57:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2023-10-26T06:57:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2023-10-26T06:57:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-26T06:57:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2023-10-26T06:57:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.351 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.351 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.351 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.351 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.352 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.352 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.352 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.352 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.353 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.353 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.353 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.354 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.354 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.354 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.354 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.355 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.355 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.355 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.356 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.356 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.356 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.356 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.357 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.357 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.357 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.357 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.358 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.358 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.358 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.359 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.359 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.359 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.359 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.360 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.360 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.360 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.360 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.362 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.362 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2023-10-26T06:57:28.363 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2023-10-26T06:57:28.363 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.363 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.363 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.364 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2023-10-26T06:57:28.364 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.364 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.364 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.365 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.365 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.365 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.366 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.366 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.366 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2023-10-26T06:57:28.366 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2023-10-26T06:57:28.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.367 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.367 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.367 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2023-10-26T06:57:28.368 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.368 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.368 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.369 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.369 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.369 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.369 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.370 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:28.370 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:28.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.371 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.371 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.371 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2023-10-26T06:57:28.372 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.372 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.372 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.373 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.373 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.373 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.373 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.375 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2023-10-26T06:57:28.375 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2023-10-26T06:57:28.375 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2023-10-26T06:57:28.376 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.376 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2023-10-26T06:57:28.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2023-10-26T06:57:28.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.377 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.377 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.377 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2023-10-26T06:57:28.377 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.377 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-26T06:57:28.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.379 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2023-10-26T06:57:28.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2023-10-26T06:57:28.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.380 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.380 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.381 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2023-10-26T06:57:28.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.382 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-26T06:57:28.382 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.382 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.383 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.383 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2023-10-26T06:57:28.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2023-10-26T06:57:28.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.384 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.384 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.385 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.385 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.385 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.385 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.386 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2023-10-26T06:57:28.386 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.386 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.387 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.387 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:28.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:28.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.388 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.389 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2023-10-26T06:57:28.389 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2023-10-26T06:57:28.389 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.389 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.390 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-26T06:57:28.390 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.391 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.391 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.393 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.393 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.393 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.393 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.394 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.394 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.394 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.395 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.395 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.395 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.395 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.396 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.396 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.396 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.397 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.397 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.398 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.398 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.399 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.399 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.399 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.399 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.400 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.400 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.400 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.401 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.401 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.402 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.402 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.403 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.403 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.403 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.404 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.404 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.404 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.404 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.405 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.405 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.405 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.406 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.406 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.406 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.406 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.407 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.407 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.408 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.408 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.408 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.409 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.409 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.409 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.409 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.410 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.410 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.411 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2023-10-26T06:57:28.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2023-10-26T06:57:28.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.412 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.413 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.413 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.413 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.413 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.414 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2023-10-26T06:57:28.414 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.415 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.415 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2023-10-26T06:57:28.416 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2023-10-26T06:57:28.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.416 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.417 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2023-10-26T06:57:28.417 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.418 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2023-10-26T06:57:28.418 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.419 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.419 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2023-10-26T06:57:28.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2023-10-26T06:57:28.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.420 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.421 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2023-10-26T06:57:28.421 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.421 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.421 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-26T06:57:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.422 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2023-10-26T06:57:28.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2023-10-26T06:57:28.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.425 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:28.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:28.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2023-10-26T06:57:28.425 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.426 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-26T06:57:28.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.427 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2023-10-26T06:57:28.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2023-10-26T06:57:28.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.429 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.429 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.429 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.429 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.430 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2023-10-26T06:57:28.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.430 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.431 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2023-10-26T06:57:28.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2023-10-26T06:57:28.432 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.432 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.432 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.432 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2023-10-26T06:57:28.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.433 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.434 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-26T06:57:28.434 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.434 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.435 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.435 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2023-10-26T06:57:28.436 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.436 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.437 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.437 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.438 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.438 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.439 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.439 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.439 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.440 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.440 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.440 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.441 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.441 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.441 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.441 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.442 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.442 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.443 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.444 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.444 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.445 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.445 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.445 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.445 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.446 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.446 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.446 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.447 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.448 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.448 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.448 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.449 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.449 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.449 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.449 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.450 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.450 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.450 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.451 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.451 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.451 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.451 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.453 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.454 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.454 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.454 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.454 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.455 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.455 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.455 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.455 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.456 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2023-10-26T06:57:28.458 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.458 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.458 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.458 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.459 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.459 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.459 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.460 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.460 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.460 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.461 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.461 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.461 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.461 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.462 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.462 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.462 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.463 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.463 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.463 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.464 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.464 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.465 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.465 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.466 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.466 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.466 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.466 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.467 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.467 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.467 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.468 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.468 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.468 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.469 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.469 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.469 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.470 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.470 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.470 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.470 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.471 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.471 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.471 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.472 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.472 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.472 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.473 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.473 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.473 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.473 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.474 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.474 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.475 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.475 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.476 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.476 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.477 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.477 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.477 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.477 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.479 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2023-10-26T06:57:28.479 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.479 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.480 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.480 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2023-10-26T06:57:28.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2023-10-26T06:57:28.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.481 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.481 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.481 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2023-10-26T06:57:28.482 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.482 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.482 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.483 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-26T06:57:28.483 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.483 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.484 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.484 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2023-10-26T06:57:28.484 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2023-10-26T06:57:28.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.485 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.485 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.486 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:28.486 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2023-10-26T06:57:28.486 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.486 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.487 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-26T06:57:28.487 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.487 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.488 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.488 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.488 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.489 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.490 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.490 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.490 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.491 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.491 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.491 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.492 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.492 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.492 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.492 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.493 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.493 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.493 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.494 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.494 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.494 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.496 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.496 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.496 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.497 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.497 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.497 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.497 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.498 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.499 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.499 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.499 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.500 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.500 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.500 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.501 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.501 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.501 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.501 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.502 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.502 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.502 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.503 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.503 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.503 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.503 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.504 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.504 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.505 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.505 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.505 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.506 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.506 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.506 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2023-10-26T06:57:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2023-10-26T06:57:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2023-10-26T06:57:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2023-10-26T06:57:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2023-10-26T06:57:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-26T06:57:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.531 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.531 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.531 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.532 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.532 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.532 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.532 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.533 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.533 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2023-10-26T06:57:28.534 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2023-10-26T06:57:28.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.534 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.534 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.535 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.535 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.535 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.535 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.536 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2023-10-26T06:57:28.536 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-26T06:57:28.536 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.537 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.537 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2023-10-26T06:57:28.538 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2023-10-26T06:57:28.538 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.538 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.538 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.539 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2023-10-26T06:57:28.539 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.539 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.540 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.540 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2023-10-26T06:57:28.540 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.540 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.541 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.541 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2023-10-26T06:57:28.541 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2023-10-26T06:57:28.542 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.542 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.542 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.543 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2023-10-26T06:57:28.543 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.543 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.543 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.544 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2023-10-26T06:57:28.544 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.544 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.545 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.545 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2023-10-26T06:57:28.545 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2023-10-26T06:57:28.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.546 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.546 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.546 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.547 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.547 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.547 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.547 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2023-10-26T06:57:28.548 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.548 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.549 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2023-10-26T06:57:28.549 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2023-10-26T06:57:28.549 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.550 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.550 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.550 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2023-10-26T06:57:28.550 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.551 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.551 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.551 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-26T06:57:28.552 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-26T06:57:28.552 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.552 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.553 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.553 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2023-10-26T06:57:28.554 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.554 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.554 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.555 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.555 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.555 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.556 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.556 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.556 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.556 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.557 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.557 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.557 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.557 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.558 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.558 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.558 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.559 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.559 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.559 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.559 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.560 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.560 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.560 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.561 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.561 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.561 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.562 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.562 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.562 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.563 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.563 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.563 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.563 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.564 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.565 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.565 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.565 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.566 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.566 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.566 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.567 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.567 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.567 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.567 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.568 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.569 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.569 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.570 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.570 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.570 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.571 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.571 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.572 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.572 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.573 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.573 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.573 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.573 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.574 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.574 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.574 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.575 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2023-10-26T06:57:28.576 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.576 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.577 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.577 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.577 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.578 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.578 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.578 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.579 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.579 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.579 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.579 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.580 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.580 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.580 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.581 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.581 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.581 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.581 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.582 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.582 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.582 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.583 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.584 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.584 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.585 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.585 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.585 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.585 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.586 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.587 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.587 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.587 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.588 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.588 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.589 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.589 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.589 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.589 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.590 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.590 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.590 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.591 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.591 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.591 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.591 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.592 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.592 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.593 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.593 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.594 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.594 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.594 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.595 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.595 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.595 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.595 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.596 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.596 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2023-10-26T06:57:28.597 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.598 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.598 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.598 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.599 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.599 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.600 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.600 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.600 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.600 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.601 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.601 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.601 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.601 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.602 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.602 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.602 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.603 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.603 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.603 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.603 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.604 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.604 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.605 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.605 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.605 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.605 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.606 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.606 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.606 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.607 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.607 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.607 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.608 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.608 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.609 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.609 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.609 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.609 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.610 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.610 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.610 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.611 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.611 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.611 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.611 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.612 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.612 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.612 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.613 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.613 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.613 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.614 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.614 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.614 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.614 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.615 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.615 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.615 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.615 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.616 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.616 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.616 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.616 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.617 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.617 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.618 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2023-10-26T06:57:28.618 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.618 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.619 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.619 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2023-10-26T06:57:28.619 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2023-10-26T06:57:28.619 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.620 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.620 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.620 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.621 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.621 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.621 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.621 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-26T06:57:28.622 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.622 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.622 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.623 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.623 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.623 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.624 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.624 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.625 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.625 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.625 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.625 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.626 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.626 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.626 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.626 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.627 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.627 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.627 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.628 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.628 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.628 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.629 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.629 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.629 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.630 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.630 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.631 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.631 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.631 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.631 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.632 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.632 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.633 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.633 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.634 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.634 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.634 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.635 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.635 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.635 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.635 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.636 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.636 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.636 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.637 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.637 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.637 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.637 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.638 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.638 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.638 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.639 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.639 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.639 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.640 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.640 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.640 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.640 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.641 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.641 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.641 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.642 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.642 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2023-10-26T06:57:28.643 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2023-10-26T06:57:28.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.643 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.643 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.644 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.644 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.644 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.644 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.645 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2023-10-26T06:57:28.645 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2023-10-26T06:57:28.645 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2023-10-26T06:57:28.646 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.646 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2023-10-26T06:57:28.646 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.646 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.647 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2023-10-26T06:57:28.648 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.649 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.649 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.649 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.650 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.650 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.651 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.651 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.651 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.652 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.652 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.652 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.652 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.653 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.653 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.653 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.654 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.654 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.654 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.654 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.655 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.655 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.656 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.657 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.657 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.657 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.658 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.658 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.658 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.658 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.659 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.659 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.660 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.660 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.660 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.661 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.661 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.661 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.662 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.662 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.662 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.662 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.663 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.663 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.663 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.664 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.664 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.664 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.664 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.665 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.665 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.666 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.666 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.667 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.667 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.667 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.667 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.668 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.668 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.668 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.669 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.669 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.670 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2023-10-26T06:57:28.670 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.670 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.671 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.671 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.672 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.673 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.674 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.674 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.674 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.674 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.675 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.675 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.675 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.676 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.676 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.676 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.676 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.677 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.678 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.678 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.678 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.678 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.679 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.679 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.679 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.680 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.680 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.680 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.680 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.681 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.681 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.682 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.682 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.682 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.683 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.683 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.683 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.683 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.684 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.684 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.684 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.685 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.685 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.685 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.685 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.686 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.686 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.686 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.687 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.688 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.688 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.688 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.688 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.689 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.689 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.689 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.690 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.690 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.692 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2023-10-26T06:57:28.692 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.692 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.692 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.693 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.693 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.693 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.694 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.694 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.694 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.695 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.695 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.695 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.695 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.696 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.696 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.696 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.697 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.697 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.697 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.698 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.698 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.698 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.699 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.699 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.699 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.699 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.700 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.700 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.700 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.701 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.701 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.701 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.702 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.702 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.703 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.703 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.704 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.704 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.704 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.704 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.705 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.705 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.705 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.706 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.706 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.706 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.706 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.707 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.707 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.707 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.708 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.708 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.708 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.708 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.709 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.709 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.710 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.710 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.711 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.711 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.711 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.712 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.712 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.712 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.712 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.713 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:28.713 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:28.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.714 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.714 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:28.715 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.715 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.715 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.716 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.716 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.716 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.716 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.717 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2023-10-26T06:57:28.717 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2023-10-26T06:57:28.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:28.718 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.718 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.718 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2023-10-26T06:57:28.719 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2023-10-26T06:57:28.719 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.719 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.720 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2023-10-26T06:57:28.720 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-26T06:57:28.720 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.720 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.722 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2023-10-26T06:57:28.722 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.722 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.723 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.723 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2023-10-26T06:57:28.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2023-10-26T06:57:28.724 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.724 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.724 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.724 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.725 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.725 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.726 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-26T06:57:28.726 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.726 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.726 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.727 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2023-10-26T06:57:28.727 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2023-10-26T06:57:28.728 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.728 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.728 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.728 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.729 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.729 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.729 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.730 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.730 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.730 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.730 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.731 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.731 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.732 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.732 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.732 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.732 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.733 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.733 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.733 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.734 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.734 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.734 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.734 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.735 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.735 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.735 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.736 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.736 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.736 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.736 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.737 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.738 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.738 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.738 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.739 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.739 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.739 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.739 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.739 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.740 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.740 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.741 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.741 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.741 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.741 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.749 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.749 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.749 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2023-10-26T06:57:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2023-10-26T06:57:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2023-10-26T06:57:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2023-10-26T06:57:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2023-10-26T06:57:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2023-10-26T06:57:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2023-10-26T06:57:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2023-10-26T06:57:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2023-10-26T06:57:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2023-10-26T06:57:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2023-10-26T06:57:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2023-10-26T06:57:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2023-10-26T06:57:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-26T06:57:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-26T06:57:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.768 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.768 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.769 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.769 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.769 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.769 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.770 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.770 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.770 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.770 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.771 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.771 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.771 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.772 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.772 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.772 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.772 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.773 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.773 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.773 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.774 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.774 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.775 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.775 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.775 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.776 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.776 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.776 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.776 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.777 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.777 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.777 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.778 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.778 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.778 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.779 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.779 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.780 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.780 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.780 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.780 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.781 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.781 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.781 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.782 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.782 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.782 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.782 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.783 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.783 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.783 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.784 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.784 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.784 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.785 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.785 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.785 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.786 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.786 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.786 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.787 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.787 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.787 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.787 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.788 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2023-10-26T06:57:28.788 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2023-10-26T06:57:28.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.789 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.789 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.789 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.790 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.790 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.790 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.791 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.791 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.791 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.791 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.792 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-26T06:57:28.792 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-26T06:57:28.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.793 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.793 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.793 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2023-10-26T06:57:28.794 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.794 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.794 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.794 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.795 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.795 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.795 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.796 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2023-10-26T06:57:28.796 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2023-10-26T06:57:28.796 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:28.797 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.797 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.797 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2023-10-26T06:57:28.798 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.798 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.798 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.798 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.799 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.799 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.799 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.800 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:28.800 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:28.800 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.801 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.801 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.801 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2023-10-26T06:57:28.801 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.802 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.802 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.802 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-26T06:57:28.803 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.803 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.803 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.804 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:28.804 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:28.804 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.805 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.805 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.805 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2023-10-26T06:57:28.805 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.806 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.806 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.806 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-26T06:57:28.807 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.807 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.807 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.808 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2023-10-26T06:57:28.808 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2023-10-26T06:57:28.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.808 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.809 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.809 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2023-10-26T06:57:28.809 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.810 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.810 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.810 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.811 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.811 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.811 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.812 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2023-10-26T06:57:28.812 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2023-10-26T06:57:28.812 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.812 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.813 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.813 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2023-10-26T06:57:28.813 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.814 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.814 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.814 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2023-10-26T06:57:28.814 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2023-10-26T06:57:28.815 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.815 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.815 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.815 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.816 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.816 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2023-10-26T06:57:28.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2023-10-26T06:57:28.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.817 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.817 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.818 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2023-10-26T06:57:28.818 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2023-10-26T06:57:28.818 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2023-10-26T06:57:28.819 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.819 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.819 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.819 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.820 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.820 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.820 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.821 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2023-10-26T06:57:28.821 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.822 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.822 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.822 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.823 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.823 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.823 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.824 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.824 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.824 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.825 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.825 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.825 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.825 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.826 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.826 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.826 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.827 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.827 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.827 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.827 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.828 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.828 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.829 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.829 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.830 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.830 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.830 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.831 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.833 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.833 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.834 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.834 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.834 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.834 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.835 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.835 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.835 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.835 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.836 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.836 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.836 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.837 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.837 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.837 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.837 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.838 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.839 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.839 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.839 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.840 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.840 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.840 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.840 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.841 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.841 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.841 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.842 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2023-10-26T06:57:28.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2023-10-26T06:57:28.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.843 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.843 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.843 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2023-10-26T06:57:28.843 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.844 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.844 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.844 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.844 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.845 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.845 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.846 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2023-10-26T06:57:28.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2023-10-26T06:57:28.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.846 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.847 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.847 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.847 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.848 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.848 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.848 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.848 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.849 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.849 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.850 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2023-10-26T06:57:28.850 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2023-10-26T06:57:28.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.850 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.851 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.851 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.851 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.852 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.852 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.852 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.852 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.853 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.853 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.853 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.854 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2023-10-26T06:57:28.855 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.855 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.855 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.855 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2023-10-26T06:57:28.856 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2023-10-26T06:57:28.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.856 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.857 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.857 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.857 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.857 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.858 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.858 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.858 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.858 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.859 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.859 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2023-10-26T06:57:28.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2023-10-26T06:57:28.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.860 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.860 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.861 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.861 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.861 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.862 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.862 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.862 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.862 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.863 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.863 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.864 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.864 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.864 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.865 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.865 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.865 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.865 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.866 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.866 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.866 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.867 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.867 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.867 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.867 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.868 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.868 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.868 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.869 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.870 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.870 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.870 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.871 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.871 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.871 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.872 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.872 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.872 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.872 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.873 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.873 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.874 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.874 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.874 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.875 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.875 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.875 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.876 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.876 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.876 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.876 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.877 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.877 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.877 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.878 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.878 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.878 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.878 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.879 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.879 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.879 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.880 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.880 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.880 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.880 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.881 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.881 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.881 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.882 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.882 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.882 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.883 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2023-10-26T06:57:28.883 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2023-10-26T06:57:28.883 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.884 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.884 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.884 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.885 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.885 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.885 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.885 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.886 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.886 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.886 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2023-10-26T06:57:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2023-10-26T06:57:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2023-10-26T06:57:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.911 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.911 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.911 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.911 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.913 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2023-10-26T06:57:28.913 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.913 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.914 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.914 INFO:teuthology.orchestra.run.smithi089.stdout: "drive_group_interval": { 2023-10-26T06:57:28.914 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "drive_group_interval", 2023-10-26T06:57:28.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:28.915 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.915 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300.0", 2023-10-26T06:57:28.916 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.916 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.916 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.916 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-26T06:57:28.917 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.917 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.917 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.918 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.918 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.918 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.919 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.919 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.919 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.920 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.920 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.920 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.920 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.921 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.921 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.921 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.922 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.922 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.922 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.922 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.923 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.923 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.923 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.924 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.924 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.925 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.925 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.925 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.926 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.926 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.926 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.926 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.927 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.927 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.927 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.928 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.928 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.928 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.929 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.929 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.929 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.929 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.930 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.930 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.930 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.931 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.931 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.931 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.931 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.932 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.932 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.932 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.933 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.933 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.933 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.934 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.934 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.934 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.935 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.935 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.935 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.935 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.936 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.936 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.936 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.936 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.937 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.937 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.938 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2023-10-26T06:57:28.938 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2023-10-26T06:57:28.938 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.938 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.939 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.939 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2023-10-26T06:57:28.939 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.940 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.940 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.940 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-26T06:57:28.940 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.941 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.941 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.941 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.942 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:28.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:28.942 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2023-10-26T06:57:28.943 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:28.943 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:28.943 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:28.944 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:28.944 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:28.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.944 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.945 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.945 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.945 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.945 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.946 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.946 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.946 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.947 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.947 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.947 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.948 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.948 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.948 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.948 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.949 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.949 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.950 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:28.950 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:28.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.950 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.951 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.951 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.951 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.952 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.952 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.952 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.952 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.953 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.953 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.954 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:28.954 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:28.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.954 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.955 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.955 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:28.955 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.956 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.956 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:28.956 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:28.956 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:28.957 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:28.957 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:28.957 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:28.957 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:28.958 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:28.958 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.958 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.959 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.959 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.959 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:28.960 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:28.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.960 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.961 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:28.961 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:28.961 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.962 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.962 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.962 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.962 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.963 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.963 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.964 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2023-10-26T06:57:28.964 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2023-10-26T06:57:28.964 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.964 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.965 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.965 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.965 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.965 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.966 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.966 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.966 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.967 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.967 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.967 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2023-10-26T06:57:28.968 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2023-10-26T06:57:28.968 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.968 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.968 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.969 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2023-10-26T06:57:28.969 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.969 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.970 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.970 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.970 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.970 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.971 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.971 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2023-10-26T06:57:28.971 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2023-10-26T06:57:28.973 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.973 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.974 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.974 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.974 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.975 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.975 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.975 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.975 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.976 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.976 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.976 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2023-10-26T06:57:28.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2023-10-26T06:57:28.977 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.977 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.977 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.978 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.978 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.978 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.979 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.979 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.979 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.979 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.980 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.980 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2023-10-26T06:57:28.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2023-10-26T06:57:28.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2023-10-26T06:57:28.981 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.981 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.982 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.982 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.982 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.983 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.983 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.983 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.983 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.984 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.984 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2023-10-26T06:57:28.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2023-10-26T06:57:28.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:28.985 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.985 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.986 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.986 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.986 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.987 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.987 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.987 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.987 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.988 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.988 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2023-10-26T06:57:28.988 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2023-10-26T06:57:28.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.989 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.989 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.990 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.990 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.990 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.990 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.991 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.991 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.991 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.992 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.992 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2023-10-26T06:57:28.992 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2023-10-26T06:57:28.993 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:28.993 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.993 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.993 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:28.994 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:28.994 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:28.994 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.995 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.995 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.995 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.995 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:28.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:28.996 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2023-10-26T06:57:28.996 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2023-10-26T06:57:28.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:28.997 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:28.997 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:28.997 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:28.998 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2023-10-26T06:57:28.998 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2023-10-26T06:57:28.998 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:28.999 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:28.999 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:28.999 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:28.999 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.000 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2023-10-26T06:57:29.000 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2023-10-26T06:57:29.001 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.001 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.001 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.001 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.002 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.002 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.002 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.003 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.003 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.003 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.003 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.004 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2023-10-26T06:57:29.004 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2023-10-26T06:57:29.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.005 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.005 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.005 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.006 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.006 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.006 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.007 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.007 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.007 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.007 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.008 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2023-10-26T06:57:29.008 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2023-10-26T06:57:29.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.009 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.009 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.009 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.010 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.010 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.010 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.011 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.011 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.011 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.011 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.012 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.012 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.013 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2023-10-26T06:57:29.013 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.013 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.014 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.014 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2023-10-26T06:57:29.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2023-10-26T06:57:29.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.015 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.015 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.015 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.016 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.016 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.016 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.016 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2023-10-26T06:57:29.017 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.017 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.017 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.018 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2023-10-26T06:57:29.018 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2023-10-26T06:57:29.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.019 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.019 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.019 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.020 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.020 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.020 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.020 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2023-10-26T06:57:29.021 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.021 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.021 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.022 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.022 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.023 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.023 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.023 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.023 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.024 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.024 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.024 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.025 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.025 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.025 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.025 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.026 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.026 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.026 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.027 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.027 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.027 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.028 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.028 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.029 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.029 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.029 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.029 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.030 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.030 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.030 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.031 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.031 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.031 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.032 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.032 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.033 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.033 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.033 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.033 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.034 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.034 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.034 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.035 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.035 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.035 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.035 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.036 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.036 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.036 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.037 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.037 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.037 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.038 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.038 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.039 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.039 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.039 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.039 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.040 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.040 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.040 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.040 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.041 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.041 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.042 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2023-10-26T06:57:29.042 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.043 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.044 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.045 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.045 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.045 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.046 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.046 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.046 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.046 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.047 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.047 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.047 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.048 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.048 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.048 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.048 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.049 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.049 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.049 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.050 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.050 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.050 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.051 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.051 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.051 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.051 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.052 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.052 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.052 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.052 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.053 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.054 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.054 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.054 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.055 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.055 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.055 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.056 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.056 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.056 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.056 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.057 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.057 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.057 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.057 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.058 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.058 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.058 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.059 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.059 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.060 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.060 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.060 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.060 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.061 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.061 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.061 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.062 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.062 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.062 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.062 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.063 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.064 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2023-10-26T06:57:29.064 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.064 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.064 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.065 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.065 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.065 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.065 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.066 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.066 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.066 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.067 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.067 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.067 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.067 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.068 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.068 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.068 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.068 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.069 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.069 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.069 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.070 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.070 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.070 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.071 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.071 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.071 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.071 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.072 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.072 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.072 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.075 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.075 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.075 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.075 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.076 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.076 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.076 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.076 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.077 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.077 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.077 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.078 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.078 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.078 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.078 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.079 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.079 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.079 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.079 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.080 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.080 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.080 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2023-10-26T06:57:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2023-10-26T06:57:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2023-10-26T06:57:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-26T06:57:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2023-10-26T06:57:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.105 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.105 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.105 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.106 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.106 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.106 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.106 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.107 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.107 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.107 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.107 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.108 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.108 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.108 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.109 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.109 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.109 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.110 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.110 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.110 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.111 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.111 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.111 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.111 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.112 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.112 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.112 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.113 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.113 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.113 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.113 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.114 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2023-10-26T06:57:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.115 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2023-10-26T06:57:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2023-10-26T06:57:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.116 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.117 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.117 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:29.117 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.117 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.118 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.118 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-26T06:57:29.118 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.119 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.119 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.119 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2023-10-26T06:57:29.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2023-10-26T06:57:29.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.120 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.120 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.121 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:29.121 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.121 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.122 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.122 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-26T06:57:29.122 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.122 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.123 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.123 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2023-10-26T06:57:29.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2023-10-26T06:57:29.124 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.124 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.124 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.125 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2023-10-26T06:57:29.125 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.125 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.126 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.126 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-26T06:57:29.126 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.126 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.127 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.127 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2023-10-26T06:57:29.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2023-10-26T06:57:29.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.128 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.128 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.129 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.129 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.129 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.130 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.130 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-26T06:57:29.130 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.130 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.131 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.131 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2023-10-26T06:57:29.131 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2023-10-26T06:57:29.132 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.132 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.132 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.133 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.133 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.133 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.134 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.134 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-26T06:57:29.134 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.135 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.135 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.135 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2023-10-26T06:57:29.136 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2023-10-26T06:57:29.136 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.136 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.137 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.137 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.137 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.137 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.138 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.138 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.138 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.139 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.139 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.139 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2023-10-26T06:57:29.140 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2023-10-26T06:57:29.140 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.140 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.141 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.141 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.141 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.141 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.142 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.142 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.142 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.143 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.143 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.143 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2023-10-26T06:57:29.144 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2023-10-26T06:57:29.144 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.144 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.144 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.145 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-26T06:57:29.145 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.145 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.146 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.146 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.146 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.146 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.147 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.147 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2023-10-26T06:57:29.147 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2023-10-26T06:57:29.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.148 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.148 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.148 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.149 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.149 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.149 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.149 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.150 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.150 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.150 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.151 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:29.151 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:29.151 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:29.152 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.152 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.152 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2023-10-26T06:57:29.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2023-10-26T06:57:29.153 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.153 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.153 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.154 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.154 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.154 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.155 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2023-10-26T06:57:29.155 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2023-10-26T06:57:29.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:29.155 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.156 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.156 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2023-10-26T06:57:29.156 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.157 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.157 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.157 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.157 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.158 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.158 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.158 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2023-10-26T06:57:29.159 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2023-10-26T06:57:29.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.159 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.159 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.160 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.160 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.160 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.161 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.161 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.161 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.162 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.162 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2023-10-26T06:57:29.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2023-10-26T06:57:29.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.163 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.163 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.163 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.164 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.164 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.164 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.164 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.165 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.165 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.165 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.166 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.167 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.167 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.167 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.167 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.168 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.168 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.168 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.168 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.169 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.169 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.169 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.170 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.170 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.170 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.170 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.171 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.171 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.172 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.172 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.172 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.173 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.173 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.173 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.173 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.174 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.174 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.174 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.175 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.175 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.175 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.176 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.177 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.177 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.178 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.178 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.178 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.178 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.179 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.179 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.179 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.179 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.180 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.180 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.180 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.181 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.181 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.181 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.181 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.182 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.182 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.183 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.183 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.184 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.184 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.184 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.185 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.185 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.185 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.185 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.186 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.186 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2023-10-26T06:57:29.187 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2023-10-26T06:57:29.187 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.187 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.187 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.188 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.188 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.188 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.188 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.189 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.189 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.189 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.190 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.190 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2023-10-26T06:57:29.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2023-10-26T06:57:29.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.191 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.191 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.192 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.192 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.192 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.192 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.193 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.193 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.193 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.194 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.194 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2023-10-26T06:57:29.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2023-10-26T06:57:29.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.195 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.195 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.195 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-26T06:57:29.196 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.196 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.196 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.197 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.197 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.197 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.197 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2023-10-26T06:57:29.199 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.199 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.200 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.200 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.201 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.201 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.201 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.202 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.202 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.202 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.202 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.203 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.203 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.203 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.204 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.204 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.204 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.204 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.205 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.205 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.205 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.206 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.207 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.207 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.207 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.207 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.208 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.208 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.208 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.208 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.209 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.209 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.210 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.210 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.210 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.210 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.211 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.211 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.211 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.212 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.212 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.212 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.212 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.213 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.213 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.213 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.214 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.214 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.214 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.214 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.215 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.215 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.216 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.216 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.216 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.216 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.217 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.217 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.218 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.218 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.218 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.218 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.219 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.219 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.219 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2023-10-26T06:57:29.221 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.221 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.221 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.221 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.222 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.222 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.223 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.223 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.223 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.224 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.224 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.224 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.224 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.225 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.225 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.225 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.225 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.226 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.226 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.226 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.227 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.228 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.229 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.230 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.231 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.231 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.231 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.232 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.232 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.233 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.233 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.233 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.233 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.234 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.234 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.234 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.235 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.235 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.235 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.235 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.236 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.236 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.236 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.236 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.237 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.237 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.237 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.238 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.238 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.239 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.239 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.239 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.239 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.240 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.240 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.240 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.240 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.241 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2023-10-26T06:57:29.241 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2023-10-26T06:57:29.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:29.242 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.242 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.242 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2023-10-26T06:57:29.243 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.243 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.243 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.244 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-26T06:57:29.244 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.244 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.244 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.245 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2023-10-26T06:57:29.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2023-10-26T06:57:29.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.246 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.246 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.247 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.247 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.247 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.247 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2023-10-26T06:57:29.248 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.248 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.249 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2023-10-26T06:57:29.249 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2023-10-26T06:57:29.249 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:29.250 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.250 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.250 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2023-10-26T06:57:29.251 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.251 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.251 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.251 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-26T06:57:29.252 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.252 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.253 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.253 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.254 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2023-10-26T06:57:29.254 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2023-10-26T06:57:29.254 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2023-10-26T06:57:29.254 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2023-10-26T06:57:29.255 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2023-10-26T06:57:29.255 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2023-10-26T06:57:29.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2023-10-26T06:57:29.256 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.256 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.256 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2023-10-26T06:57:29.256 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.257 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.257 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.257 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.258 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.258 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.258 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.259 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2023-10-26T06:57:29.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2023-10-26T06:57:29.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.260 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.260 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.260 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.260 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.261 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.261 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.261 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.261 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.262 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.262 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.263 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2023-10-26T06:57:29.263 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2023-10-26T06:57:29.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2023-10-26T06:57:29.264 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.264 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.264 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2023-10-26T06:57:29.264 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.265 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.265 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.265 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.266 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.266 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.266 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.267 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2023-10-26T06:57:29.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2023-10-26T06:57:29.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.268 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.268 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.268 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.268 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.269 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.269 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.269 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.269 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.270 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.270 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.270 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.271 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.271 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.271 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.271 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.272 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.272 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.273 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2023-10-26T06:57:29.273 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2023-10-26T06:57:29.273 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.274 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.274 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.274 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.275 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.275 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.275 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.275 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.276 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.276 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.276 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.277 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2023-10-26T06:57:29.277 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2023-10-26T06:57:29.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.278 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.278 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2023-10-26T06:57:29.279 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.279 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.279 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2023-10-26T06:57:29.279 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2023-10-26T06:57:29.280 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2023-10-26T06:57:29.280 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2023-10-26T06:57:29.280 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2023-10-26T06:57:29.280 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2023-10-26T06:57:29.281 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2023-10-26T06:57:29.281 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.281 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.282 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.282 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.282 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.283 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2023-10-26T06:57:29.283 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2023-10-26T06:57:29.283 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2023-10-26T06:57:29.284 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.284 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2023-10-26T06:57:29.284 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2023-10-26T06:57:29.284 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.285 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.285 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.285 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.286 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.286 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.286 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.287 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2023-10-26T06:57:29.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2023-10-26T06:57:29.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.287 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.288 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.288 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2023-10-26T06:57:29.288 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.288 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.289 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.289 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.289 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.289 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.290 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.290 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2023-10-26T06:57:29.291 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2023-10-26T06:57:29.291 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2023-10-26T06:57:29.291 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.291 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.292 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2023-10-26T06:57:29.292 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.292 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.293 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.293 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.293 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.293 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.294 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.294 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2023-10-26T06:57:29.294 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2023-10-26T06:57:29.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2023-10-26T06:57:29.295 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2023-10-26T06:57:29.295 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2023-10-26T06:57:29.296 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-26T06:57:29.296 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2023-10-26T06:57:29.296 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2023-10-26T06:57:29.296 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2023-10-26T06:57:29.297 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2023-10-26T06:57:29.297 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2023-10-26T06:57:29.297 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2023-10-26T06:57:29.298 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2023-10-26T06:57:29.298 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.298 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.298 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.299 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.299 INFO:teuthology.orchestra.run.smithi089.stdout: "services": { 2023-10-26T06:57:29.299 INFO:teuthology.orchestra.run.smithi089.stdout: "dashboard": "https://172.21.15.89:8443/" 2023-10-26T06:57:29.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.300 INFO:teuthology.orchestra.run.smithi089.stdout: "always_on_modules": { 2023-10-26T06:57:29.300 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus": [ 2023-10-26T06:57:29.300 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2023-10-26T06:57:29.301 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2023-10-26T06:57:29.301 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2023-10-26T06:57:29.301 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2023-10-26T06:57:29.302 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2023-10-26T06:57:29.302 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2023-10-26T06:57:29.302 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2023-10-26T06:57:29.302 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2023-10-26T06:57:29.303 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2023-10-26T06:57:29.303 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2023-10-26T06:57:29.303 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.304 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific": [ 2023-10-26T06:57:29.304 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2023-10-26T06:57:29.304 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2023-10-26T06:57:29.304 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2023-10-26T06:57:29.305 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2023-10-26T06:57:29.305 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2023-10-26T06:57:29.305 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2023-10-26T06:57:29.305 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2023-10-26T06:57:29.306 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2023-10-26T06:57:29.306 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2023-10-26T06:57:29.306 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2023-10-26T06:57:29.307 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.307 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy": [ 2023-10-26T06:57:29.307 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2023-10-26T06:57:29.307 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2023-10-26T06:57:29.308 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2023-10-26T06:57:29.308 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2023-10-26T06:57:29.308 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2023-10-26T06:57:29.308 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2023-10-26T06:57:29.309 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2023-10-26T06:57:29.309 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2023-10-26T06:57:29.309 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2023-10-26T06:57:29.309 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2023-10-26T06:57:29.310 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2023-10-26T06:57:29.310 INFO:teuthology.orchestra.run.smithi089.stdout: "reef": [ 2023-10-26T06:57:29.310 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2023-10-26T06:57:29.311 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2023-10-26T06:57:29.311 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2023-10-26T06:57:29.311 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2023-10-26T06:57:29.311 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2023-10-26T06:57:29.312 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2023-10-26T06:57:29.312 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2023-10-26T06:57:29.312 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2023-10-26T06:57:29.312 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2023-10-26T06:57:29.313 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2023-10-26T06:57:29.313 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.314 INFO:teuthology.orchestra.run.smithi089.stdout: "last_failure_osd_epoch": 5, 2023-10-26T06:57:29.314 INFO:teuthology.orchestra.run.smithi089.stdout: "active_clients": [ 2023-10-26T06:57:29.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2023-10-26T06:57:29.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-10-26T06:57:29.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2023-10-26T06:57:29.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-26T06:57:29.316 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1736390285 2023-10-26T06:57:29.316 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.316 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "libcephsqlite", 2023-10-26T06:57:29.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-10-26T06:57:29.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.318 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2023-10-26T06:57:29.318 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-26T06:57:29.318 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 962950851 2023-10-26T06:57:29.319 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.319 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2023-10-26T06:57:29.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-10-26T06:57:29.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2023-10-26T06:57:29.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-26T06:57:29.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 3681015116 2023-10-26T06:57:29.321 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.322 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2023-10-26T06:57:29.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-10-26T06:57:29.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2023-10-26T06:57:29.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-26T06:57:29.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1475532166 2023-10-26T06:57:29.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.325 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2023-10-26T06:57:29.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-10-26T06:57:29.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2023-10-26T06:57:29.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-26T06:57:29.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 3691082765 2023-10-26T06:57:29.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.327 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:29.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2023-10-26T06:57:29.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2023-10-26T06:57:29.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2023-10-26T06:57:29.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2023-10-26T06:57:29.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2023-10-26T06:57:29.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 4099734451 2023-10-26T06:57:29.330 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.330 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.330 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:29.331 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-10-26T06:57:29.331 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:57:29.333 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:24 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4214504118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:57:29.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:24 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4049085878' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-26T06:57:29.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:25 smithi135 ceph-mon[131403]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:57:29.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:27 smithi135 ceph-mon[131403]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:29.334 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:24 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4214504118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:57:29.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:24 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4049085878' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-26T06:57:29.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:25 smithi192 ceph-mon[131661]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:57:29.335 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:27 smithi192 ceph-mon[131661]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:29.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4840a15f0 msgr2=0x7fd4840a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:29.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4840a15f0 0x7fd4840a3ab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd49c19a240 tx=0x7fd490009260 comp rx=0 tx=0).stop 2023-10-26T06:57:29.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 msgr2=0x7fd49c19ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:29.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 0x7fd49c19ea30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e9c0 tx=0x7fd48c00ecd0 comp rx=0 tx=0).stop 2023-10-26T06:57:29.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 shutdown_connections 2023-10-26T06:57:29.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd4840a15f0 0x7fd4840a3ab0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:29.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49c1066b0 0x7fd49c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:29.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c1006a0 0x7fd49c19ea30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:29.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 --2- 172.21.15.89:0/4049085878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd49c101060 0x7fd49c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:29.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.136+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 >> 172.21.15.89:0/4049085878 conn(0x7fd49c0fc170 msgr2=0x7fd49c10bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:29.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.137+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 shutdown_connections 2023-10-26T06:57:29.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:24.137+0000 7fd4a0dff700 1 -- 172.21.15.89:0/4049085878 wait complete. 2023-10-26T06:57:29.347 DEBUG:tasks.check_counter:Getting stats from a 2023-10-26T06:57:29.348 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-26T06:57:29.349 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:24 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4214504118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:57:29.349 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:24 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4049085878' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-26T06:57:29.349 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:25 smithi089 ceph-mon[95031]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2023-10-26T06:57:29.350 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:27 smithi089 ceph-mon[95031]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:29.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:29 smithi192 ceph-mon[131661]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:29.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:29 smithi135 ceph-mon[131403]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:29.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:29 smithi089 ceph-mon[95031]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:30.082 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:57:31.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:31 smithi192 ceph-mon[131661]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:31.686 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:57:31.687 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:31.687 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2023-10-26T06:57:31.687 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2023-10-26T06:57:31.687 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2023-10-26T06:57:31.688 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2023-10-26T06:57:31.688 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:31.688 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:31.689 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.003678415, 2023-10-26T06:57:31.689 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2023-10-26T06:57:31.689 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000780593, 2023-10-26T06:57:31.689 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:31.690 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:31.690 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:31.690 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:31.691 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:31.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.691 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:31.691 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:31.692 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:31.692 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:31.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.693 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:31.693 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:31.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.693 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:31.694 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 208, 2023-10-26T06:57:31.694 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 203, 2023-10-26T06:57:31.694 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 741600, 2023-10-26T06:57:31.694 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 121664, 2023-10-26T06:57:31.695 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:31.695 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:31.695 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.050106587, 2023-10-26T06:57:31.696 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.024073019, 2023-10-26T06:57:31.696 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.022168658, 2023-10-26T06:57:31.696 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.004833000, 2023-10-26T06:57:31.696 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:31.696 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 202, 2023-10-26T06:57:31.697 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.020064694, 2023-10-26T06:57:31.697 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000099330 2023-10-26T06:57:31.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.698 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:31.698 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:31.698 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:31.698 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:31.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.699 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 741600, 2023-10-26T06:57:31.699 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 121664 2023-10-26T06:57:31.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.700 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:31.700 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 5, 2023-10-26T06:57:31.700 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 156, 2023-10-26T06:57:31.701 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3104, 2023-10-26T06:57:31.701 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 104032, 2023-10-26T06:57:31.701 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:31.701 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:31.702 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.018599900, 2023-10-26T06:57:31.702 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.014266016, 2023-10-26T06:57:31.702 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.002191810, 2023-10-26T06:57:31.702 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:31.703 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:31.703 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 155, 2023-10-26T06:57:31.703 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.016484029, 2023-10-26T06:57:31.703 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000106348 2023-10-26T06:57:31.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.704 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:31.704 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:31.704 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:31.705 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:31.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.705 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-26T06:57:31.706 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 104032 2023-10-26T06:57:31.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.706 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-10-26T06:57:31.706 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2023-10-26T06:57:31.707 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-10-26T06:57:31.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.707 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-10-26T06:57:31.707 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:31.708 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:31.708 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:31.708 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:31.709 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:31.709 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:31.709 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:31.709 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:31.710 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:31.710 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:31.710 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:31.710 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:31.711 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:31.711 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:31.711 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:31.712 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:31.712 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:31.712 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:31.712 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:31.713 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:31.713 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:31.713 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:31.714 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:31.714 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:31.714 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:31.714 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:31.715 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:31.715 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:31.715 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:31.716 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:31.716 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:31.716 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-10-26T06:57:31.716 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:31.717 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:31.717 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:31.717 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:31.718 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148967, 2023-10-26T06:57:31.718 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 29, 2023-10-26T06:57:31.718 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:31.718 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-10-26T06:57:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-10-26T06:57:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2023-10-26T06:57:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-10-26T06:57:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2023-10-26T06:57:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-10-26T06:57:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-10-26T06:57:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-10-26T06:57:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-10-26T06:57:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-10-26T06:57:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 209, 2023-10-26T06:57:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 712217, 2023-10-26T06:57:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 209, 2023-10-26T06:57:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-10-26T06:57:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-10-26T06:57:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 209, 2023-10-26T06:57:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 712217, 2023-10-26T06:57:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-10-26T06:57:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:31.729 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:57:31.730 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:31 smithi135 ceph-mon[131403]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:31.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:31 smithi089 ceph-mon[95031]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2023-10-26T06:57:32.182 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-10-26T06:57:32.183 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-10-26T06:57:32.183 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-10-26T06:57:32.183 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-10-26T06:57:32.184 DEBUG:tasks.check_counter:Getting stats from d 2023-10-26T06:57:32.184 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-26T06:57:32.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:32 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:57:32.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:32 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:57:32.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:32 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-26T06:57:32.945 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:57:33.575 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:33 smithi089 ceph-mon[95031]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:57:33.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:33 smithi192 ceph-mon[131661]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:57:33.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:33 smithi135 ceph-mon[131403]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2023-10-26T06:57:34.662 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:57:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 218, 2023-10-26T06:57:34.663 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 211, 2023-10-26T06:57:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 746240, 2023-10-26T06:57:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 125392, 2023-10-26T06:57:34.664 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.043915279, 2023-10-26T06:57:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.014917614, 2023-10-26T06:57:34.665 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.022994990, 2023-10-26T06:57:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.005569103, 2023-10-26T06:57:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:34.666 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 211, 2023-10-26T06:57:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.022135816, 2023-10-26T06:57:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000104909 2023-10-26T06:57:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.667 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 746240, 2023-10-26T06:57:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 125392 2023-10-26T06:57:34.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1, 2023-10-26T06:57:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 158, 2023-10-26T06:57:34.670 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 160, 2023-10-26T06:57:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 104288, 2023-10-26T06:57:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:34.671 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.017344799, 2023-10-26T06:57:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.013428035, 2023-10-26T06:57:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001736175, 2023-10-26T06:57:34.672 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 156, 2023-10-26T06:57:34.673 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.015018267, 2023-10-26T06:57:34.674 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000096270 2023-10-26T06:57:34.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.675 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:34.676 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:34.676 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:34.677 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:34.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.677 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-26T06:57:34.678 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 104288 2023-10-26T06:57:34.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.678 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:34.679 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2023-10-26T06:57:34.679 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2023-10-26T06:57:34.679 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2023-10-26T06:57:34.680 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2023-10-26T06:57:34.680 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:34.680 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:34.681 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.001022005, 2023-10-26T06:57:34.681 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2023-10-26T06:57:34.681 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000910513, 2023-10-26T06:57:34.682 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:34.682 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:34.682 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:34.683 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:34.683 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:34.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.683 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:34.684 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:34.684 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:34.684 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:34.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.685 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:34.685 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:34.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.686 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-10-26T06:57:34.686 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2023-10-26T06:57:34.687 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-10-26T06:57:34.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.687 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-10-26T06:57:34.687 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:34.688 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:34.688 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:34.688 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:34.689 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:34.689 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:34.689 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:34.689 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:34.690 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:34.690 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:34.690 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:34.691 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:34.691 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:34.691 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:34.692 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:34.692 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:34.692 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:34.693 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:34.693 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:34.693 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:34.693 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:34.694 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:34.694 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:34.694 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:34.695 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:34.695 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:34.695 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:34.696 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:34.696 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:34.696 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:34.697 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:34.697 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-10-26T06:57:34.697 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:34.697 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:34.698 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:34.698 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:34.698 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 149998, 2023-10-26T06:57:34.699 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 30, 2023-10-26T06:57:34.699 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:34.699 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:34.700 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-10-26T06:57:34.700 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-10-26T06:57:34.700 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:34.700 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:34.701 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:34.701 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:34.701 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:34.702 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2023-10-26T06:57:34.702 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:34.702 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:34.703 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:34.703 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-10-26T06:57:34.703 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:34.703 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2023-10-26T06:57:34.704 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:34.704 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-10-26T06:57:34.704 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:34.705 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-10-26T06:57:34.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:34.705 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:34.705 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-10-26T06:57:34.706 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-10-26T06:57:34.706 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-10-26T06:57:34.706 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 215, 2023-10-26T06:57:34.707 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 713061, 2023-10-26T06:57:34.707 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:34.707 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 215, 2023-10-26T06:57:34.708 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-10-26T06:57:34.708 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-10-26T06:57:34.708 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 215, 2023-10-26T06:57:34.709 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 713061, 2023-10-26T06:57:34.709 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-10-26T06:57:34.709 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:34.709 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:34.710 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:34.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:34.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:34.711 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:57:35.269 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-10-26T06:57:35.270 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-10-26T06:57:35.270 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-10-26T06:57:35.271 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-10-26T06:57:35.271 DEBUG:tasks.check_counter:Getting stats from g 2023-10-26T06:57:35.272 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-26T06:57:35.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:35 smithi089 ceph-mon[95031]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:35.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:35 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:35 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:35 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.541 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:35 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:35 smithi192 ceph-mon[131661]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:35.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:35 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:35 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:35 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:35 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:35 smithi135 ceph-mon[131403]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:35.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:35 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:35 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:35 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:35.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:35 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:36.087 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:57:37.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:37 smithi192 ceph-mon[131661]: pgmap v641: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.396 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:57:37.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:57:37.397 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:37 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.467 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:37 smithi089 ceph-mon[95031]: pgmap v641: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:37.467 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.468 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.468 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:57:37.468 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:57:37.468 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:37 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:37 smithi135 ceph-mon[131403]: pgmap v641: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:37.469 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-26T06:57:37.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-26T06:57:37.470 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:37 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' 2023-10-26T06:57:37.750 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:57:37.751 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:37.751 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2023-10-26T06:57:37.752 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2023-10-26T06:57:37.752 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2023-10-26T06:57:37.752 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2023-10-26T06:57:37.752 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:37.753 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:37.753 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.003383224, 2023-10-26T06:57:37.753 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2023-10-26T06:57:37.753 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000647655, 2023-10-26T06:57:37.754 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:37.754 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:37.754 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:37.755 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:37.755 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:37.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.755 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:37.756 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:37.756 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:37.756 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:37.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.757 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:37.757 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:37.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.757 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:37.758 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1, 2023-10-26T06:57:37.758 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 156, 2023-10-26T06:57:37.758 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 160, 2023-10-26T06:57:37.759 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 103936, 2023-10-26T06:57:37.759 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:37.759 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:37.759 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.019292306, 2023-10-26T06:57:37.760 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.013468246, 2023-10-26T06:57:37.760 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.002274877, 2023-10-26T06:57:37.760 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:37.760 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:37.761 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 154, 2023-10-26T06:57:37.761 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.022500858, 2023-10-26T06:57:37.761 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000146109 2023-10-26T06:57:37.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.762 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:37.762 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:37.762 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:37.762 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:37.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.763 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-26T06:57:37.763 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 103936 2023-10-26T06:57:37.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.764 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:37.764 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 215, 2023-10-26T06:57:37.764 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 208, 2023-10-26T06:57:37.765 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 745360, 2023-10-26T06:57:37.765 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 123984, 2023-10-26T06:57:37.765 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:37.765 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:37.766 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.052406343, 2023-10-26T06:57:37.766 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.025541395, 2023-10-26T06:57:37.766 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.023983949, 2023-10-26T06:57:37.766 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.005259180, 2023-10-26T06:57:37.767 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:37.767 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 208, 2023-10-26T06:57:37.767 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.024540994, 2023-10-26T06:57:37.768 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000117985 2023-10-26T06:57:37.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.768 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:37.768 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:37.769 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:37.769 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:37.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.769 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 745360, 2023-10-26T06:57:37.770 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 123984 2023-10-26T06:57:37.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.770 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-10-26T06:57:37.770 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2023-10-26T06:57:37.771 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-10-26T06:57:37.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.771 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-10-26T06:57:37.771 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:37.772 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:37.772 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:37.772 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:37.773 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:37.773 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:37.773 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:37.773 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:37.774 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:37.774 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:37.774 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:37.774 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:37.775 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:37.775 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:37.775 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:37.775 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:37.776 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:37.776 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:37.776 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:37.776 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:37.777 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:37.777 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:37.777 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:37.778 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:37.778 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:37.778 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:37.778 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:37.779 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:37.779 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:37.779 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:37.779 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:37.780 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-10-26T06:57:37.780 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:37.780 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:37.780 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:37.781 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:37.781 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148967, 2023-10-26T06:57:37.781 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 29, 2023-10-26T06:57:37.782 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:37.782 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:37.782 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-10-26T06:57:37.782 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-10-26T06:57:37.783 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:37.783 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:37.783 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:37.783 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:37.784 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:37.784 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2023-10-26T06:57:37.784 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:37.784 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:37.785 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:37.785 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-10-26T06:57:37.785 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:37.786 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2023-10-26T06:57:37.786 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:37.786 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-10-26T06:57:37.786 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:37.787 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-10-26T06:57:37.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:37.787 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:37.788 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-10-26T06:57:37.788 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-10-26T06:57:37.788 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-10-26T06:57:37.788 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 212, 2023-10-26T06:57:37.789 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 712607, 2023-10-26T06:57:37.789 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:37.789 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 212, 2023-10-26T06:57:37.789 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-10-26T06:57:37.790 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-10-26T06:57:37.790 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 212, 2023-10-26T06:57:37.790 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 712607, 2023-10-26T06:57:37.790 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-10-26T06:57:37.791 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:37.791 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:37.791 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:37.792 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:37.792 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:37.792 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:57:38.284 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-10-26T06:57:38.285 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-10-26T06:57:38.285 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-10-26T06:57:38.286 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-10-26T06:57:38.286 DEBUG:tasks.check_counter:Getting stats from j 2023-10-26T06:57:38.286 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-26T06:57:39.021 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.a/config 2023-10-26T06:57:39.353 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:39 smithi089 ceph-mon[95031]: pgmap v642: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:39.395 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:39 smithi192 ceph-mon[131661]: pgmap v642: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:39.467 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:39 smithi135 ceph-mon[131403]: pgmap v642: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:40.545 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-10-26T06:57:40.546 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:40.546 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 217, 2023-10-26T06:57:40.546 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 212, 2023-10-26T06:57:40.546 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 744112, 2023-10-26T06:57:40.547 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 126816, 2023-10-26T06:57:40.547 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:40.547 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:40.548 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.048594261, 2023-10-26T06:57:40.548 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.021057802, 2023-10-26T06:57:40.548 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.021799619, 2023-10-26T06:57:40.548 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.004881630, 2023-10-26T06:57:40.549 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:40.549 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 211, 2023-10-26T06:57:40.549 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.018453408, 2023-10-26T06:57:40.550 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000087456 2023-10-26T06:57:40.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.550 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:40.550 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:40.551 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:40.551 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:40.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.552 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 744112, 2023-10-26T06:57:40.552 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 126816 2023-10-26T06:57:40.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.552 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:40.553 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 5, 2023-10-26T06:57:40.553 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 162, 2023-10-26T06:57:40.553 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3104, 2023-10-26T06:57:40.554 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 105088, 2023-10-26T06:57:40.554 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:40.554 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:40.554 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.018726057, 2023-10-26T06:57:40.555 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.014207959, 2023-10-26T06:57:40.555 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.002338664, 2023-10-26T06:57:40.555 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:40.555 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:40.556 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 161, 2023-10-26T06:57:40.556 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.017396915, 2023-10-26T06:57:40.556 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000108055 2023-10-26T06:57:40.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.557 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:40.557 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:40.557 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:40.558 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:40.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.558 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-26T06:57:40.559 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 105088 2023-10-26T06:57:40.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.561 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:40.561 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2023-10-26T06:57:40.562 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2023-10-26T06:57:40.562 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2023-10-26T06:57:40.563 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2023-10-26T06:57:40.563 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:40.563 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:40.563 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.000931601, 2023-10-26T06:57:40.564 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2023-10-26T06:57:40.564 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000792278, 2023-10-26T06:57:40.564 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:40.565 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:40.565 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:40.565 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:40.566 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:40.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.567 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:40.567 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:40.567 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:40.568 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:40.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.568 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:40.569 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:40.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.569 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-10-26T06:57:40.570 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2023-10-26T06:57:40.570 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-10-26T06:57:40.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.571 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-10-26T06:57:40.571 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:40.572 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:40.572 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:40.572 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:40.573 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:40.573 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:40.574 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:40.574 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:40.575 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:40.575 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:40.575 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:40.576 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:40.576 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:40.576 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:40.577 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:40.577 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:40.577 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:40.578 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:40.578 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:40.578 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:40.579 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:40.579 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:40.579 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:40.580 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:40.580 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:40.580 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:40.581 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:40.581 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:40.581 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:40.581 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:40.582 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:40.582 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-10-26T06:57:40.582 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:40.583 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:40.583 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:40.584 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:40.584 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148967, 2023-10-26T06:57:40.584 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 29, 2023-10-26T06:57:40.585 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:40.585 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:40.585 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-10-26T06:57:40.586 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-10-26T06:57:40.586 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:40.587 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:40.587 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:40.588 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:40.588 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:40.589 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2023-10-26T06:57:40.589 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:40.590 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:40.590 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:40.591 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-10-26T06:57:40.591 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:40.592 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2023-10-26T06:57:40.592 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:40.593 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-10-26T06:57:40.593 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:40.593 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-10-26T06:57:40.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-10-26T06:57:40.594 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:40.594 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-10-26T06:57:40.595 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-10-26T06:57:40.595 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-10-26T06:57:40.595 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 218, 2023-10-26T06:57:40.596 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 713451, 2023-10-26T06:57:40.596 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:40.596 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 218, 2023-10-26T06:57:40.597 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-10-26T06:57:40.597 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-10-26T06:57:40.597 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 218, 2023-10-26T06:57:40.597 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 713451, 2023-10-26T06:57:40.598 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-10-26T06:57:40.598 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-10-26T06:57:40.598 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-10-26T06:57:40.599 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-10-26T06:57:40.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:40.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-10-26T06:57:40.600 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-10-26T06:57:41.011 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-10-26T06:57:41.012 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-10-26T06:57:41.012 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-10-26T06:57:41.013 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-10-26T06:57:41.013 DEBUG:tasks.check_counter:Getting stats from b 2023-10-26T06:57:41.013 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-26T06:57:41.593 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:57:41.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:41 smithi192 ceph-mon[131661]: pgmap v643: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:41.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:41 smithi135 ceph-mon[131403]: pgmap v643: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:41.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:41 smithi089 ceph-mon[95031]: pgmap v643: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:42.796 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2023-10-26T06:57:42.796 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:42.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 3944, 2023-10-26T06:57:42.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 3946, 2023-10-26T06:57:42.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 1007940, 2023-10-26T06:57:42.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 13510150, 2023-10-26T06:57:42.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 11, 2023-10-26T06:57:42.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2023-10-26T06:57:42.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.250729134, 2023-10-26T06:57:42.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.084155104, 2023-10-26T06:57:42.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.051543064, 2023-10-26T06:57:42.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.085433724, 2023-10-26T06:57:42.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:42.799 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3944, 2023-10-26T06:57:42.800 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.434254524, 2023-10-26T06:57:42.800 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000110105 2023-10-26T06:57:42.800 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:42.801 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2023-10-26T06:57:42.801 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000121, 2023-10-26T06:57:42.801 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000060 2023-10-26T06:57:42.801 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:42.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:42.802 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.803 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:42.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 5520, 2023-10-26T06:57:42.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4904, 2023-10-26T06:57:42.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 2287165, 2023-10-26T06:57:42.804 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 16984944, 2023-10-26T06:57:42.804 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 11, 2023-10-26T06:57:42.804 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 6, 2023-10-26T06:57:42.805 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.453855792, 2023-10-26T06:57:42.805 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.152484295, 2023-10-26T06:57:42.805 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.121221491, 2023-10-26T06:57:42.805 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.158640089, 2023-10-26T06:57:42.806 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:42.806 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4904, 2023-10-26T06:57:42.806 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1.325877170, 2023-10-26T06:57:42.806 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000270366 2023-10-26T06:57:42.807 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.807 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:42.807 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 931, 2023-10-26T06:57:42.808 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000162033, 2023-10-26T06:57:42.808 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000174 2023-10-26T06:57:42.808 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.808 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1125296, 2023-10-26T06:57:42.809 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 255360 2023-10-26T06:57:42.809 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.809 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:42.809 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 15843, 2023-10-26T06:57:42.810 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 19170, 2023-10-26T06:57:42.810 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 5278546, 2023-10-26T06:57:42.810 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 10858877, 2023-10-26T06:57:42.811 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 158, 2023-10-26T06:57:42.811 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 9, 2023-10-26T06:57:42.811 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.884898705, 2023-10-26T06:57:42.811 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.340496795, 2023-10-26T06:57:42.812 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.791054219, 2023-10-26T06:57:42.812 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.079146262, 2023-10-26T06:57:42.812 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:42.813 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 19168, 2023-10-26T06:57:42.813 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 8.428562331, 2023-10-26T06:57:42.813 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000439720 2023-10-26T06:57:42.813 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.814 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:42.814 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 12451, 2023-10-26T06:57:42.814 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.004423541, 2023-10-26T06:57:42.814 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000355 2023-10-26T06:57:42.815 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.815 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 54416, 2023-10-26T06:57:42.815 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 384992 2023-10-26T06:57:42.815 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.816 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2023-10-26T06:57:42.816 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2023-10-26T06:57:42.816 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2023-10-26T06:57:42.817 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.817 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2023-10-26T06:57:42.817 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2023-10-26T06:57:42.817 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2023-10-26T06:57:42.818 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 8139, 2023-10-26T06:57:42.818 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 6.370342017, 2023-10-26T06:57:42.818 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000782693 2023-10-26T06:57:42.818 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.819 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.819 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2023-10-26T06:57:42.819 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2023-10-26T06:57:42.819 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2023-10-26T06:57:42.820 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 590, 2023-10-26T06:57:42.820 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.043015739, 2023-10-26T06:57:42.820 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000072908 2023-10-26T06:57:42.821 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.821 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.821 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2023-10-26T06:57:42.821 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 2212, 2023-10-26T06:57:42.822 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 2212, 2023-10-26T06:57:42.822 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2023-10-26T06:57:42.822 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2212, 2023-10-26T06:57:42.822 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 15.947651674, 2023-10-26T06:57:42.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.007209607 2023-10-26T06:57:42.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.823 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2023-10-26T06:57:42.824 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2023-10-26T06:57:42.824 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2023-10-26T06:57:42.824 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2023-10-26T06:57:42.824 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 38, 2023-10-26T06:57:42.825 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 3, 2023-10-26T06:57:42.825 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2023-10-26T06:57:42.825 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 1041, 2023-10-26T06:57:42.825 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 112, 2023-10-26T06:57:42.826 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 929, 2023-10-26T06:57:42.826 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2023-10-26T06:57:42.826 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 881, 2023-10-26T06:57:42.826 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2023-10-26T06:57:42.827 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 406, 2023-10-26T06:57:42.827 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 407, 2023-10-26T06:57:42.827 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 2, 2023-10-26T06:57:42.828 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 2942, 2023-10-26T06:57:42.828 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 2813, 2023-10-26T06:57:42.828 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2023-10-26T06:57:42.828 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2023-10-26T06:57:42.829 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2023-10-26T06:57:42.829 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2023-10-26T06:57:42.829 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 62, 2023-10-26T06:57:42.829 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2023-10-26T06:57:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2023-10-26T06:57:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2023-10-26T06:57:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2023-10-26T06:57:42.830 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2023-10-26T06:57:42.831 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2023-10-26T06:57:42.831 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2023-10-26T06:57:42.831 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 6585, 2023-10-26T06:57:42.832 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2023-10-26T06:57:42.832 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 324, 2023-10-26T06:57:42.832 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 13424350693, 2023-10-26T06:57:42.832 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 6, 2023-10-26T06:57:42.833 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 6585, 2023-10-26T06:57:42.833 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2023-10-26T06:57:42.833 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2023-10-26T06:57:42.833 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2023-10-26T06:57:42.834 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 1463, 2023-10-26T06:57:42.834 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 19, 2023-10-26T06:57:42.834 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 1444, 2023-10-26T06:57:42.835 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 5122, 2023-10-26T06:57:42.835 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2023-10-26T06:57:42.835 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 811, 2023-10-26T06:57:42.835 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 3662, 2023-10-26T06:57:42.836 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 6, 2023-10-26T06:57:42.836 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-26T06:57:42.836 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-26T06:57:42.836 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 11283, 2023-10-26T06:57:42.837 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 9321, 2023-10-26T06:57:42.837 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 132, 2023-10-26T06:57:42.837 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 1073 2023-10-26T06:57:42.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.838 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2023-10-26T06:57:42.838 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2023-10-26T06:57:42.838 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2023-10-26T06:57:42.838 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2023-10-26T06:57:42.839 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2023-10-26T06:57:42.839 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2023-10-26T06:57:42.839 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2023-10-26T06:57:42.840 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2023-10-26T06:57:42.840 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2023-10-26T06:57:42.840 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 529, 2023-10-26T06:57:42.840 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 468, 2023-10-26T06:57:42.841 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 61, 2023-10-26T06:57:42.841 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2023-10-26T06:57:42.841 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2023-10-26T06:57:42.841 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2023-10-26T06:57:42.842 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2023-10-26T06:57:42.842 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2023-10-26T06:57:42.842 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2023-10-26T06:57:42.843 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 19, 2023-10-26T06:57:42.843 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2023-10-26T06:57:42.843 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2023-10-26T06:57:42.843 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 3, 2023-10-26T06:57:42.844 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2023-10-26T06:57:42.844 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2023-10-26T06:57:42.844 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.844 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2023-10-26T06:57:42.845 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2023-10-26T06:57:42.845 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 13161, 2023-10-26T06:57:42.845 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2023-10-26T06:57:42.845 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2023-10-26T06:57:42.846 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 13161, 2023-10-26T06:57:42.846 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2023-10-26T06:57:42.846 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2023-10-26T06:57:42.847 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 20, 2023-10-26T06:57:42.847 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2023-10-26T06:57:42.847 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2023-10-26T06:57:42.847 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 20, 2023-10-26T06:57:42.848 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 3, 2023-10-26T06:57:42.848 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2023-10-26T06:57:42.848 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2023-10-26T06:57:42.848 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2023-10-26T06:57:42.849 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 42012103, 2023-10-26T06:57:42.849 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2023-10-26T06:57:42.849 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2023-10-26T06:57:42.849 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2076, 2023-10-26T06:57:42.850 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 432.682470011, 2023-10-26T06:57:42.850 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.208421228 2023-10-26T06:57:42.850 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.851 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2023-10-26T06:57:42.851 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.851 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2023-10-26T06:57:42.851 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 848, 2023-10-26T06:57:42.852 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 1335, 2023-10-26T06:57:42.852 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 487, 2023-10-26T06:57:42.852 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 122, 2023-10-26T06:57:42.852 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 141, 2023-10-26T06:57:42.853 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 19, 2023-10-26T06:57:42.853 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 1041, 2023-10-26T06:57:42.853 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 1599, 2023-10-26T06:57:42.854 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 558, 2023-10-26T06:57:42.854 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 407, 2023-10-26T06:57:42.854 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 886, 2023-10-26T06:57:42.854 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 479, 2023-10-26T06:57:42.855 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 70644, 2023-10-26T06:57:42.855 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 207152 2023-10-26T06:57:42.855 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.855 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2023-10-26T06:57:42.856 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 2746, 2023-10-26T06:57:42.856 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2023-10-26T06:57:42.856 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 2212, 2023-10-26T06:57:42.857 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 24, 2023-10-26T06:57:42.857 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2023-10-26T06:57:42.857 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2023-10-26T06:57:42.857 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 763, 2023-10-26T06:57:42.858 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.841181066, 2023-10-26T06:57:42.858 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001102465 2023-10-26T06:57:42.858 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.858 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2023-10-26T06:57:42.859 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 184, 2023-10-26T06:57:42.859 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.042696481, 2023-10-26T06:57:42.859 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000232046 2023-10-26T06:57:42.860 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.860 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2023-10-26T06:57:42.860 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.860 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.861 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.861 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.861 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2023-10-26T06:57:42.861 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 61, 2023-10-26T06:57:42.862 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.057639194, 2023-10-26T06:57:42.862 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000944904 2023-10-26T06:57:42.862 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.862 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2023-10-26T06:57:42.863 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 57, 2023-10-26T06:57:42.863 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.087167329, 2023-10-26T06:57:42.863 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001529251 2023-10-26T06:57:42.864 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.864 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2023-10-26T06:57:42.864 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.864 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.865 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.865 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.865 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2023-10-26T06:57:42.865 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2023-10-26T06:57:42.866 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000360056, 2023-10-26T06:57:42.866 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000180028 2023-10-26T06:57:42.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.866 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2023-10-26T06:57:42.867 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.867 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.867 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.868 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.868 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2023-10-26T06:57:42.868 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.868 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.869 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.869 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.869 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2023-10-26T06:57:42.869 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.870 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.870 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.870 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.871 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2023-10-26T06:57:42.871 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 11, 2023-10-26T06:57:42.871 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.001974913, 2023-10-26T06:57:42.871 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000179537 2023-10-26T06:57:42.872 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.872 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2023-10-26T06:57:42.872 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 105, 2023-10-26T06:57:42.872 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.114189098, 2023-10-26T06:57:42.873 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001087515 2023-10-26T06:57:42.873 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.873 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2023-10-26T06:57:42.873 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.874 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.874 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.874 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.875 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2023-10-26T06:57:42.875 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 11, 2023-10-26T06:57:42.875 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.314107419, 2023-10-26T06:57:42.875 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.028555219 2023-10-26T06:57:42.876 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.876 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2023-10-26T06:57:42.876 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 47, 2023-10-26T06:57:42.877 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.014763897, 2023-10-26T06:57:42.877 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000314125 2023-10-26T06:57:42.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.877 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2023-10-26T06:57:42.878 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 327, 2023-10-26T06:57:42.878 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1.073733060, 2023-10-26T06:57:42.878 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003283587 2023-10-26T06:57:42.878 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.879 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2023-10-26T06:57:42.879 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 113, 2023-10-26T06:57:42.879 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 12.506088309, 2023-10-26T06:57:42.879 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.110673347 2023-10-26T06:57:42.880 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.880 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2023-10-26T06:57:42.880 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.881 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.881 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.881 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.881 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2023-10-26T06:57:42.882 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.882 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.882 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.882 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.883 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2023-10-26T06:57:42.883 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 16, 2023-10-26T06:57:42.883 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.158308629, 2023-10-26T06:57:42.884 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.009894289 2023-10-26T06:57:42.884 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.884 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2023-10-26T06:57:42.884 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 5, 2023-10-26T06:57:42.885 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.019579037, 2023-10-26T06:57:42.885 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003915807 2023-10-26T06:57:42.885 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.885 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2023-10-26T06:57:42.886 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.886 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.886 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.887 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.887 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2023-10-26T06:57:42.887 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 7, 2023-10-26T06:57:42.887 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.032611351, 2023-10-26T06:57:42.888 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.004658764 2023-10-26T06:57:42.888 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.888 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2023-10-26T06:57:42.888 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.889 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.889 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.889 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.889 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2023-10-26T06:57:42.890 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4, 2023-10-26T06:57:42.890 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.001311479, 2023-10-26T06:57:42.890 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000327869 2023-10-26T06:57:42.891 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.891 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2023-10-26T06:57:42.891 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.891 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.892 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.892 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.892 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2023-10-26T06:57:42.892 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 6, 2023-10-26T06:57:42.893 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.018173916, 2023-10-26T06:57:42.893 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003028986 2023-10-26T06:57:42.893 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.894 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2023-10-26T06:57:42.894 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 13, 2023-10-26T06:57:42.894 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.008718530, 2023-10-26T06:57:42.894 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000670656 2023-10-26T06:57:42.895 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.895 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2023-10-26T06:57:42.895 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 480, 2023-10-26T06:57:42.896 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.655047910, 2023-10-26T06:57:42.896 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001364683 2023-10-26T06:57:42.896 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.896 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2023-10-26T06:57:42.897 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2023-10-26T06:57:42.897 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2023-10-26T06:57:42.897 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.897 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.898 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.898 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.898 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.899 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2023-10-26T06:57:42.899 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 2, 2023-10-26T06:57:42.899 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 3, 2023-10-26T06:57:42.899 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 1, 2023-10-26T06:57:42.900 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 2, 2023-10-26T06:57:42.900 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2023-10-26T06:57:42.900 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 31, 2023-10-26T06:57:42.900 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 15, 2023-10-26T06:57:42.901 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 647, 2023-10-26T06:57:42.901 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2023-10-26T06:57:42.901 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.901 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2023-10-26T06:57:42.902 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:42.902 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:42.902 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:42.903 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:42.903 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:42.903 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:42.903 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:42.904 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:42.904 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:42.904 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:42.904 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:42.905 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:42.905 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:42.905 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:42.905 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:42.906 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:42.906 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:42.906 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:42.907 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:42.907 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:42.907 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:42.907 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:42.908 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:42.908 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:42.908 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:42.908 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:42.909 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:42.909 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:42.909 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:42.910 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:42.910 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:42.910 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2023-10-26T06:57:42.910 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:42.911 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:42.913 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:42.913 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:42.914 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 322114, 2023-10-26T06:57:42.914 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 543, 2023-10-26T06:57:42.914 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 1232, 2023-10-26T06:57:42.914 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 14, 2023-10-26T06:57:42.915 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2023-10-26T06:57:42.915 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2023-10-26T06:57:42.915 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:42.916 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:42.916 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:42.916 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:42.916 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 8064, 2023-10-26T06:57:42.917 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 168, 2023-10-26T06:57:42.917 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:42.917 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:42.917 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:42.918 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2023-10-26T06:57:42.918 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 3044327, 2023-10-26T06:57:42.918 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 36511, 2023-10-26T06:57:42.918 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:42.919 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2023-10-26T06:57:42.919 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:42.919 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2023-10-26T06:57:42.920 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.920 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2023-10-26T06:57:42.920 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2023-10-26T06:57:42.920 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2023-10-26T06:57:42.921 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 9864, 2023-10-26T06:57:42.921 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 38366989, 2023-10-26T06:57:42.921 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 37, 2023-10-26T06:57:42.921 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 9864, 2023-10-26T06:57:42.922 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2023-10-26T06:57:42.922 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 9864, 2023-10-26T06:57:42.922 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 539.082633197, 2023-10-26T06:57:42.923 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.054651524 2023-10-26T06:57:42.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.923 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2023-10-26T06:57:42.923 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2023-10-26T06:57:42.924 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 9864, 2023-10-26T06:57:42.924 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 9988 2023-10-26T06:57:42.924 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.924 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 9864, 2023-10-26T06:57:42.925 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 6636, 2023-10-26T06:57:42.925 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 3228, 2023-10-26T06:57:42.925 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2023-10-26T06:57:42.925 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2023-10-26T06:57:42.926 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2023-10-26T06:57:42.926 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 11, 2023-10-26T06:57:42.926 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 32, 2023-10-26T06:57:42.927 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 2089, 2023-10-26T06:57:42.927 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 40, 2023-10-26T06:57:42.927 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2023-10-26T06:57:42.927 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2023-10-26T06:57:42.928 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2023-10-26T06:57:42.928 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 2, 2023-10-26T06:57:42.928 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 990, 2023-10-26T06:57:42.928 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2023-10-26T06:57:42.929 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2023-10-26T06:57:42.929 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2023-10-26T06:57:42.929 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 6604, 2023-10-26T06:57:42.929 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 22, 2023-10-26T06:57:42.930 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2023-10-26T06:57:42.930 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2023-10-26T06:57:42.930 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2023-10-26T06:57:42.931 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2023-10-26T06:57:42.931 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2023-10-26T06:57:42.931 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2023-10-26T06:57:42.931 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2023-10-26T06:57:42.932 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2023-10-26T06:57:42.932 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2023-10-26T06:57:42.932 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 6, 2023-10-26T06:57:42.933 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2023-10-26T06:57:42.933 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2023-10-26T06:57:42.933 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2023-10-26T06:57:42.933 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2023-10-26T06:57:42.934 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2023-10-26T06:57:42.934 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2023-10-26T06:57:42.934 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2023-10-26T06:57:42.934 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2023-10-26T06:57:42.935 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2023-10-26T06:57:42.935 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2023-10-26T06:57:42.935 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2023-10-26T06:57:42.935 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2023-10-26T06:57:42.936 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2023-10-26T06:57:42.936 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2023-10-26T06:57:42.936 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2023-10-26T06:57:42.937 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2023-10-26T06:57:42.937 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 96, 2023-10-26T06:57:42.937 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2023-10-26T06:57:42.937 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 13, 2023-10-26T06:57:42.938 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 12, 2023-10-26T06:57:42.938 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 12, 2023-10-26T06:57:42.938 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2023-10-26T06:57:42.939 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2023-10-26T06:57:42.939 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 149, 2023-10-26T06:57:42.939 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2023-10-26T06:57:42.939 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 43 2023-10-26T06:57:42.940 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.940 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2023-10-26T06:57:42.940 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2023-10-26T06:57:42.940 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2023-10-26T06:57:42.941 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2023-10-26T06:57:42.941 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2023-10-26T06:57:42.941 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.941 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2023-10-26T06:57:42.942 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2023-10-26T06:57:42.942 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.942 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2023-10-26T06:57:42.943 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2023-10-26T06:57:42.943 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 32, 2023-10-26T06:57:42.943 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2023-10-26T06:57:42.943 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 32, 2023-10-26T06:57:42.944 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 468, 2023-10-26T06:57:42.944 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2023-10-26T06:57:42.944 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.945 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:42.945 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:42.945 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2023-10-26T06:57:42.945 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:42.946 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 25303, 2023-10-26T06:57:42.946 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 6461763, 2023-10-26T06:57:42.946 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:42.946 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 25303, 2023-10-26T06:57:42.947 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2023-10-26T06:57:42.947 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2023-10-26T06:57:42.947 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 25303, 2023-10-26T06:57:42.948 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 6461763, 2023-10-26T06:57:42.948 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:42.948 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.948 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.949 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.949 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.950 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2023-10-26T06:57:42.950 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:42.950 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2023-10-26T06:57:42.950 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:42.951 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2023-10-26T06:57:42.951 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2023-10-26T06:57:42.951 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:42.952 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2023-10-26T06:57:42.952 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 9864, 2023-10-26T06:57:42.952 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 38414281, 2023-10-26T06:57:42.952 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 9864, 2023-10-26T06:57:42.953 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 38414281, 2023-10-26T06:57:42.953 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:42.953 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.954 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.954 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.954 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.954 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.955 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2023-10-26T06:57:42.955 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:42.955 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2023-10-26T06:57:42.955 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:42.956 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2023-10-26T06:57:42.956 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2023-10-26T06:57:42.956 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:42.957 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2023-10-26T06:57:42.957 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 9864, 2023-10-26T06:57:42.957 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 9864, 2023-10-26T06:57:42.957 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 9864, 2023-10-26T06:57:42.958 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 9864, 2023-10-26T06:57:42.958 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:42.958 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.958 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.959 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.959 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.959 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.960 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2023-10-26T06:57:42.960 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:42.960 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2023-10-26T06:57:42.960 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:42.961 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 468, 2023-10-26T06:57:42.961 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 47292, 2023-10-26T06:57:42.961 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:42.961 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 468, 2023-10-26T06:57:42.962 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2023-10-26T06:57:42.962 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2023-10-26T06:57:42.962 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 13, 2023-10-26T06:57:42.963 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 47292, 2023-10-26T06:57:42.963 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:42.963 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.963 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.964 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.964 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.964 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:42.964 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x564280b4e6e0": { 2023-10-26T06:57:42.965 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:42.965 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2023-10-26T06:57:42.965 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:42.965 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 13161, 2023-10-26T06:57:42.966 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 37817799, 2023-10-26T06:57:42.966 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:42.966 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 13161, 2023-10-26T06:57:42.967 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2023-10-26T06:57:42.967 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2023-10-26T06:57:42.967 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 2076, 2023-10-26T06:57:42.967 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 37817799, 2023-10-26T06:57:42.968 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:42.968 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:42.968 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:42.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:42.969 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.969 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:42.970 INFO:teuthology.orchestra.run.smithi135.stdout:} 2023-10-26T06:57:43.137 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-10-26T06:57:43.138 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2023-10-26T06:57:43.138 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2023-10-26T06:57:43.138 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=3 2023-10-26T06:57:43.139 DEBUG:tasks.check_counter:Getting stats from e 2023-10-26T06:57:43.139 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-26T06:57:43.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:43 smithi192 ceph-mon[131661]: pgmap v644: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:43.702 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:57:43.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:43 smithi135 ceph-mon[131403]: pgmap v644: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:43.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:43 smithi089 ceph-mon[95031]: pgmap v644: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:44.990 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2023-10-26T06:57:44.991 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:44.991 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 0, 2023-10-26T06:57:44.992 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 2, 2023-10-26T06:57:44.992 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 0, 2023-10-26T06:57:44.992 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 0, 2023-10-26T06:57:44.993 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:44.993 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:44.993 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.001220260, 2023-10-26T06:57:44.993 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000000000, 2023-10-26T06:57:44.994 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000327068, 2023-10-26T06:57:44.994 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:44.994 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:44.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:44.995 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:44.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:44.995 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:44.996 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:44.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:44.996 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:44.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:44.997 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:44.997 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:44.997 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:44.998 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:44.998 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:44.998 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 5, 2023-10-26T06:57:44.998 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 162, 2023-10-26T06:57:44.999 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 3104, 2023-10-26T06:57:44.999 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 105088, 2023-10-26T06:57:44.999 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:45.000 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:45.000 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.008855296, 2023-10-26T06:57:45.000 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.006220828, 2023-10-26T06:57:45.000 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000993052, 2023-10-26T06:57:45.001 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:45.001 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:45.001 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 161, 2023-10-26T06:57:45.002 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.034853382, 2023-10-26T06:57:45.002 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000216480 2023-10-26T06:57:45.002 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.002 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:45.003 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:45.003 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:45.003 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:45.004 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.004 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-26T06:57:45.004 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 105088 2023-10-26T06:57:45.005 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.005 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:45.005 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 216, 2023-10-26T06:57:45.006 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 211, 2023-10-26T06:57:45.006 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 743840, 2023-10-26T06:57:45.006 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 126176, 2023-10-26T06:57:45.006 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:45.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:45.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.035747971, 2023-10-26T06:57:45.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.012676514, 2023-10-26T06:57:45.007 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.018494578, 2023-10-26T06:57:45.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.006056929, 2023-10-26T06:57:45.008 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:45.008 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 210, 2023-10-26T06:57:45.009 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.019769699, 2023-10-26T06:57:45.009 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000094141 2023-10-26T06:57:45.009 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.009 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:45.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:45.010 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:45.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:45.011 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 743840, 2023-10-26T06:57:45.011 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 126176 2023-10-26T06:57:45.011 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.012 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2023-10-26T06:57:45.012 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2023-10-26T06:57:45.012 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2023-10-26T06:57:45.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.013 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2023-10-26T06:57:45.013 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:45.013 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:45.014 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:45.014 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:45.014 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:45.014 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:45.015 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:45.015 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:45.015 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:45.016 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:45.016 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:45.016 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:45.016 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:45.017 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:45.017 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:45.017 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:45.018 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:45.018 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:45.018 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:45.018 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:45.019 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:45.019 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:45.019 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:45.020 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:45.020 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:45.020 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:45.020 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:45.021 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:45.021 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:45.021 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:45.022 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:45.022 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2023-10-26T06:57:45.022 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:45.022 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:45.023 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:45.023 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:45.023 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 148967, 2023-10-26T06:57:45.024 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 29, 2023-10-26T06:57:45.024 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:45.024 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:45.024 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2023-10-26T06:57:45.025 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2023-10-26T06:57:45.025 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:45.025 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:45.026 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:45.026 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:45.026 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:45.027 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2023-10-26T06:57:45.027 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:45.027 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:45.028 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:45.028 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2023-10-26T06:57:45.028 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:45.028 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2023-10-26T06:57:45.029 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:45.029 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2023-10-26T06:57:45.029 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:45.029 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2023-10-26T06:57:45.030 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:45.030 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:45.030 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:45.031 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2023-10-26T06:57:45.031 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:45.031 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 217, 2023-10-26T06:57:45.031 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 713321, 2023-10-26T06:57:45.032 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:45.032 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 217, 2023-10-26T06:57:45.032 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2023-10-26T06:57:45.032 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2023-10-26T06:57:45.033 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 217, 2023-10-26T06:57:45.033 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 713321, 2023-10-26T06:57:45.033 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:45.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:45.034 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:45.034 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:45.034 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:45.035 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:45.035 INFO:teuthology.orchestra.run.smithi135.stdout:} 2023-10-26T06:57:45.356 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-10-26T06:57:45.357 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-10-26T06:57:45.357 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-10-26T06:57:45.358 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2023-10-26T06:57:45.358 DEBUG:tasks.check_counter:Getting stats from h 2023-10-26T06:57:45.359 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-26T06:57:45.384 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:45 smithi135 ceph-mon[131403]: pgmap v645: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:45.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:45 smithi192 ceph-mon[131661]: pgmap v645: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:45.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:45 smithi089 ceph-mon[95031]: pgmap v645: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:45.914 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:57:47.050 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2023-10-26T06:57:47.051 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:47.051 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 4, 2023-10-26T06:57:47.051 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4, 2023-10-26T06:57:47.051 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 2944, 2023-10-26T06:57:47.052 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 448, 2023-10-26T06:57:47.052 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:47.052 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:47.053 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.001563314, 2023-10-26T06:57:47.053 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000043982, 2023-10-26T06:57:47.053 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000609076, 2023-10-26T06:57:47.053 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:47.054 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:47.054 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3, 2023-10-26T06:57:47.054 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000768944, 2023-10-26T06:57:47.054 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000256314 2023-10-26T06:57:47.055 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.055 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:47.055 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:47.056 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:47.056 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:47.056 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.056 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-26T06:57:47.057 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2023-10-26T06:57:47.057 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.057 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:47.057 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1, 2023-10-26T06:57:47.058 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 159, 2023-10-26T06:57:47.058 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 160, 2023-10-26T06:57:47.058 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 104464, 2023-10-26T06:57:47.059 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:47.059 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:47.059 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.008637940, 2023-10-26T06:57:47.059 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.006143587, 2023-10-26T06:57:47.060 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000690606, 2023-10-26T06:57:47.060 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:47.060 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:47.060 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 157, 2023-10-26T06:57:47.061 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.032347213, 2023-10-26T06:57:47.061 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000206033 2023-10-26T06:57:47.061 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.061 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:47.062 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:47.062 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:47.062 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:47.063 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.063 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-26T06:57:47.063 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 104464 2023-10-26T06:57:47.063 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.064 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:47.064 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 214, 2023-10-26T06:57:47.064 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 209, 2023-10-26T06:57:47.064 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 743232, 2023-10-26T06:57:47.065 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 125392, 2023-10-26T06:57:47.065 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:47.065 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:47.065 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.035281218, 2023-10-26T06:57:47.066 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.012185763, 2023-10-26T06:57:47.066 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.018371941, 2023-10-26T06:57:47.067 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.006118961, 2023-10-26T06:57:47.067 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:47.067 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 208, 2023-10-26T06:57:47.068 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.027642108, 2023-10-26T06:57:47.068 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000132894 2023-10-26T06:57:47.068 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.068 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:47.069 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:47.069 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:47.069 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:47.070 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.070 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 743232, 2023-10-26T06:57:47.070 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 125392 2023-10-26T06:57:47.070 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.071 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2023-10-26T06:57:47.071 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2023-10-26T06:57:47.071 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2023-10-26T06:57:47.072 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.072 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2023-10-26T06:57:47.072 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:47.072 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:47.073 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:47.073 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:47.074 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:47.074 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:47.074 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:47.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:47.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:47.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:47.075 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:47.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:47.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:47.076 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:47.077 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:47.077 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:47.077 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:47.077 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:47.078 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:47.078 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:47.078 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:47.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:47.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:47.079 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:47.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:47.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:47.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:47.080 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:47.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:47.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:47.081 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:47.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2023-10-26T06:57:47.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:47.082 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:47.083 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:47.083 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:47.083 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 149007, 2023-10-26T06:57:47.083 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 30, 2023-10-26T06:57:47.084 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:47.084 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:47.084 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2023-10-26T06:57:47.084 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2023-10-26T06:57:47.085 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:47.085 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:47.085 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:47.085 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:47.086 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:47.086 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2023-10-26T06:57:47.086 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:47.087 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:47.087 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:47.087 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2023-10-26T06:57:47.088 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:47.088 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2023-10-26T06:57:47.088 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:47.089 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2023-10-26T06:57:47.089 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:47.090 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2023-10-26T06:57:47.090 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:47.091 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:47.091 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:47.091 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2023-10-26T06:57:47.091 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:47.092 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 215, 2023-10-26T06:57:47.092 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 712997, 2023-10-26T06:57:47.092 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:47.092 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 215, 2023-10-26T06:57:47.093 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2023-10-26T06:57:47.093 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2023-10-26T06:57:47.093 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 215, 2023-10-26T06:57:47.094 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 712997, 2023-10-26T06:57:47.094 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:47.094 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:47.095 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:47.095 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:47.095 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:47.095 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:47.096 INFO:teuthology.orchestra.run.smithi135.stdout:} 2023-10-26T06:57:47.455 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:47 smithi135 ceph-mon[131403]: pgmap v646: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:47.456 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2023-10-26T06:57:47.457 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2023-10-26T06:57:47.457 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2023-10-26T06:57:47.458 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.h 2023-10-26T06:57:47.458 DEBUG:tasks.check_counter:Getting stats from k 2023-10-26T06:57:47.459 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-26T06:57:47.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:47 smithi192 ceph-mon[131661]: pgmap v646: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:47.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:47 smithi089 ceph-mon[95031]: pgmap v646: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:48.052 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.b/config 2023-10-26T06:57:49.295 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2023-10-26T06:57:49.296 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:49.296 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 4, 2023-10-26T06:57:49.297 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4, 2023-10-26T06:57:49.297 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 2944, 2023-10-26T06:57:49.297 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 448, 2023-10-26T06:57:49.297 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:49.298 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:49.298 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.001559996, 2023-10-26T06:57:49.298 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000037283, 2023-10-26T06:57:49.299 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000639543, 2023-10-26T06:57:49.299 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:49.299 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:49.299 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3, 2023-10-26T06:57:49.300 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000843151, 2023-10-26T06:57:49.300 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000281050 2023-10-26T06:57:49.300 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.300 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:49.301 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:49.301 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:49.301 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:49.302 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.302 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-26T06:57:49.302 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2023-10-26T06:57:49.302 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.303 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:49.303 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1, 2023-10-26T06:57:49.303 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 163, 2023-10-26T06:57:49.304 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 160, 2023-10-26T06:57:49.304 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 105168, 2023-10-26T06:57:49.304 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:49.304 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:49.305 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.008539965, 2023-10-26T06:57:49.305 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.006189390, 2023-10-26T06:57:49.305 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000650221, 2023-10-26T06:57:49.306 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:49.306 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:49.306 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 161, 2023-10-26T06:57:49.306 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.010652078, 2023-10-26T06:57:49.307 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000066161 2023-10-26T06:57:49.307 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.307 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:49.307 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:49.308 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-26T06:57:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 105168 2023-10-26T06:57:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.309 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 219, 2023-10-26T06:57:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 214, 2023-10-26T06:57:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 744656, 2023-10-26T06:57:49.310 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 128048, 2023-10-26T06:57:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.037099002, 2023-10-26T06:57:49.311 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.013486966, 2023-10-26T06:57:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.018678005, 2023-10-26T06:57:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.006123007, 2023-10-26T06:57:49.312 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 213, 2023-10-26T06:57:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.020467836, 2023-10-26T06:57:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000096093 2023-10-26T06:57:49.313 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:49.314 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 744656, 2023-10-26T06:57:49.315 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 128048 2023-10-26T06:57:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2023-10-26T06:57:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2023-10-26T06:57:49.316 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2023-10-26T06:57:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2023-10-26T06:57:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:49.317 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:49.318 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:49.319 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:49.320 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:49.321 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:49.322 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:49.323 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:49.324 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:49.325 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2023-10-26T06:57:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:49.326 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 148984, 2023-10-26T06:57:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 30, 2023-10-26T06:57:49.327 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2023-10-26T06:57:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2023-10-26T06:57:49.328 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:49.329 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2023-10-26T06:57:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:49.330 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2023-10-26T06:57:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:49.331 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2023-10-26T06:57:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2023-10-26T06:57:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:49.332 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2023-10-26T06:57:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2023-10-26T06:57:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2023-10-26T06:57:49.333 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2023-10-26T06:57:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2023-10-26T06:57:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 220, 2023-10-26T06:57:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 713711, 2023-10-26T06:57:49.334 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 220, 2023-10-26T06:57:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2023-10-26T06:57:49.335 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2023-10-26T06:57:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 220, 2023-10-26T06:57:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 713711, 2023-10-26T06:57:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2023-10-26T06:57:49.336 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2023-10-26T06:57:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2023-10-26T06:57:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2023-10-26T06:57:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:49.337 INFO:teuthology.orchestra.run.smithi135.stdout: } 2023-10-26T06:57:49.338 INFO:teuthology.orchestra.run.smithi135.stdout:} 2023-10-26T06:57:49.596 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:49 smithi135 ceph-mon[131403]: pgmap v647: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:49.598 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-10-26T06:57:49.598 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-10-26T06:57:49.598 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-10-26T06:57:49.599 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-10-26T06:57:49.599 DEBUG:tasks.check_counter:Getting stats from c 2023-10-26T06:57:49.600 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-26T06:57:49.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:49 smithi192 ceph-mon[131661]: pgmap v647: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:49.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:49 smithi089 ceph-mon[95031]: pgmap v647: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:50.182 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:57:51.438 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-10-26T06:57:51.439 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:51.439 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 217, 2023-10-26T06:57:51.439 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 212, 2023-10-26T06:57:51.439 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 744112, 2023-10-26T06:57:51.440 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 126800, 2023-10-26T06:57:51.440 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:51.440 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:51.441 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.037789291, 2023-10-26T06:57:51.441 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.012620199, 2023-10-26T06:57:51.441 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.019051623, 2023-10-26T06:57:51.441 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.006742144, 2023-10-26T06:57:51.442 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:51.442 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 211, 2023-10-26T06:57:51.442 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.024142223, 2023-10-26T06:57:51.442 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000114418 2023-10-26T06:57:51.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.443 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:51.443 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:51.443 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:51.444 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:51.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.444 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 744112, 2023-10-26T06:57:51.445 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 126800 2023-10-26T06:57:51.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.445 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:51.445 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 5, 2023-10-26T06:57:51.446 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 163, 2023-10-26T06:57:51.446 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 3104, 2023-10-26T06:57:51.446 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 105264, 2023-10-26T06:57:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.009013010, 2023-10-26T06:57:51.447 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.006350843, 2023-10-26T06:57:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000969539, 2023-10-26T06:57:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:51.448 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 162, 2023-10-26T06:57:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.026058624, 2023-10-26T06:57:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000160855 2023-10-26T06:57:51.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:51.450 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-26T06:57:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 105264 2023-10-26T06:57:51.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 0, 2023-10-26T06:57:51.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2, 2023-10-26T06:57:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 0, 2023-10-26T06:57:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 0, 2023-10-26T06:57:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.000339524, 2023-10-26T06:57:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000000000, 2023-10-26T06:57:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000279819, 2023-10-26T06:57:51.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:51.455 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:51.456 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.457 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.458 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-10-26T06:57:51.459 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2023-10-26T06:57:51.459 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-10-26T06:57:51.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.459 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-10-26T06:57:51.460 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:51.460 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:51.460 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:51.460 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:51.461 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:51.461 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:51.461 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:51.461 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:51.462 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:51.463 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:51.464 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:51.465 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:51.466 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:51.467 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-10-26T06:57:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:51.468 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:51.469 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 149024, 2023-10-26T06:57:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 31, 2023-10-26T06:57:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:51.470 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-10-26T06:57:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-10-26T06:57:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:51.471 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:51.472 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2023-10-26T06:57:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:51.473 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-10-26T06:57:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2023-10-26T06:57:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:51.474 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-10-26T06:57:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-10-26T06:57:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-10-26T06:57:51.476 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 218, 2023-10-26T06:57:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 713451, 2023-10-26T06:57:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:51.477 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 218, 2023-10-26T06:57:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-10-26T06:57:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-10-26T06:57:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 218, 2023-10-26T06:57:51.478 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 713451, 2023-10-26T06:57:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:51.479 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:51.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:51.480 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-10-26T06:57:51.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:51 smithi192 ceph-mon[131661]: pgmap v648: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:51.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:51 smithi135 ceph-mon[131403]: pgmap v648: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:51.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:51 smithi089 ceph-mon[95031]: pgmap v648: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:51.837 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2023-10-26T06:57:51.837 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2023-10-26T06:57:51.838 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2023-10-26T06:57:51.838 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2023-10-26T06:57:51.838 DEBUG:tasks.check_counter:Getting stats from f 2023-10-26T06:57:51.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-26T06:57:52.396 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:57:53.512 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:53 smithi192 ceph-mon[131661]: pgmap v649: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:53.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:53 smithi135 ceph-mon[131403]: pgmap v649: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:53.727 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-10-26T06:57:53.727 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:53.728 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 218, 2023-10-26T06:57:53.728 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 213, 2023-10-26T06:57:53.728 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 744384, 2023-10-26T06:57:53.728 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 127424, 2023-10-26T06:57:53.729 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:53.729 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:53.729 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.037423201, 2023-10-26T06:57:53.730 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.013052925, 2023-10-26T06:57:53.730 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.019042333, 2023-10-26T06:57:53.730 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005992525, 2023-10-26T06:57:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 212, 2023-10-26T06:57:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.011409470, 2023-10-26T06:57:53.731 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000053818 2023-10-26T06:57:53.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.732 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:53.732 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:53.733 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:53.733 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:53.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.733 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 744384, 2023-10-26T06:57:53.734 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 127424 2023-10-26T06:57:53.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.734 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:53.735 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 5, 2023-10-26T06:57:53.735 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 163, 2023-10-26T06:57:53.735 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 3104, 2023-10-26T06:57:53.736 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 105264, 2023-10-26T06:57:53.736 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:53.736 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.010251316, 2023-10-26T06:57:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.006907177, 2023-10-26T06:57:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001430917, 2023-10-26T06:57:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 162, 2023-10-26T06:57:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.021192630, 2023-10-26T06:57:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000130818 2023-10-26T06:57:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-26T06:57:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 105264 2023-10-26T06:57:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 0, 2023-10-26T06:57:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2, 2023-10-26T06:57:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 0, 2023-10-26T06:57:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 0, 2023-10-26T06:57:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.000358005, 2023-10-26T06:57:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000000000, 2023-10-26T06:57:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000286874, 2023-10-26T06:57:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-26T06:57:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2023-10-26T06:57:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-10-26T06:57:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2023-10-26T06:57:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-10-26T06:57:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-10-26T06:57:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-10-26T06:57:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 149007, 2023-10-26T06:57:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 30, 2023-10-26T06:57:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-10-26T06:57:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-10-26T06:57:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2023-10-26T06:57:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-10-26T06:57:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2023-10-26T06:57:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-10-26T06:57:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-10-26T06:57:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-10-26T06:57:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 219, 2023-10-26T06:57:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 713581, 2023-10-26T06:57:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 219, 2023-10-26T06:57:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-10-26T06:57:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-10-26T06:57:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 219, 2023-10-26T06:57:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 713581, 2023-10-26T06:57:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:53.771 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-10-26T06:57:53.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:53 smithi089 ceph-mon[95031]: pgmap v649: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:54.141 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2023-10-26T06:57:54.141 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2023-10-26T06:57:54.141 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2023-10-26T06:57:54.142 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2023-10-26T06:57:54.142 DEBUG:tasks.check_counter:Getting stats from i 2023-10-26T06:57:54.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-26T06:57:54.698 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:57:55.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:55 smithi192 ceph-mon[131661]: pgmap v650: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:55.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:55 smithi135 ceph-mon[131403]: pgmap v650: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:55.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:55 smithi089 ceph-mon[95031]: pgmap v650: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:57:56.106 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-10-26T06:57:56.107 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:56.107 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1915, 2023-10-26T06:57:56.108 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1915, 2023-10-26T06:57:56.108 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 13202446, 2023-10-26T06:57:56.108 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 594779, 2023-10-26T06:57:56.108 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 7, 2023-10-26T06:57:56.109 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2023-10-26T06:57:56.109 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.180805275, 2023-10-26T06:57:56.109 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.049538711, 2023-10-26T06:57:56.110 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.050904395, 2023-10-26T06:57:56.110 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.064300412, 2023-10-26T06:57:56.110 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:56.110 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1914, 2023-10-26T06:57:56.111 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.114066051, 2023-10-26T06:57:56.111 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000059595 2023-10-26T06:57:56.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.112 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:56.112 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.112 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.112 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.113 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-26T06:57:56.113 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2023-10-26T06:57:56.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.114 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:56.114 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 216, 2023-10-26T06:57:56.114 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 375, 2023-10-26T06:57:56.114 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 16836115, 2023-10-26T06:57:56.115 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 422915, 2023-10-26T06:57:56.115 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2023-10-26T06:57:56.115 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2023-10-26T06:57:56.115 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.031704972, 2023-10-26T06:57:56.116 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.010264346, 2023-10-26T06:57:56.116 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.012976960, 2023-10-26T06:57:56.116 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005997992, 2023-10-26T06:57:56.117 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:56.117 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 373, 2023-10-26T06:57:56.117 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.025932767, 2023-10-26T06:57:56.117 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000069524 2023-10-26T06:57:56.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.118 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:56.118 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.118 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.119 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.119 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-26T06:57:56.119 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 355984 2023-10-26T06:57:56.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.120 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:56.120 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 465, 2023-10-26T06:57:56.121 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 459, 2023-10-26T06:57:56.121 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 9213083, 2023-10-26T06:57:56.121 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 201275, 2023-10-26T06:57:56.121 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:56.122 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2023-10-26T06:57:56.122 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.065229262, 2023-10-26T06:57:56.122 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.020988812, 2023-10-26T06:57:56.122 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.029496738, 2023-10-26T06:57:56.123 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.014052441, 2023-10-26T06:57:56.123 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:56.123 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 458, 2023-10-26T06:57:56.124 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.043604802, 2023-10-26T06:57:56.124 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000095206 2023-10-26T06:57:56.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.124 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:56.125 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.125 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.125 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.126 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 767120, 2023-10-26T06:57:56.126 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 129744 2023-10-26T06:57:56.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.127 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-10-26T06:57:56.127 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2023-10-26T06:57:56.127 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-10-26T06:57:56.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.128 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2023-10-26T06:57:56.128 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2023-10-26T06:57:56.128 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2023-10-26T06:57:56.129 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3947, 2023-10-26T06:57:56.129 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.111051930, 2023-10-26T06:57:56.129 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000028135 2023-10-26T06:57:56.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.130 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2023-10-26T06:57:56.130 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2023-10-26T06:57:56.131 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2023-10-26T06:57:56.131 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.131 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.131 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.132 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2023-10-26T06:57:56.132 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2023-10-26T06:57:56.133 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2023-10-26T06:57:56.133 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2023-10-26T06:57:56.133 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.134 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.134 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.134 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2023-10-26T06:57:56.135 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2023-10-26T06:57:56.135 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2023-10-26T06:57:56.135 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2023-10-26T06:57:56.135 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2023-10-26T06:57:56.136 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2023-10-26T06:57:56.136 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2023-10-26T06:57:56.136 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 1504, 2023-10-26T06:57:56.136 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2023-10-26T06:57:56.137 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 0, 2023-10-26T06:57:56.137 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 1504, 2023-10-26T06:57:56.137 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 1504, 2023-10-26T06:57:56.138 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2023-10-26T06:57:56.138 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2023-10-26T06:57:56.138 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2023-10-26T06:57:56.138 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 2, 2023-10-26T06:57:56.139 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2023-10-26T06:57:56.139 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2023-10-26T06:57:56.139 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2023-10-26T06:57:56.140 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2023-10-26T06:57:56.140 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2023-10-26T06:57:56.140 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2023-10-26T06:57:56.140 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2023-10-26T06:57:56.141 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2023-10-26T06:57:56.141 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2023-10-26T06:57:56.141 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2023-10-26T06:57:56.141 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2023-10-26T06:57:56.142 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2023-10-26T06:57:56.142 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2023-10-26T06:57:56.142 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2023-10-26T06:57:56.142 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2023-10-26T06:57:56.143 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2023-10-26T06:57:56.143 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 324, 2023-10-26T06:57:56.143 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 13424350693, 2023-10-26T06:57:56.144 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 6, 2023-10-26T06:57:56.144 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2023-10-26T06:57:56.144 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2023-10-26T06:57:56.144 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2023-10-26T06:57:56.145 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2023-10-26T06:57:56.145 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2023-10-26T06:57:56.145 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2023-10-26T06:57:56.145 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2023-10-26T06:57:56.146 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2023-10-26T06:57:56.146 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2023-10-26T06:57:56.146 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2023-10-26T06:57:56.146 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2023-10-26T06:57:56.147 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2023-10-26T06:57:56.147 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-26T06:57:56.147 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-26T06:57:56.148 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2023-10-26T06:57:56.148 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2023-10-26T06:57:56.148 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2023-10-26T06:57:56.148 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2023-10-26T06:57:56.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.149 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2023-10-26T06:57:56.149 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2023-10-26T06:57:56.149 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2023-10-26T06:57:56.150 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2023-10-26T06:57:56.150 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2023-10-26T06:57:56.150 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2023-10-26T06:57:56.151 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2023-10-26T06:57:56.151 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2023-10-26T06:57:56.151 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2023-10-26T06:57:56.151 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2023-10-26T06:57:56.152 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2023-10-26T06:57:56.152 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2023-10-26T06:57:56.152 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2023-10-26T06:57:56.153 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2023-10-26T06:57:56.153 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2023-10-26T06:57:56.153 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2023-10-26T06:57:56.153 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2023-10-26T06:57:56.154 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2023-10-26T06:57:56.154 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2023-10-26T06:57:56.154 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2023-10-26T06:57:56.155 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2023-10-26T06:57:56.155 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2023-10-26T06:57:56.155 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2023-10-26T06:57:56.155 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2023-10-26T06:57:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2023-10-26T06:57:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2023-10-26T06:57:56.156 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2023-10-26T06:57:56.157 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2023-10-26T06:57:56.157 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2023-10-26T06:57:56.157 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 13161, 2023-10-26T06:57:56.158 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2023-10-26T06:57:56.158 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2023-10-26T06:57:56.158 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2023-10-26T06:57:56.158 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2023-10-26T06:57:56.159 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2023-10-26T06:57:56.159 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 20, 2023-10-26T06:57:56.159 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 3, 2023-10-26T06:57:56.159 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2023-10-26T06:57:56.160 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2023-10-26T06:57:56.160 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2023-10-26T06:57:56.160 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 42012103, 2023-10-26T06:57:56.161 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 42011991, 2023-10-26T06:57:56.161 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2023-10-26T06:57:56.161 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.161 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.162 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.162 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 13161 2023-10-26T06:57:56.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.163 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2023-10-26T06:57:56.163 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 841, 2023-10-26T06:57:56.163 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 1309, 2023-10-26T06:57:56.164 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 468, 2023-10-26T06:57:56.164 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 115, 2023-10-26T06:57:56.164 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 134, 2023-10-26T06:57:56.164 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 19, 2023-10-26T06:57:56.165 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 1504, 2023-10-26T06:57:56.165 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 1552, 2023-10-26T06:57:56.165 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 48, 2023-10-26T06:57:56.165 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2023-10-26T06:57:56.166 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2023-10-26T06:57:56.166 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2023-10-26T06:57:56.166 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 53740, 2023-10-26T06:57:56.166 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 207152 2023-10-26T06:57:56.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.167 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2023-10-26T06:57:56.167 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2023-10-26T06:57:56.167 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2023-10-26T06:57:56.168 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2023-10-26T06:57:56.168 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2023-10-26T06:57:56.168 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2023-10-26T06:57:56.169 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2023-10-26T06:57:56.169 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.169 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.169 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.170 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2023-10-26T06:57:56.170 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.171 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.171 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.171 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2023-10-26T06:57:56.172 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.172 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.172 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.173 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2023-10-26T06:57:56.173 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.173 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.174 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.174 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2023-10-26T06:57:56.174 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.175 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.175 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.175 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2023-10-26T06:57:56.176 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.176 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.176 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.177 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2023-10-26T06:57:56.177 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.177 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.178 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.178 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2023-10-26T06:57:56.179 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.179 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.179 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.180 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2023-10-26T06:57:56.180 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.180 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.180 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.181 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2023-10-26T06:57:56.181 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.182 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.182 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.182 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2023-10-26T06:57:56.183 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.183 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.183 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.184 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2023-10-26T06:57:56.184 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.184 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.185 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.185 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2023-10-26T06:57:56.185 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.186 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.186 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.186 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2023-10-26T06:57:56.187 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.187 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.187 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.188 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2023-10-26T06:57:56.188 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.188 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2023-10-26T06:57:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.190 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.190 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.191 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2023-10-26T06:57:56.191 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.191 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.191 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.192 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2023-10-26T06:57:56.192 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.193 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.193 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.193 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2023-10-26T06:57:56.194 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.194 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.194 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.195 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2023-10-26T06:57:56.195 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.195 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.196 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.196 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2023-10-26T06:57:56.196 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.197 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.197 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.198 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2023-10-26T06:57:56.198 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.198 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.198 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.199 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2023-10-26T06:57:56.199 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.199 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.200 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.200 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2023-10-26T06:57:56.200 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.201 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.201 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.202 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2023-10-26T06:57:56.202 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.202 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.202 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.203 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2023-10-26T06:57:56.203 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.204 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.204 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.204 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2023-10-26T06:57:56.205 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.205 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.205 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.206 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2023-10-26T06:57:56.206 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.206 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.207 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.207 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2023-10-26T06:57:56.208 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.208 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.208 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.209 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2023-10-26T06:57:56.209 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2023-10-26T06:57:56.209 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2023-10-26T06:57:56.209 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.210 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.210 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.211 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2023-10-26T06:57:56.211 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2023-10-26T06:57:56.211 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 3, 2023-10-26T06:57:56.212 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 1, 2023-10-26T06:57:56.212 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 2, 2023-10-26T06:57:56.212 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2023-10-26T06:57:56.212 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2023-10-26T06:57:56.213 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2023-10-26T06:57:56.213 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 2617, 2023-10-26T06:57:56.213 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2023-10-26T06:57:56.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.214 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-10-26T06:57:56.214 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:56.214 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:56.215 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:56.215 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:56.215 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:56.215 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:56.216 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:56.216 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:56.216 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:56.216 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:56.217 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:56.217 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:56.217 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:56.218 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:56.218 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:56.218 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:56.218 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:56.219 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:56.219 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:56.219 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:56.219 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:56.220 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:56.220 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:56.220 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:56.221 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:56.221 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:56.221 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:56.221 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:56.222 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:56.222 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:56.222 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:56.222 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-10-26T06:57:56.223 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:56.223 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:56.223 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:56.224 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:56.224 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 237891, 2023-10-26T06:57:56.224 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 989, 2023-10-26T06:57:56.224 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:56.225 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:56.225 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-10-26T06:57:56.225 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-10-26T06:57:56.226 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:56.226 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:56.226 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:56.226 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:56.227 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 8064, 2023-10-26T06:57:56.227 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 168, 2023-10-26T06:57:56.227 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:56.227 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:56.228 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:56.228 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-10-26T06:57:56.228 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 3211389, 2023-10-26T06:57:56.229 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 36998, 2023-10-26T06:57:56.229 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:56.229 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-10-26T06:57:56.229 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:56.230 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-10-26T06:57:56.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.230 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2023-10-26T06:57:56.230 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2023-10-26T06:57:56.231 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2023-10-26T06:57:56.231 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 2356, 2023-10-26T06:57:56.231 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 26, 2023-10-26T06:57:56.232 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 4, 2023-10-26T06:57:56.232 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 2356, 2023-10-26T06:57:56.232 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2023-10-26T06:57:56.232 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2356, 2023-10-26T06:57:56.233 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 38.497242787, 2023-10-26T06:57:56.233 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.016340086 2023-10-26T06:57:56.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.233 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2023-10-26T06:57:56.234 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2023-10-26T06:57:56.234 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2356, 2023-10-26T06:57:56.234 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2357 2023-10-26T06:57:56.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.235 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 2356, 2023-10-26T06:57:56.235 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 2356, 2023-10-26T06:57:56.235 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2023-10-26T06:57:56.236 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2023-10-26T06:57:56.236 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2023-10-26T06:57:56.236 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 1445, 2023-10-26T06:57:56.236 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2023-10-26T06:57:56.237 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 908, 2023-10-26T06:57:56.237 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2023-10-26T06:57:56.237 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2023-10-26T06:57:56.237 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2023-10-26T06:57:56.238 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2023-10-26T06:57:56.238 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2023-10-26T06:57:56.238 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2023-10-26T06:57:56.239 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2023-10-26T06:57:56.239 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2023-10-26T06:57:56.239 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2023-10-26T06:57:56.239 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2023-10-26T06:57:56.240 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 2, 2023-10-26T06:57:56.240 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2023-10-26T06:57:56.240 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2023-10-26T06:57:56.240 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2023-10-26T06:57:56.241 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2023-10-26T06:57:56.241 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2023-10-26T06:57:56.241 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2023-10-26T06:57:56.241 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2023-10-26T06:57:56.242 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2023-10-26T06:57:56.242 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2023-10-26T06:57:56.242 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2023-10-26T06:57:56.242 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2023-10-26T06:57:56.243 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2023-10-26T06:57:56.243 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2023-10-26T06:57:56.243 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2023-10-26T06:57:56.244 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2023-10-26T06:57:56.244 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2023-10-26T06:57:56.244 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2023-10-26T06:57:56.244 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2023-10-26T06:57:56.245 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2023-10-26T06:57:56.245 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2023-10-26T06:57:56.245 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2023-10-26T06:57:56.245 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2023-10-26T06:57:56.246 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2023-10-26T06:57:56.246 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2023-10-26T06:57:56.246 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2023-10-26T06:57:56.247 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2023-10-26T06:57:56.247 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2023-10-26T06:57:56.247 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 96, 2023-10-26T06:57:56.247 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2023-10-26T06:57:56.248 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 12, 2023-10-26T06:57:56.248 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 7, 2023-10-26T06:57:56.248 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 7, 2023-10-26T06:57:56.248 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2023-10-26T06:57:56.249 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2023-10-26T06:57:56.249 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2023-10-26T06:57:56.249 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2023-10-26T06:57:56.250 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2023-10-26T06:57:56.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.250 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2023-10-26T06:57:56.250 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2023-10-26T06:57:56.251 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2023-10-26T06:57:56.251 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2023-10-26T06:57:56.252 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2023-10-26T06:57:56.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.253 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2023-10-26T06:57:56.253 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2023-10-26T06:57:56.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.254 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2023-10-26T06:57:56.254 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2023-10-26T06:57:56.254 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2023-10-26T06:57:56.255 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2023-10-26T06:57:56.255 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2023-10-26T06:57:56.255 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2023-10-26T06:57:56.256 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2023-10-26T06:57:56.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.256 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:56.256 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:56.257 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-10-26T06:57:56.257 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:56.257 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 2592, 2023-10-26T06:57:56.258 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 39003372, 2023-10-26T06:57:56.258 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:56.258 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 2592, 2023-10-26T06:57:56.258 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-10-26T06:57:56.259 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-10-26T06:57:56.259 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2592, 2023-10-26T06:57:56.259 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 39003372, 2023-10-26T06:57:56.259 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:56.260 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.260 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.260 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.261 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2023-10-26T06:57:56.261 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:56.262 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-10-26T06:57:56.262 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:56.262 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2023-10-26T06:57:56.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2023-10-26T06:57:56.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:56.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2023-10-26T06:57:56.263 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2356, 2023-10-26T06:57:56.264 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 37822602, 2023-10-26T06:57:56.264 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2356, 2023-10-26T06:57:56.264 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 37822602, 2023-10-26T06:57:56.264 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:56.265 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.265 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.265 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.266 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2023-10-26T06:57:56.266 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:56.267 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2023-10-26T06:57:56.267 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:56.267 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2023-10-26T06:57:56.268 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2023-10-26T06:57:56.268 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:56.268 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2023-10-26T06:57:56.268 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2356, 2023-10-26T06:57:56.269 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 2356, 2023-10-26T06:57:56.269 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2356, 2023-10-26T06:57:56.269 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 2356, 2023-10-26T06:57:56.269 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:56.270 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.270 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.270 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.271 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2023-10-26T06:57:56.271 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:56.272 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2023-10-26T06:57:56.272 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:56.272 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2023-10-26T06:57:56.272 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2023-10-26T06:57:56.273 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:56.273 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2023-10-26T06:57:56.273 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-10-26T06:57:56.273 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-10-26T06:57:56.274 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2023-10-26T06:57:56.274 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2023-10-26T06:57:56.274 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:56.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.275 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:56.276 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x559d7b474960": { 2023-10-26T06:57:56.276 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:56.276 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2023-10-26T06:57:56.277 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:56.277 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2023-10-26T06:57:56.277 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2023-10-26T06:57:56.278 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:56.278 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2023-10-26T06:57:56.278 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-10-26T06:57:56.278 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-10-26T06:57:56.279 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2023-10-26T06:57:56.279 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2023-10-26T06:57:56.279 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:56.279 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:56.280 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:56.280 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:56.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:56.281 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-10-26T06:57:56.557 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-10-26T06:57:56.557 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-10-26T06:57:56.558 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-10-26T06:57:56.558 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2023-10-26T06:57:56.558 DEBUG:tasks.check_counter:Getting stats from l 2023-10-26T06:57:56.559 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:384993681abceb8cb8affea10538d607052db4b4 shell --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-26T06:57:57.115 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/mon.c/config 2023-10-26T06:57:57.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:57 smithi192 ceph-mon[131661]: pgmap v651: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:57.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:57 smithi135 ceph-mon[131403]: pgmap v651: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:57.789 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:57 smithi089 ceph-mon[95031]: pgmap v651: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2023-10-26T06:57:58.345 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-10-26T06:57:58.345 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2023-10-26T06:57:58.346 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 4, 2023-10-26T06:57:58.346 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 4, 2023-10-26T06:57:58.346 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 2944, 2023-10-26T06:57:58.346 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 448, 2023-10-26T06:57:58.347 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2023-10-26T06:57:58.347 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2023-10-26T06:57:58.347 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.001551148, 2023-10-26T06:57:58.348 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000031982, 2023-10-26T06:57:58.348 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000614318, 2023-10-26T06:57:58.348 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:58.348 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:58.349 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3, 2023-10-26T06:57:58.349 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000820699, 2023-10-26T06:57:58.349 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000273566 2023-10-26T06:57:58.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.350 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:58.350 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:58.351 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:58.351 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:58.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.352 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-26T06:57:58.352 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2023-10-26T06:57:58.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.352 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2023-10-26T06:57:58.353 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1, 2023-10-26T06:57:58.353 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 165, 2023-10-26T06:57:58.353 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 160, 2023-10-26T06:57:58.353 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 105520, 2023-10-26T06:57:58.354 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 3, 2023-10-26T06:57:58.354 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2023-10-26T06:57:58.354 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.008715140, 2023-10-26T06:57:58.354 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.006152020, 2023-10-26T06:57:58.355 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000759298, 2023-10-26T06:57:58.355 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-26T06:57:58.355 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:58.356 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 163, 2023-10-26T06:57:58.356 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.014974070, 2023-10-26T06:57:58.356 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000091865 2023-10-26T06:57:58.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.359 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:58.359 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:58.360 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:58.360 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:58.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.361 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-26T06:57:58.361 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 105520 2023-10-26T06:57:58.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.362 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2023-10-26T06:57:58.362 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 222, 2023-10-26T06:57:58.362 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 217, 2023-10-26T06:57:58.363 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 745472, 2023-10-26T06:57:58.363 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 129920, 2023-10-26T06:57:58.363 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2023-10-26T06:57:58.363 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2023-10-26T06:57:58.364 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.036708258, 2023-10-26T06:57:58.364 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.012484085, 2023-10-26T06:57:58.364 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.019365306, 2023-10-26T06:57:58.365 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.006507795, 2023-10-26T06:57:58.365 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2023-10-26T06:57:58.365 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 216, 2023-10-26T06:57:58.365 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.027839466, 2023-10-26T06:57:58.366 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000128886 2023-10-26T06:57:58.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.366 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2023-10-26T06:57:58.367 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:58.367 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:58.367 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:58.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.368 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 745472, 2023-10-26T06:57:58.368 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 129920 2023-10-26T06:57:58.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.369 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2023-10-26T06:57:58.369 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2023-10-26T06:57:58.369 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2023-10-26T06:57:58.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.370 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2023-10-26T06:57:58.370 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2023-10-26T06:57:58.370 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2023-10-26T06:57:58.370 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2023-10-26T06:57:58.371 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2023-10-26T06:57:58.371 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2023-10-26T06:57:58.372 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2023-10-26T06:57:58.372 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-26T06:57:58.372 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2023-10-26T06:57:58.372 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-26T06:57:58.373 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2023-10-26T06:57:58.373 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2023-10-26T06:57:58.373 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2023-10-26T06:57:58.374 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-26T06:57:58.374 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2023-10-26T06:57:58.374 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2023-10-26T06:57:58.374 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2023-10-26T06:57:58.375 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2023-10-26T06:57:58.375 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2023-10-26T06:57:58.375 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-26T06:57:58.376 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2023-10-26T06:57:58.376 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-26T06:57:58.376 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2023-10-26T06:57:58.376 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2023-10-26T06:57:58.377 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2023-10-26T06:57:58.377 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2023-10-26T06:57:58.377 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2023-10-26T06:57:58.377 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-26T06:57:58.378 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2023-10-26T06:57:58.378 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2023-10-26T06:57:58.378 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2023-10-26T06:57:58.379 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2023-10-26T06:57:58.379 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2023-10-26T06:57:58.379 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2023-10-26T06:57:58.380 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2023-10-26T06:57:58.380 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2023-10-26T06:57:58.380 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2023-10-26T06:57:58.380 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 148967, 2023-10-26T06:57:58.381 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 29, 2023-10-26T06:57:58.381 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2023-10-26T06:57:58.381 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2023-10-26T06:57:58.382 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2023-10-26T06:57:58.382 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2023-10-26T06:57:58.382 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2023-10-26T06:57:58.382 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2023-10-26T06:57:58.383 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2023-10-26T06:57:58.383 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2023-10-26T06:57:58.383 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2023-10-26T06:57:58.384 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2023-10-26T06:57:58.384 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2023-10-26T06:57:58.384 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2023-10-26T06:57:58.384 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2023-10-26T06:57:58.385 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2023-10-26T06:57:58.385 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2023-10-26T06:57:58.385 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2023-10-26T06:57:58.386 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2023-10-26T06:57:58.386 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2023-10-26T06:57:58.386 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2023-10-26T06:57:58.386 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2023-10-26T06:57:58.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-10-26T06:57:58.387 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-26T06:57:58.387 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2023-10-26T06:57:58.388 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2023-10-26T06:57:58.388 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2023-10-26T06:57:58.388 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 223, 2023-10-26T06:57:58.388 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 714101, 2023-10-26T06:57:58.389 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2023-10-26T06:57:58.389 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 223, 2023-10-26T06:57:58.389 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2023-10-26T06:57:58.390 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2023-10-26T06:57:58.390 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 223, 2023-10-26T06:57:58.390 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 714101, 2023-10-26T06:57:58.390 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2023-10-26T06:57:58.391 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2023-10-26T06:57:58.391 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2023-10-26T06:57:58.391 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2023-10-26T06:57:58.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:58.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-10-26T06:57:58.392 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-10-26T06:57:58.759 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-10-26T06:57:58.760 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-10-26T06:57:58.760 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-10-26T06:57:58.760 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-10-26T06:57:58.761 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-26T06:57:58.773 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-26T06:57:58.774 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:57:58.774 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T06:57:58.826 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-10-26T06:57:58.826 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T06:57:58.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:57:59.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 -- 172.21.15.89:0/699258745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 msgr2=0x7f7e74100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/699258745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e74100b20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009990 tx=0x7f7e64009ca0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 -- 172.21.15.89:0/699258745 shutdown_connections 2023-10-26T06:57:59.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/699258745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e74101060 0x7f7e7410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/699258745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e741066b0 0x7f7e74106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/699258745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e74100b20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.120 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 -- 172.21.15.89:0/699258745 >> 172.21.15.89:0/699258745 conn(0x7f7e740fc170 msgr2=0x7f7e740fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:59.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.115+0000 7f7e7bfff700 1 -- 172.21.15.89:0/699258745 shutdown_connections 2023-10-26T06:57:59.121 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.116+0000 7f7e7bfff700 1 -- 172.21.15.89:0/699258745 wait complete. 2023-10-26T06:57:59.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.116+0000 7f7e7bfff700 1 Processor -- start 2023-10-26T06:57:59.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7bfff700 1 -- start start 2023-10-26T06:57:59.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e7419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7bfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e74101060 0x7f7e7419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e741066b0 0x7f7e74198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7bfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74199390 con 0x7f7e741006a0 2023-10-26T06:57:59.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7bfff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74199500 con 0x7f7e74101060 2023-10-26T06:57:59.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7bfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741996c0 con 0x7f7e741066b0 2023-10-26T06:57:59.129 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e79d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e7419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e79d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e7419ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60926/0 (socket says 172.21.15.89:60926) 2023-10-26T06:57:59.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7959a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e74101060 0x7f7e7419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e79d9b700 1 -- 172.21.15.89:0/1197897001 learned_addr learned my addr 172.21.15.89:0/1197897001 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:59.130 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.117+0000 7f7e7a59c700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e741066b0 0x7f7e74198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e79d9b700 1 -- 172.21.15.89:0/1197897001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e741066b0 msgr2=0x7f7e74198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e79d9b700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e741066b0 0x7f7e74198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.131 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e79d9b700 1 -- 172.21.15.89:0/1197897001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e74101060 msgr2=0x7f7e7419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e79d9b700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e74101060 0x7f7e7419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e79d9b700 1 -- 172.21.15.89:0/1197897001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009520 con 0x7f7e741006a0 2023-10-26T06:57:59.132 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e79d9b700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e7419ead0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000eb30 tx=0x7f7e7000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:59.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e6affd700 1 -- 172.21.15.89:0/1197897001 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70009880 con 0x7f7e741006a0 2023-10-26T06:57:59.133 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e6affd700 1 -- 172.21.15.89:0/1197897001 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e700044e0 con 0x7f7e741006a0 2023-10-26T06:57:59.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e640095f0 con 0x7f7e741006a0 2023-10-26T06:57:59.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e6affd700 1 -- 172.21.15.89:0/1197897001 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70003d80 con 0x7f7e741006a0 2023-10-26T06:57:59.134 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.118+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e741a5a50 con 0x7f7e741006a0 2023-10-26T06:57:59.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.119+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e74101de0 con 0x7f7e741006a0 2023-10-26T06:57:59.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.121+0000 7f7e6affd700 1 -- 172.21.15.89:0/1197897001 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7e70003fa0 con 0x7f7e741006a0 2023-10-26T06:57:59.135 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.122+0000 7f7e6affd700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e600a1670 0x7f7e600a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.123+0000 7f7e6affd700 1 -- 172.21.15.89:0/1197897001 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f7e70014070 con 0x7f7e741006a0 2023-10-26T06:57:59.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.123+0000 7f7e7959a700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e600a1670 0x7f7e600a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.136 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.123+0000 7f7e7959a700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e600a1670 0x7f7e600a3b30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7e7419a2e0 tx=0x7f7e6400b260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:59.137 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.127+0000 7f7e6affd700 1 -- 172.21.15.89:0/1197897001 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e70096870 con 0x7f7e741006a0 2023-10-26T06:57:59.332 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.330+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e7419a080 con 0x7f7e741006a0 2023-10-26T06:57:59.333 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.331+0000 7f7e6affd700 1 -- 172.21.15.89:0/1197897001 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+22978 (secure 0 0 0) 0x7f7e7419a080 con 0x7f7e741006a0 2023-10-26T06:57:59.334 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:57:59.334 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:57:20.318854+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:57:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e600a1670 msgr2=0x7f7e600a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.336 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e600a1670 0x7f7e600a3b30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7e7419a2e0 tx=0x7f7e6400b260 comp rx=0 tx=0).stop 2023-10-26T06:57:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 msgr2=0x7f7e7419ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e7419ead0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000eb30 tx=0x7f7e7000ee40 comp rx=0 tx=0).stop 2023-10-26T06:57:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 shutdown_connections 2023-10-26T06:57:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f7e600a1670 0x7f7e600a3b30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e741006a0 0x7f7e7419ead0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e741066b0 0x7f7e74198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.338 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 --2- 172.21.15.89:0/1197897001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e74101060 0x7f7e7419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 >> 172.21.15.89:0/1197897001 conn(0x7f7e740fc170 msgr2=0x7f7e740fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:59.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 shutdown_connections 2023-10-26T06:57:59.339 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.334+0000 7f7e7bfff700 1 -- 172.21.15.89:0/1197897001 wait complete. 2023-10-26T06:57:59.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T06:40:50.238940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T06:45:03.596309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T06:45:04.877450+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T06:57:59.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T06:57:59.558 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 -- 172.21.15.89:0/134779508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e0100710 msgr2=0x7fd7e0100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/134779508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e0100710 0x7fd7e0100b90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009990 tx=0x7fd7d4009ca0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 -- 172.21.15.89:0/134779508 shutdown_connections 2023-10-26T06:57:59.559 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/134779508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e01010d0 0x7fd7e010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/134779508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0106720 0x7fd7e0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/134779508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e0100710 0x7fd7e0100b90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.560 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 -- 172.21.15.89:0/134779508 >> 172.21.15.89:0/134779508 conn(0x7fd7e00fc200 msgr2=0x7fd7e00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:59.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 -- 172.21.15.89:0/134779508 shutdown_connections 2023-10-26T06:57:59.561 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.556+0000 7fd7e4dff700 1 -- 172.21.15.89:0/134779508 wait complete. 2023-10-26T06:57:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 Processor -- start 2023-10-26T06:57:59.563 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 -- start start 2023-10-26T06:57:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100710 0x7fd7e0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e01010d0 0x7fd7e01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e0106720 0x7fd7e019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.564 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e019dcc0 con 0x7fd7e01010d0 2023-10-26T06:57:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e019de30 con 0x7fd7e0106720 2023-10-26T06:57:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.557+0000 7fd7e4dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e019dff0 con 0x7fd7e0100710 2023-10-26T06:57:59.565 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7ddd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e01010d0 0x7fd7e01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100710 0x7fd7e0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100710 0x7fd7e0198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:53690/0 (socket says 172.21.15.89:53690) 2023-10-26T06:57:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 -- 172.21.15.89:0/3891289210 learned_addr learned my addr 172.21.15.89:0/3891289210 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:57:59.566 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7ded9d700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e0106720 0x7fd7e019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 -- 172.21.15.89:0/3891289210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e0106720 msgr2=0x7fd7e019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e0106720 0x7fd7e019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 -- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e01010d0 msgr2=0x7fd7e01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.567 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e01010d0 0x7fd7e01992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 -- 172.21.15.89:0/3891289210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d40095f0 con 0x7fd7e0100710 2023-10-26T06:57:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7ded9d700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e0106720 0x7fd7e019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:57:59.568 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7ddd9b700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e01010d0 0x7fd7e01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:57:59.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.558+0000 7fd7de59c700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100710 0x7fd7e0198da0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e9a0 tx=0x7fd7d000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:59.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.559+0000 7fd7cf7fe700 1 -- 172.21.15.89:0/3891289210 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0009790 con 0x7fd7e0100710 2023-10-26T06:57:59.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.559+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e019e2d0 con 0x7fd7e0100710 2023-10-26T06:57:59.569 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.559+0000 7fd7cf7fe700 1 -- 172.21.15.89:0/3891289210 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d000cbd0 con 0x7fd7e0100710 2023-10-26T06:57:59.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.559+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e01a5b10 con 0x7fd7e0100710 2023-10-26T06:57:59.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.559+0000 7fd7cf7fe700 1 -- 172.21.15.89:0/3891289210 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0018680 con 0x7fd7e0100710 2023-10-26T06:57:59.570 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.560+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e0101e50 con 0x7fd7e0100710 2023-10-26T06:57:59.571 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.562+0000 7fd7cf7fe700 1 -- 172.21.15.89:0/3891289210 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd7d0010430 con 0x7fd7e0100710 2023-10-26T06:57:59.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.563+0000 7fd7cf7fe700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7c80a1670 0x7fd7c80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:57:59.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.563+0000 7fd7cf7fe700 1 -- 172.21.15.89:0/3891289210 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7fd7d0014070 con 0x7fd7e0100710 2023-10-26T06:57:59.572 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.563+0000 7fd7ddd9b700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7c80a1670 0x7fd7c80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:57:59.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.564+0000 7fd7ddd9b700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7c80a1670 0x7fd7c80a3b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7e019a2f0 tx=0x7fd7d4005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:57:59.573 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.567+0000 7fd7cf7fe700 1 -- 172.21.15.89:0/3891289210 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd7d0052100 con 0x7fd7e0100710 2023-10-26T06:57:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:59 smithi192 ceph-mon[131661]: pgmap v652: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:59.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:57:59 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1197897001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:57:59.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:59 smithi135 ceph-mon[131403]: pgmap v652: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:59.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:57:59 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1197897001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:57:59.768 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.766+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7e019a0a0 con 0x7fd7e0100710 2023-10-26T06:57:59.769 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.767+0000 7fd7cf7fe700 1 -- 172.21.15.89:0/3891289210 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7d009a040 con 0x7fd7e0100710 2023-10-26T06:57:59.770 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2023-10-26T06:57:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7c80a1670 msgr2=0x7fd7c80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7c80a1670 0x7fd7c80a3b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7e019a2f0 tx=0x7fd7d4005fd0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100710 msgr2=0x7fd7e0198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100710 0x7fd7e0198da0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e9a0 tx=0x7fd7d000ecb0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.773 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 shutdown_connections 2023-10-26T06:57:59.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fd7c80a1670 0x7fd7c80a3b30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7e0106720 0x7fd7e019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.774 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100710 0x7fd7e0198da0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.769+0000 7fd7e4dff700 1 --2- 172.21.15.89:0/3891289210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7e01010d0 0x7fd7e01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.770+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 >> 172.21.15.89:0/3891289210 conn(0x7fd7e00fc200 msgr2=0x7fd7e0072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:59.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.770+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 shutdown_connections 2023-10-26T06:57:59.775 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.770+0000 7fd7e4dff700 1 -- 172.21.15.89:0/3891289210 wait complete. 2023-10-26T06:57:59.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:59 smithi089 ceph-mon[95031]: pgmap v652: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2023-10-26T06:57:59.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:57:59 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1197897001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:57:59.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T06:57:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 -- 172.21.15.89:0/2149894625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df4100720 msgr2=0x7f0df4100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:57:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 --2- 172.21.15.89:0/2149894625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df4100720 0x7f0df4100ba0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009990 tx=0x7f0de0009ca0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 -- 172.21.15.89:0/2149894625 shutdown_connections 2023-10-26T06:57:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 --2- 172.21.15.89:0/2149894625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0df41010e0 0x7f0df410d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 --2- 172.21.15.89:0/2149894625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0df4106730 0x7f0df4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 --2- 172.21.15.89:0/2149894625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df4100720 0x7f0df4100ba0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:57:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 -- 172.21.15.89:0/2149894625 >> 172.21.15.89:0/2149894625 conn(0x7f0df40fc1b0 msgr2=0x7f0df40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:57:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 -- 172.21.15.89:0/2149894625 shutdown_connections 2023-10-26T06:57:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.990+0000 7f0dfabff700 1 -- 172.21.15.89:0/2149894625 wait complete. 2023-10-26T06:57:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.991+0000 7f0dfabff700 1 Processor -- start 2023-10-26T06:57:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0dfabff700 1 -- start start 2023-10-26T06:58:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0dfabff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0df4100720 0x7f0df4072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0dfabff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df41010e0 0x7f0df4073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0dfabff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0df4106730 0x7f0df4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0dfabff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df41a4f70 con 0x7f0df41010e0 2023-10-26T06:58:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0dfabff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df41a50e0 con 0x7f0df4106730 2023-10-26T06:58:00.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0dfabff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df41a52a0 con 0x7f0df4100720 2023-10-26T06:58:00.004 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0df3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df41010e0 0x7f0df4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0df3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df41010e0 0x7f0df4073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55684/0 (socket says 172.21.15.89:55684) 2023-10-26T06:58:00.005 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0df3fff700 1 -- 172.21.15.89:0/1915379529 learned_addr learned my addr 172.21.15.89:0/1915379529 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:00.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0df899b700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0df4100720 0x7f0df4072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.006 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0df919c700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0df4106730 0x7f0df4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.992+0000 7f0df3fff700 1 -- 172.21.15.89:0/1915379529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0df4100720 msgr2=0x7f0df4072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.007 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df3fff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0df4100720 0x7f0df4072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df3fff700 1 -- 172.21.15.89:0/1915379529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0df4106730 msgr2=0x7f0df4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df3fff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0df4106730 0x7f0df4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df3fff700 1 -- 172.21.15.89:0/1915379529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec009520 con 0x7f0df41010e0 2023-10-26T06:58:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df899b700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0df4100720 0x7f0df4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:58:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df3fff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df41010e0 0x7f0df4073030 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df1ffb700 1 -- 172.21.15.89:0/1915379529 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de001d070 con 0x7f0df41010e0 2023-10-26T06:58:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df1ffb700 1 -- 172.21.15.89:0/1915379529 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de0022470 con 0x7f0df41010e0 2023-10-26T06:58:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de00095f0 con 0x7f0df41010e0 2023-10-26T06:58:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0df1ffb700 1 -- 172.21.15.89:0/1915379529 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0017650 con 0x7f0df41010e0 2023-10-26T06:58:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.993+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df41a5810 con 0x7f0df41010e0 2023-10-26T06:58:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.994+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df4108d40 con 0x7f0df41010e0 2023-10-26T06:58:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.996+0000 7f0df1ffb700 1 -- 172.21.15.89:0/1915379529 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0de0022850 con 0x7f0df41010e0 2023-10-26T06:58:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.998+0000 7f0df1ffb700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0de40a15f0 0x7f0de40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.998+0000 7f0df1ffb700 1 -- 172.21.15.89:0/1915379529 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f0de00ce410 con 0x7f0df41010e0 2023-10-26T06:58:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.998+0000 7f0df899b700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0de40a15f0 0x7f0de40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:57:59.999+0000 7f0df899b700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0de40a15f0 0x7f0de40a3ab0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009480 tx=0x7f0de800b040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.001+0000 7f0df1ffb700 1 -- 172.21.15.89:0/1915379529 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0de005c100 con 0x7f0df41010e0 2023-10-26T06:58:00.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.204+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0df4073fd0 con 0x7f0df41010e0 2023-10-26T06:58:00.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.204+0000 7f0df1ffb700 1 -- 172.21.15.89:0/1915379529 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0de0097770 con 0x7f0df41010e0 2023-10-26T06:58:00.208 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2023-10-26T06:58:00.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0de40a15f0 msgr2=0x7f0de40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0de40a15f0 0x7f0de40a3ab0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009480 tx=0x7f0de800b040 comp rx=0 tx=0).stop 2023-10-26T06:58:00.210 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df41010e0 msgr2=0x7f0df4073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df41010e0 0x7f0df4073030 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de0005740 comp rx=0 tx=0).stop 2023-10-26T06:58:00.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 shutdown_connections 2023-10-26T06:58:00.212 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f0de40a15f0 0x7f0de40a3ab0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.213 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0df4106730 0x7f0df4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.214 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0df4100720 0x7f0df4072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 --2- 172.21.15.89:0/1915379529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0df41010e0 0x7f0df4073030 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.215 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 >> 172.21.15.89:0/1915379529 conn(0x7f0df40fc1b0 msgr2=0x7f0df40fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:00.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 shutdown_connections 2023-10-26T06:58:00.216 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.207+0000 7f0dfabff700 1 -- 172.21.15.89:0/1915379529 wait complete. 2023-10-26T06:58:00.228 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T06:58:00.455 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 -- 172.21.15.89:0/1111764553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9298106730 msgr2=0x7f9298106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 --2- 172.21.15.89:0/1111764553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9298106730 0x7f9298106b10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9288009990 tx=0x7f9288009ca0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 -- 172.21.15.89:0/1111764553 shutdown_connections 2023-10-26T06:58:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 --2- 172.21.15.89:0/1111764553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92981010e0 0x7f929810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 --2- 172.21.15.89:0/1111764553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9298106730 0x7f9298106b10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 --2- 172.21.15.89:0/1111764553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9298100720 0x7f9298100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 -- 172.21.15.89:0/1111764553 >> 172.21.15.89:0/1111764553 conn(0x7f92980fc1b0 msgr2=0x7f92980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 -- 172.21.15.89:0/1111764553 shutdown_connections 2023-10-26T06:58:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.453+0000 7f929e3ff700 1 -- 172.21.15.89:0/1111764553 wait complete. 2023-10-26T06:58:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.454+0000 7f929e3ff700 1 Processor -- start 2023-10-26T06:58:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.454+0000 7f929e3ff700 1 -- start start 2023-10-26T06:58:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f929e3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9298100720 0x7f9298072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f929e3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92981010e0 0x7f9298073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f929e3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9298106730 0x7f9298073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f929e3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92981a4f10 con 0x7f9298106730 2023-10-26T06:58:00.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f929e3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92981a5080 con 0x7f92981010e0 2023-10-26T06:58:00.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f929e3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92981a5240 con 0x7f9298100720 2023-10-26T06:58:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9298100720 0x7f9298072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f929c99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9298106730 0x7f9298073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9298100720 0x7f9298072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33590/0 (socket says 172.21.15.89:33590) 2023-10-26T06:58:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 -- 172.21.15.89:0/3786811356 learned_addr learned my addr 172.21.15.89:0/3786811356 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f92977fe700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92981010e0 0x7f9298073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 -- 172.21.15.89:0/3786811356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92981010e0 msgr2=0x7f9298073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92981010e0 0x7f9298073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 -- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9298106730 msgr2=0x7f9298073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9298106730 0x7f9298073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.455+0000 7f9297fff700 1 -- 172.21.15.89:0/3786811356 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92880095f0 con 0x7f9298100720 2023-10-26T06:58:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.456+0000 7f9297fff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9298100720 0x7f9298072af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9288005910 tx=0x7f9288004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.456+0000 7f92957fa700 1 -- 172.21.15.89:0/3786811356 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928801d070 con 0x7f9298100720 2023-10-26T06:58:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.456+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92981a54c0 con 0x7f9298100720 2023-10-26T06:58:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.456+0000 7f92957fa700 1 -- 172.21.15.89:0/3786811356 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9288022470 con 0x7f9298100720 2023-10-26T06:58:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.456+0000 7f92957fa700 1 -- 172.21.15.89:0/3786811356 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288017650 con 0x7f9298100720 2023-10-26T06:58:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.456+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92981a59f0 con 0x7f9298100720 2023-10-26T06:58:00.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.457+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9298108cb0 con 0x7f9298100720 2023-10-26T06:58:00.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.459+0000 7f92957fa700 1 -- 172.21.15.89:0/3786811356 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f92880225e0 con 0x7f9298100720 2023-10-26T06:58:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.460+0000 7f92957fa700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f92800a15f0 0x7f92800a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.460+0000 7f92957fa700 1 -- 172.21.15.89:0/3786811356 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f92880ce370 con 0x7f9298100720 2023-10-26T06:58:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.461+0000 7f92977fe700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f92800a15f0 0x7f92800a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.461+0000 7f92977fe700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f92800a15f0 0x7f92800a3ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9298074220 tx=0x7f928c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:00.474 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.462+0000 7f92957fa700 1 -- 172.21.15.89:0/3786811356 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f92880976d0 con 0x7f9298100720 2023-10-26T06:58:00.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:00 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3891289210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:58:00.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:00 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/1915379529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:58:00.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:00 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3891289210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:58:00.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:00 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/1915379529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:58:00.667 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.665+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9298101e60 con 0x7f9298100720 2023-10-26T06:58:00.668 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.666+0000 7f92957fa700 1 -- 172.21.15.89:0/3786811356 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f928809acf0 con 0x7f9298100720 2023-10-26T06:58:00.668 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2023-10-26T06:58:00.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f92800a15f0 msgr2=0x7f92800a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f92800a15f0 0x7f92800a3ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9298074220 tx=0x7f928c009260 comp rx=0 tx=0).stop 2023-10-26T06:58:00.671 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9298100720 msgr2=0x7f9298072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9298100720 0x7f9298072af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9288005910 tx=0x7f9288004730 comp rx=0 tx=0).stop 2023-10-26T06:58:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 shutdown_connections 2023-10-26T06:58:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f92800a15f0 0x7f92800a3ab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.672 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9298106730 0x7f9298073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9298100720 0x7f9298072af0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 --2- 172.21.15.89:0/3786811356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92981010e0 0x7f9298073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.673 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 >> 172.21.15.89:0/3786811356 conn(0x7f92980fc1b0 msgr2=0x7f92980fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 shutdown_connections 2023-10-26T06:58:00.674 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.668+0000 7f929e3ff700 1 -- 172.21.15.89:0/3786811356 wait complete. 2023-10-26T06:58:00.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T06:58:00.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:00 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3891289210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-26T06:58:00.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:00 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/1915379529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-26T06:58:00.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 -- 172.21.15.89:0/2431517217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 msgr2=0x7f4ac8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.889 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 --2- 172.21.15.89:0/2431517217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 0x7f4ac8106a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009990 tx=0x7f4abc009ca0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 -- 172.21.15.89:0/2431517217 shutdown_connections 2023-10-26T06:58:00.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 --2- 172.21.15.89:0/2431517217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac8101060 0x7f4ac810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.890 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 --2- 172.21.15.89:0/2431517217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 0x7f4ac8106a90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 --2- 172.21.15.89:0/2431517217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac81006a0 0x7f4ac8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 -- 172.21.15.89:0/2431517217 >> 172.21.15.89:0/2431517217 conn(0x7f4ac80fc170 msgr2=0x7f4ac80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:00.891 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.886+0000 7f4acedff700 1 -- 172.21.15.89:0/2431517217 shutdown_connections 2023-10-26T06:58:00.892 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.887+0000 7f4acedff700 1 -- 172.21.15.89:0/2431517217 wait complete. 2023-10-26T06:58:00.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.887+0000 7f4acedff700 1 Processor -- start 2023-10-26T06:58:00.895 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acedff700 1 -- start start 2023-10-26T06:58:00.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acedff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac81006a0 0x7f4ac819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acedff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac8101060 0x7f4ac819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.896 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acedff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 0x7f4ac8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acedff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8199100 con 0x7f4ac81006a0 2023-10-26T06:58:00.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acedff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8199270 con 0x7f4ac81066b0 2023-10-26T06:58:00.897 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acedff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8199430 con 0x7f4ac8101060 2023-10-26T06:58:00.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4accb9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac81006a0 0x7f4ac819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4accb9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac81006a0 0x7f4ac819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55718/0 (socket says 172.21.15.89:55718) 2023-10-26T06:58:00.898 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4accb9b700 1 -- 172.21.15.89:0/4095765768 learned_addr learned my addr 172.21.15.89:0/4095765768 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:00.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4ac7fff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac8101060 0x7f4ac819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.888+0000 7f4acd39c700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 0x7f4ac8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.899 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4accb9b700 1 -- 172.21.15.89:0/4095765768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac8101060 msgr2=0x7f4ac819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4accb9b700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac8101060 0x7f4ac819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4accb9b700 1 -- 172.21.15.89:0/4095765768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 msgr2=0x7f4ac8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:00.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4accb9b700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 0x7f4ac8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:00.900 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4accb9b700 1 -- 172.21.15.89:0/4095765768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc0095f0 con 0x7f4ac81006a0 2023-10-26T06:58:00.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4acd39c700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 0x7f4ac8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T06:58:00.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4accb9b700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac81006a0 0x7f4ac819e970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005770 tx=0x7f4abc0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:00.901 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4ac5ffb700 1 -- 172.21.15.89:0/4095765768 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc01d070 con 0x7f4ac81006a0 2023-10-26T06:58:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac8199710 con 0x7f4ac81006a0 2023-10-26T06:58:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4ac5ffb700 1 -- 172.21.15.89:0/4095765768 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4abc021410 con 0x7f4ac81006a0 2023-10-26T06:58:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.889+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac81a5960 con 0x7f4ac81006a0 2023-10-26T06:58:00.902 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.890+0000 7f4ac5ffb700 1 -- 172.21.15.89:0/4095765768 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc02b640 con 0x7f4ac81006a0 2023-10-26T06:58:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.890+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac8101de0 con 0x7f4ac81006a0 2023-10-26T06:58:00.903 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.892+0000 7f4ac5ffb700 1 -- 172.21.15.89:0/4095765768 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4abc0217f0 con 0x7f4ac81006a0 2023-10-26T06:58:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.894+0000 7f4ac5ffb700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ab80a15f0 0x7f4ab80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:00.904 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.895+0000 7f4ac5ffb700 1 -- 172.21.15.89:0/4095765768 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f4abc0ce130 con 0x7f4ac81006a0 2023-10-26T06:58:00.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.895+0000 7f4ac7fff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ab80a15f0 0x7f4ab80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:00.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.895+0000 7f4ac7fff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ab80a15f0 0x7f4ab80a3ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8199c60 tx=0x7f4ab4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:00.905 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:00.899+0000 7f4ac5ffb700 1 -- 172.21.15.89:0/4095765768 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4abc05c100 con 0x7f4ac81006a0 2023-10-26T06:58:01.118 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.116+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ac8067530 con 0x7f4ac81006a0 2023-10-26T06:58:01.119 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.116+0000 7f4ac5ffb700 1 -- 172.21.15.89:0/4095765768 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v96) v1 ==== 81+0+882 (secure 0 0 0) 0x7f4abc097490 con 0x7f4ac81006a0 2023-10-26T06:58:01.119 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/692487046 2023-10-27T06:39:01.639977+0000 2023-10-26T06:58:01.120 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1796653810 2023-10-27T06:37:50.043554+0000 2023-10-26T06:58:01.120 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2906183572 2023-10-27T06:37:50.043554+0000 2023-10-26T06:58:01.120 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/925229622 2023-10-27T06:37:50.043554+0000 2023-10-26T06:58:01.121 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/4177523596 2023-10-27T06:39:01.639977+0000 2023-10-26T06:58:01.121 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/2105103929 2023-10-27T06:37:23.538291+0000 2023-10-26T06:58:01.121 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/746092036 2023-10-27T06:39:01.639977+0000 2023-10-26T06:58:01.121 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/2105103929 2023-10-27T06:37:23.538291+0000 2023-10-26T06:58:01.122 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1900326324 2023-10-27T06:37:23.538291+0000 2023-10-26T06:58:01.122 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/692487046 2023-10-27T06:39:01.639977+0000 2023-10-26T06:58:01.122 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2757333567 2023-10-27T06:37:50.043554+0000 2023-10-26T06:58:01.123 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/516070855 2023-10-27T06:37:23.538291+0000 2023-10-26T06:58:01.123 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2016135884 2023-10-27T06:39:01.639977+0000 2023-10-26T06:58:01.123 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2689379544 2023-10-27T06:37:23.538291+0000 2023-10-26T06:58:01.124 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/925229622 2023-10-27T06:37:50.043554+0000 2023-10-26T06:58:01.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ab80a15f0 msgr2=0x7f4ab80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:01.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ab80a15f0 0x7f4ab80a3ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8199c60 tx=0x7f4ab4009260 comp rx=0 tx=0).stop 2023-10-26T06:58:01.125 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac81006a0 msgr2=0x7f4ac819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac81006a0 0x7f4ac819e970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005770 tx=0x7f4abc0057a0 comp rx=0 tx=0).stop 2023-10-26T06:58:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 shutdown_connections 2023-10-26T06:58:01.126 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f4ab80a15f0 0x7f4ab80a3ab0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ac81006a0 0x7f4ac819e970 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ac81066b0 0x7f4ac8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 --2- 172.21.15.89:0/4095765768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac8101060 0x7f4ac819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:01.127 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 >> 172.21.15.89:0/4095765768 conn(0x7f4ac80fc170 msgr2=0x7f4ac80fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 shutdown_connections 2023-10-26T06:58:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:01.119+0000 7f4acedff700 1 -- 172.21.15.89:0/4095765768 wait complete. 2023-10-26T06:58:01.128 INFO:teuthology.orchestra.run.smithi089.stderr:listed 15 entries 2023-10-26T06:58:01.140 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T06:58:01.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:58:01.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T06:58:01.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:01 smithi089 ceph-mon[95031]: pgmap v653: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:58:01.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:01 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3786811356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:58:01.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:01 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4095765768' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-26T06:58:01.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:01 smithi192 ceph-mon[131661]: pgmap v653: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:58:01.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:01 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3786811356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:58:01.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:01 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4095765768' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-26T06:58:01.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:01 smithi135 ceph-mon[131403]: pgmap v653: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2023-10-26T06:58:01.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:01 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3786811356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-26T06:58:01.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:01 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4095765768' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-26T06:58:03.443 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T06:58:03.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:58:03.444 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T06:58:03.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:58:03.544 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2023-10-26T06:58:03.589 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T06:58:03.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:58:03.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T06:58:03.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:03 smithi192 ceph-mon[131661]: pgmap v654: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2023-10-26T06:58:03.689 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:03 smithi089 ceph-mon[95031]: pgmap v654: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2023-10-26T06:58:03.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T06:58:03.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2023-10-26T06:58:03.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:03 smithi135 ceph-mon[131403]: pgmap v654: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2023-10-26T06:58:03.820 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-26T06:58:03.831 INFO:tasks.cephadm:Teardown begin 2023-10-26T06:58:03.832 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T06:58:03.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:58:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 -- 172.21.15.89:0/271340633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 msgr2=0x7f1a74106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 --2- 172.21.15.89:0/271340633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a74106a90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009990 tx=0x7f1a64009ca0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 -- 172.21.15.89:0/271340633 shutdown_connections 2023-10-26T06:58:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 --2- 172.21.15.89:0/271340633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a74106a90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 --2- 172.21.15.89:0/271340633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a74101060 0x7f1a7410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 --2- 172.21.15.89:0/271340633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a741006a0 0x7f1a74100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 -- 172.21.15.89:0/271340633 >> 172.21.15.89:0/271340633 conn(0x7f1a740fc1d0 msgr2=0x7f1a740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 -- 172.21.15.89:0/271340633 shutdown_connections 2023-10-26T06:58:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.035+0000 7f1a797ff700 1 -- 172.21.15.89:0/271340633 wait complete. 2023-10-26T06:58:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.036+0000 7f1a797ff700 1 Processor -- start 2023-10-26T06:58:04.042 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.036+0000 7f1a797ff700 1 -- start start 2023-10-26T06:58:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a797ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a741006a0 0x7f1a74198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a797ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a74101060 0x7f1a74199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a797ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a7419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.043 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a797ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419dc20 con 0x7f1a741066b0 2023-10-26T06:58:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a797ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419dd90 con 0x7f1a74101060 2023-10-26T06:58:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a797ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419df50 con 0x7f1a741006a0 2023-10-26T06:58:04.046 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a7419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a72ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a741006a0 0x7f1a74198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a737fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a7419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55738/0 (socket says 172.21.15.89:55738) 2023-10-26T06:58:04.047 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a737fe700 1 -- 172.21.15.89:0/4008312691 learned_addr learned my addr 172.21.15.89:0/4008312691 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a6bfff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a74101060 0x7f1a74199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a72ffd700 1 -- 172.21.15.89:0/4008312691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a74101060 msgr2=0x7f1a74199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a72ffd700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a74101060 0x7f1a74199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.048 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a72ffd700 1 -- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 msgr2=0x7f1a7419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a72ffd700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a7419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a72ffd700 1 -- 172.21.15.89:0/4008312691 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a640095f0 con 0x7f1a741006a0 2023-10-26T06:58:04.049 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.037+0000 7f1a737fe700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a7419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:58:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.038+0000 7f1a72ffd700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a741006a0 0x7f1a74198df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1a640098e0 tx=0x7f1a64005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.038+0000 7f1a70ff9700 1 -- 172.21.15.89:0/4008312691 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6401d070 con 0x7f1a741006a0 2023-10-26T06:58:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.038+0000 7f1a70ff9700 1 -- 172.21.15.89:0/4008312691 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a64022470 con 0x7f1a741006a0 2023-10-26T06:58:04.050 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.038+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7419e1d0 con 0x7f1a741006a0 2023-10-26T06:58:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.038+0000 7f1a70ff9700 1 -- 172.21.15.89:0/4008312691 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64017650 con 0x7f1a741006a0 2023-10-26T06:58:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.038+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a741a5a70 con 0x7f1a741006a0 2023-10-26T06:58:04.051 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.040+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a74101de0 con 0x7f1a741006a0 2023-10-26T06:58:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.041+0000 7f1a70ff9700 1 -- 172.21.15.89:0/4008312691 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1a640225e0 con 0x7f1a741006a0 2023-10-26T06:58:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.043+0000 7f1a70ff9700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1a600a15f0 0x7f1a600a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.043+0000 7f1a70ff9700 1 -- 172.21.15.89:0/4008312691 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8499+0+0 (secure 0 0 0) 0x7f1a640ce3a0 con 0x7f1a741006a0 2023-10-26T06:58:04.053 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.043+0000 7f1a6bfff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1a600a15f0 0x7f1a600a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.043+0000 7f1a6bfff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1a600a15f0 0x7f1a600a3ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a7419a2f0 tx=0x7f1a5c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:04.054 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.047+0000 7f1a70ff9700 1 -- 172.21.15.89:0/4008312691 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1a64097760 con 0x7f1a741006a0 2023-10-26T06:58:04.298 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.296+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a74067530 con 0x7f1a741006a0 2023-10-26T06:58:04.299 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.298+0000 7f1a70ff9700 1 -- 172.21.15.89:0/4008312691 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f1a6409ad80 con 0x7f1a741006a0 2023-10-26T06:58:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:58:04.300 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:58:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1a600a15f0 msgr2=0x7f1a600a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1a600a15f0 0x7f1a600a3ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a7419a2f0 tx=0x7f1a5c009260 comp rx=0 tx=0).stop 2023-10-26T06:58:04.302 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a741006a0 msgr2=0x7f1a74198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a741006a0 0x7f1a74198df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1a640098e0 tx=0x7f1a64005740 comp rx=0 tx=0).stop 2023-10-26T06:58:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 shutdown_connections 2023-10-26T06:58:04.303 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f1a600a15f0 0x7f1a600a3ab0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a741066b0 0x7f1a7419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a741006a0 0x7f1a74198df0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.304 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 --2- 172.21.15.89:0/4008312691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a74101060 0x7f1a74199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 >> 172.21.15.89:0/4008312691 conn(0x7f1a740fc1d0 msgr2=0x7f1a740fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 shutdown_connections 2023-10-26T06:58:04.305 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.300+0000 7f1a797ff700 1 -- 172.21.15.89:0/4008312691 wait complete. 2023-10-26T06:58:04.306 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:58:04.321 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T06:58:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.521+0000 7f8acf9ff700 1 -- 172.21.15.89:0/3146508965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac8101060 msgr2=0x7f8ac810d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.524 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.521+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/3146508965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac8101060 0x7f8ac810d570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000b0a0 tx=0x7f8ac000b3b0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.522+0000 7f8acf9ff700 1 -- 172.21.15.89:0/3146508965 shutdown_connections 2023-10-26T06:58:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.522+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/3146508965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac8101060 0x7f8ac810d570 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.525 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.522+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/3146508965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ac81066b0 0x7f8ac8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.522+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/3146508965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ac81006a0 0x7f8ac8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.522+0000 7f8acf9ff700 1 -- 172.21.15.89:0/3146508965 >> 172.21.15.89:0/3146508965 conn(0x7f8ac80fc1d0 msgr2=0x7f8ac80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.522+0000 7f8acf9ff700 1 -- 172.21.15.89:0/3146508965 shutdown_connections 2023-10-26T06:58:04.526 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.522+0000 7f8acf9ff700 1 -- 172.21.15.89:0/3146508965 wait complete. 2023-10-26T06:58:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 Processor -- start 2023-10-26T06:58:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 -- start start 2023-10-26T06:58:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac81006a0 0x7f8ac819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ac8101060 0x7f8ac819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ac81066b0 0x7f8ac8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac81992c0 con 0x7f8ac81006a0 2023-10-26T06:58:04.530 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac8199430 con 0x7f8ac81066b0 2023-10-26T06:58:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acf9ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac81995f0 con 0x7f8ac8101060 2023-10-26T06:58:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acd79b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac81006a0 0x7f8ac819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8accf9a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ac8101060 0x7f8ac819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acd79b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac81006a0 0x7f8ac819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55760/0 (socket says 172.21.15.89:55760) 2023-10-26T06:58:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acd79b700 1 -- 172.21.15.89:0/4293138226 learned_addr learned my addr 172.21.15.89:0/4293138226 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.523+0000 7f8acdf9c700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ac81066b0 0x7f8ac8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8accf9a700 1 -- 172.21.15.89:0/4293138226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ac81066b0 msgr2=0x7f8ac8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8accf9a700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ac81066b0 0x7f8ac8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8accf9a700 1 -- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac81006a0 msgr2=0x7f8ac819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8accf9a700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac81006a0 0x7f8ac819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8accf9a700 1 -- 172.21.15.89:0/4293138226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0009d00 con 0x7f8ac8101060 2023-10-26T06:58:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8acd79b700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac81006a0 0x7f8ac819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:58:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8accf9a700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ac8101060 0x7f8ac819ef90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8abe7fc700 1 -- 172.21.15.89:0/4293138226 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800c9c0 con 0x7f8ac8101060 2023-10-26T06:58:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac81998d0 con 0x7f8ac8101060 2023-10-26T06:58:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8abe7fc700 1 -- 172.21.15.89:0/4293138226 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab8010430 con 0x7f8ac8101060 2023-10-26T06:58:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.524+0000 7f8abe7fc700 1 -- 172.21.15.89:0/4293138226 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8018680 con 0x7f8ac8101060 2023-10-26T06:58:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.525+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac81a5a20 con 0x7f8ac8101060 2023-10-26T06:58:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.525+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac8101de0 con 0x7f8ac8101060 2023-10-26T06:58:04.536 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.527+0000 7f8abe7fc700 1 -- 172.21.15.89:0/4293138226 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8ab8010860 con 0x7f8ac8101060 2023-10-26T06:58:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.529+0000 7f8abe7fc700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8ab40a15f0 0x7f8ab40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.529+0000 7f8abe7fc700 1 -- 172.21.15.89:0/4293138226 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f8ab8014070 con 0x7f8ac8101060 2023-10-26T06:58:04.537 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.529+0000 7f8acd79b700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8ab40a15f0 0x7f8ab40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.529+0000 7f8acd79b700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8ab40a15f0 0x7f8ab40a3ab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4011550 tx=0x7f8ac4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:04.538 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.531+0000 7f8abe7fc700 1 -- 172.21.15.89:0/4293138226 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8ab80966f0 con 0x7f8ac8101060 2023-10-26T06:58:04.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:04 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4008312691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:58:04.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:04 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4008312691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:58:04.782 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.780+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ac819a030 con 0x7f8ac8101060 2023-10-26T06:58:04.784 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.782+0000 7f8abe7fc700 1 -- 172.21.15.89:0/4293138226 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11271 (secure 0 0 0) 0x7f8ab8099d10 con 0x7f8ac8101060 2023-10-26T06:58:04.784 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:58:04.784 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/229884175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":229884175},{"type":"v1","addr":"172.21.15.89:6835","nonce":229884175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/1053435135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":1053435135},{"type":"v1","addr":"172.21.15.89:6839","nonce":1053435135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3761987618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3761987618},{"type":"v1","addr":"172.21.15.89:6841","nonce":3761987618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/1842410141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1842410141},{"type":"v1","addr":"172.21.15.192:6833","nonce":1842410141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24635,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3774068069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3774068069},{"type":"v1","addr":"172.21.15.135:6833","nonce":3774068069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/104209914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":104209914},{"type":"v1","addr":"172.21.15.89:6837","nonce":104209914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24671,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/4067855863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":4067855863},{"type":"v1","addr":"172.21.15.135:6835","nonce":4067855863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24676,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/2073006190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2073006190},{"type":"v1","addr":"172.21.15.192:6835","nonce":2073006190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1332439881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1332439881},{"type":"v1","addr":"172.21.15.192:6837","nonce":1332439881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24692,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1484649325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1484649325},{"type":"v1","addr":"172.21.15.135:6837","nonce":1484649325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-26T06:45:05.445103+0000","modified":"2023-10-26T06:45:09.558391+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24703":{"gid":24703,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6839/2442523019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2442523019},{"type":"v1","addr":"172.21.15.192:6839","nonce":2442523019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/609246385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":609246385},{"type":"v1","addr":"172.21.15.135:6839","nonce":609246385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T06:58:04.786 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8ab40a15f0 msgr2=0x7f8ab40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8ab40a15f0 0x7f8ab40a3ab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4011550 tx=0x7f8ac4009260 comp rx=0 tx=0).stop 2023-10-26T06:58:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ac8101060 msgr2=0x7f8ac819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ac8101060 0x7f8ac819ef90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800ed00 comp rx=0 tx=0).stop 2023-10-26T06:58:04.787 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 shutdown_connections 2023-10-26T06:58:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f8ab40a15f0 0x7f8ab40a3ab0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ac81006a0 0x7f8ac819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.788 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ac81066b0 0x7f8ac8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 --2- 172.21.15.89:0/4293138226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ac8101060 0x7f8ac819ef90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:04.789 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 >> 172.21.15.89:0/4293138226 conn(0x7f8ac80fc1d0 msgr2=0x7f8ac810bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:04.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 shutdown_connections 2023-10-26T06:58:04.790 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:04.785+0000 7f8acf9ff700 1 -- 172.21.15.89:0/4293138226 wait complete. 2023-10-26T06:58:04.791 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 20 2023-10-26T06:58:04.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:04 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4008312691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:58:04.807 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T06:58:05.011 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 -- 172.21.15.89:0/41075969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 msgr2=0x7f7308100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 --2- 172.21.15.89:0/41075969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 0x7f7308100b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009990 tx=0x7f72f8009ca0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 -- 172.21.15.89:0/41075969 shutdown_connections 2023-10-26T06:58:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 --2- 172.21.15.89:0/41075969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 0x7f730810d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 --2- 172.21.15.89:0/41075969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73081066b0 0x7f7308106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 --2- 172.21.15.89:0/41075969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 0x7f7308100b20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 -- 172.21.15.89:0/41075969 >> 172.21.15.89:0/41075969 conn(0x7f73080fc170 msgr2=0x7f73080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 -- 172.21.15.89:0/41075969 shutdown_connections 2023-10-26T06:58:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.008+0000 7f730ffff700 1 -- 172.21.15.89:0/41075969 wait complete. 2023-10-26T06:58:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.009+0000 7f730ffff700 1 Processor -- start 2023-10-26T06:58:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730ffff700 1 -- start start 2023-10-26T06:58:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 0x7f7308198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 0x7f7308199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730ffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73081066b0 0x7f730819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730ffff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819cb50 con 0x7f7308101060 2023-10-26T06:58:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730ffff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819ccc0 con 0x7f73081006a0 2023-10-26T06:58:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730ffff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819ce80 con 0x7f73081066b0 2023-10-26T06:58:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730d59a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 0x7f7308199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730d59a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 0x7f7308199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55778/0 (socket says 172.21.15.89:55778) 2023-10-26T06:58:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730d59a700 1 -- 172.21.15.89:0/27254983 learned_addr learned my addr 172.21.15.89:0/27254983 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730e59c700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73081066b0 0x7f730819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.010+0000 7f730dd9b700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 0x7f7308198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730d59a700 1 -- 172.21.15.89:0/27254983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73081066b0 msgr2=0x7f730819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730d59a700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73081066b0 0x7f730819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730d59a700 1 -- 172.21.15.89:0/27254983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 msgr2=0x7f7308198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730d59a700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 0x7f7308198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730d59a700 1 -- 172.21.15.89:0/27254983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7300009c40 con 0x7f7308101060 2023-10-26T06:58:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730e59c700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73081066b0 0x7f730819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:58:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730d59a700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 0x7f7308199260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f72f8000c00 tx=0x7f72f8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f72feffd700 1 -- 172.21.15.89:0/27254983 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f801d070 con 0x7f7308101060 2023-10-26T06:58:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f72feffd700 1 -- 172.21.15.89:0/27254983 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72f8022470 con 0x7f7308101060 2023-10-26T06:58:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f80095f0 con 0x7f7308101060 2023-10-26T06:58:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f72feffd700 1 -- 172.21.15.89:0/27254983 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8017650 con 0x7f7308101060 2023-10-26T06:58:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.011+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73081a58e0 con 0x7f7308101060 2023-10-26T06:58:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.012+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7308101de0 con 0x7f7308101060 2023-10-26T06:58:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.014+0000 7f72feffd700 1 -- 172.21.15.89:0/27254983 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f72f80225e0 con 0x7f7308101060 2023-10-26T06:58:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.015+0000 7f72feffd700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72f40a15f0 0x7f72f40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.016+0000 7f72feffd700 1 -- 172.21.15.89:0/27254983 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8543+0+0 (secure 0 0 0) 0x7f72f80ce3d0 con 0x7f7308101060 2023-10-26T06:58:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.016+0000 7f730dd9b700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72f40a15f0 0x7f72f40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.016+0000 7f730dd9b700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72f40a15f0 0x7f72f40a3ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7304009be0 tx=0x7f7304009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.020+0000 7f72feffd700 1 -- 172.21.15.89:0/27254983 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f72f8097710 con 0x7f7308101060 2023-10-26T06:58:05.230 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.229+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7308110670 con 0x7f7308101060 2023-10-26T06:58:05.232 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.231+0000 7f72feffd700 1 -- 172.21.15.89:0/27254983 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+23045 (secure 0 0 0) 0x7f72f809ad30 con 0x7f7308101060 2023-10-26T06:58:05.232 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-26T06:58:05.233 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":97,"fsid":"e22b2d1a-73c9-11ee-8db9-212e2dc638e7","created":"2023-10-26T06:36:45.901565+0000","modified":"2023-10-26T06:58:04.372425+0000","last_up_change":"2023-10-26T06:44:17.769926+0000","last_in_change":"2023-10-26T06:43:59.757115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T06:40:50.238940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T06:45:03.596309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T06:45:04.877450+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54028a78-a197-4bc3-b38b-f3e792bb42b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6803","nonce":2463649651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6805","nonce":2463649651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6809","nonce":2463649651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2463649651},{"type":"v1","addr":"172.21.15.89:6807","nonce":2463649651}]},"public_addr":"172.21.15.89:6803/2463649651","cluster_addr":"172.21.15.89:6805/2463649651","heartbeat_back_addr":"172.21.15.89:6809/2463649651","heartbeat_front_addr":"172.21.15.89:6807/2463649651","state":["exists","up"]},{"osd":1,"uuid":"e7c5f2af-c64a-4c23-9fdb-34d8e1c0c6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6801","nonce":77037565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6803","nonce":77037565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6807","nonce":77037565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":77037565},{"type":"v1","addr":"172.21.15.135:6805","nonce":77037565}]},"public_addr":"172.21.15.135:6801/77037565","cluster_addr":"172.21.15.135:6803/77037565","heartbeat_back_addr":"172.21.15.135:6807/77037565","heartbeat_front_addr":"172.21.15.135:6805/77037565","state":["exists","up"]},{"osd":2,"uuid":"f05df73a-080f-410a-84ad-50df3f2717f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6801","nonce":2357909960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6803","nonce":2357909960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6807","nonce":2357909960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":2357909960},{"type":"v1","addr":"172.21.15.192:6805","nonce":2357909960}]},"public_addr":"172.21.15.192:6801/2357909960","cluster_addr":"172.21.15.192:6803/2357909960","heartbeat_back_addr":"172.21.15.192:6807/2357909960","heartbeat_front_addr":"172.21.15.192:6805/2357909960","state":["exists","up"]},{"osd":3,"uuid":"4b4e4079-2075-4d49-9a5b-457e210aa444","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6811","nonce":1365528390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6813","nonce":1365528390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6817","nonce":1365528390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1365528390},{"type":"v1","addr":"172.21.15.89:6815","nonce":1365528390}]},"public_addr":"172.21.15.89:6811/1365528390","cluster_addr":"172.21.15.89:6813/1365528390","heartbeat_back_addr":"172.21.15.89:6817/1365528390","heartbeat_front_addr":"172.21.15.89:6815/1365528390","state":["exists","up"]},{"osd":4,"uuid":"391e09b6-9475-4b8e-914f-34ed06f36e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6809","nonce":3181790377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6811","nonce":3181790377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6815","nonce":3181790377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3181790377},{"type":"v1","addr":"172.21.15.135:6813","nonce":3181790377}]},"public_addr":"172.21.15.135:6809/3181790377","cluster_addr":"172.21.15.135:6811/3181790377","heartbeat_back_addr":"172.21.15.135:6815/3181790377","heartbeat_front_addr":"172.21.15.135:6813/3181790377","state":["exists","up"]},{"osd":5,"uuid":"529984c4-0e50-4d78-b4ba-572a2f7df23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6809","nonce":2176647372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6811","nonce":2176647372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6815","nonce":2176647372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2176647372},{"type":"v1","addr":"172.21.15.192:6813","nonce":2176647372}]},"public_addr":"172.21.15.192:6809/2176647372","cluster_addr":"172.21.15.192:6811/2176647372","heartbeat_back_addr":"172.21.15.192:6815/2176647372","heartbeat_front_addr":"172.21.15.192:6813/2176647372","state":["exists","up"]},{"osd":6,"uuid":"0a0f5c79-d3fe-41e8-8ca4-bb2b3713d984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6819","nonce":2393166048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6821","nonce":2393166048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6825","nonce":2393166048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2393166048},{"type":"v1","addr":"172.21.15.89:6823","nonce":2393166048}]},"public_addr":"172.21.15.89:6819/2393166048","cluster_addr":"172.21.15.89:6821/2393166048","heartbeat_back_addr":"172.21.15.89:6825/2393166048","heartbeat_front_addr":"172.21.15.89:6823/2393166048","state":["exists","up"]},{"osd":7,"uuid":"d9fc3b27-d54b-49e6-accb-8e02a7782a6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6817","nonce":1440944869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6819","nonce":1440944869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6823","nonce":1440944869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1440944869},{"type":"v1","addr":"172.21.15.135:6821","nonce":1440944869}]},"public_addr":"172.21.15.135:6817/1440944869","cluster_addr":"172.21.15.135:6819/1440944869","heartbeat_back_addr":"172.21.15.135:6823/1440944869","heartbeat_front_addr":"172.21.15.135:6821/1440944869","state":["exists","up"]},{"osd":8,"uuid":"809944cc-518e-49bc-b6fb-b56823f81148","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6817","nonce":87624122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6819","nonce":87624122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6823","nonce":87624122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":87624122},{"type":"v1","addr":"172.21.15.192:6821","nonce":87624122}]},"public_addr":"172.21.15.192:6817/87624122","cluster_addr":"172.21.15.192:6819/87624122","heartbeat_back_addr":"172.21.15.192:6823/87624122","heartbeat_front_addr":"172.21.15.192:6821/87624122","state":["exists","up"]},{"osd":9,"uuid":"0f38fdca-e054-402a-b88e-7abc1659d361","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6827","nonce":781872938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6829","nonce":781872938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6833","nonce":781872938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":781872938},{"type":"v1","addr":"172.21.15.89:6831","nonce":781872938}]},"public_addr":"172.21.15.89:6827/781872938","cluster_addr":"172.21.15.89:6829/781872938","heartbeat_back_addr":"172.21.15.89:6833/781872938","heartbeat_front_addr":"172.21.15.89:6831/781872938","state":["exists","up"]},{"osd":10,"uuid":"bd567e81-805b-489f-8005-fb636fa97372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6825","nonce":3703256177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6827","nonce":3703256177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6831","nonce":3703256177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3703256177},{"type":"v1","addr":"172.21.15.135:6829","nonce":3703256177}]},"public_addr":"172.21.15.135:6825/3703256177","cluster_addr":"172.21.15.135:6827/3703256177","heartbeat_back_addr":"172.21.15.135:6831/3703256177","heartbeat_front_addr":"172.21.15.135:6829/3703256177","state":["exists","up"]},{"osd":11,"uuid":"1551f98d-9cc1-45b1-9b7c-5a5b2f814669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6825","nonce":2369975497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6827","nonce":2369975497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6831","nonce":2369975497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2369975497},{"type":"v1","addr":"172.21.15.192:6829","nonce":2369975497}]},"public_addr":"172.21.15.192:6825/2369975497","cluster_addr":"172.21.15.192:6827/2369975497","heartbeat_back_addr":"172.21.15.192:6831/2369975497","heartbeat_front_addr":"172.21.15.192:6829/2369975497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:04.876980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:26.896967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:40:47.915732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:14.623202+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:36.306955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:41:57.243371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:23.557007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:42:45.761606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:06.632556+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:32.745765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:43:54.590354+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T06:44:15.944881+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:6800/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/1796653810":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/2906183572":"2023-10-27T06:37:50.043554+0000","172.21.15.89:6801/925229622":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/4177523596":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6801/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/746092036":"2023-10-27T06:39:01.639977+0000","172.21.15.89:6800/2105103929":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/1900326324":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6801/692487046":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2757333567":"2023-10-27T06:37:50.043554+0000","172.21.15.89:0/516070855":"2023-10-27T06:37:23.538291+0000","172.21.15.89:0/2016135884":"2023-10-27T06:39:01.639977+0000","172.21.15.89:0/2689379544":"2023-10-27T06:37:23.538291+0000","172.21.15.89:6800/925229622":"2023-10-27T06:37:50.043554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T06:58:05.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72f40a15f0 msgr2=0x7f72f40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.234 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72f40a15f0 0x7f72f40a3ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7304009be0 tx=0x7f7304009260 comp rx=0 tx=0).stop 2023-10-26T06:58:05.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 msgr2=0x7f7308199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 0x7f7308199260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f72f8000c00 tx=0x7f72f8005740 comp rx=0 tx=0).stop 2023-10-26T06:58:05.235 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 shutdown_connections 2023-10-26T06:58:05.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f72f40a15f0 0x7f72f40a3ab0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73081066b0 0x7f730819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73081006a0 0x7f7308198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.236 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 --2- 172.21.15.89:0/27254983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7308101060 0x7f7308199260 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 >> 172.21.15.89:0/27254983 conn(0x7f73080fc170 msgr2=0x7f730810bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:05.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 shutdown_connections 2023-10-26T06:58:05.237 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.233+0000 7f730ffff700 1 -- 172.21.15.89:0/27254983 wait complete. 2023-10-26T06:58:05.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T06:58:05.456 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 -- 172.21.15.89:0/2862904691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8441066b0 msgr2=0x7ff844106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 --2- 172.21.15.89:0/2862904691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8441066b0 0x7ff844106a90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff8340098f0 tx=0x7ff834009c00 comp rx=0 tx=0).stop 2023-10-26T06:58:05.457 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 -- 172.21.15.89:0/2862904691 shutdown_connections 2023-10-26T06:58:05.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 --2- 172.21.15.89:0/2862904691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8441066b0 0x7ff844106a90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 --2- 172.21.15.89:0/2862904691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff844101060 0x7ff84410d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.458 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 --2- 172.21.15.89:0/2862904691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8441006a0 0x7ff844100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 -- 172.21.15.89:0/2862904691 >> 172.21.15.89:0/2862904691 conn(0x7ff8440fc170 msgr2=0x7ff8440fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:05.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 -- 172.21.15.89:0/2862904691 shutdown_connections 2023-10-26T06:58:05.459 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.455+0000 7ff849dff700 1 -- 172.21.15.89:0/2862904691 wait complete. 2023-10-26T06:58:05.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 Processor -- start 2023-10-26T06:58:05.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 -- start start 2023-10-26T06:58:05.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441006a0 0x7ff84419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff844101060 0x7ff84419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8441066b0 0x7ff844196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844196fe0 con 0x7ff844101060 2023-10-26T06:58:05.463 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844197150 con 0x7ff8441066b0 2023-10-26T06:58:05.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.456+0000 7ff849dff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844197310 con 0x7ff8441006a0 2023-10-26T06:58:05.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff844101060 0x7ff84419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff844101060 0x7ff84419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55790/0 (socket says 172.21.15.89:55790) 2023-10-26T06:58:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 -- 172.21.15.89:0/252189061 learned_addr learned my addr 172.21.15.89:0/252189061 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff8437fe700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441006a0 0x7ff84419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff843fff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8441066b0 0x7ff844196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 -- 172.21.15.89:0/252189061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441006a0 msgr2=0x7ff84419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441006a0 0x7ff84419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 -- 172.21.15.89:0/252189061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8441066b0 msgr2=0x7ff844196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8441066b0 0x7ff844196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 -- 172.21.15.89:0/252189061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff834009520 con 0x7ff844101060 2023-10-26T06:58:05.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff83bfff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff844101060 0x7ff84419cd90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00e9a0 tx=0x7ff82c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:05.467 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.457+0000 7ff8417fa700 1 -- 172.21.15.89:0/252189061 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c009790 con 0x7ff844101060 2023-10-26T06:58:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.458+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8441975f0 con 0x7ff844101060 2023-10-26T06:58:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.458+0000 7ff8417fa700 1 -- 172.21.15.89:0/252189061 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82c00cbd0 con 0x7ff844101060 2023-10-26T06:58:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.458+0000 7ff8417fa700 1 -- 172.21.15.89:0/252189061 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c018680 con 0x7ff844101060 2023-10-26T06:58:05.468 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.458+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8441a3840 con 0x7ff844101060 2023-10-26T06:58:05.469 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.459+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff844108ca0 con 0x7ff844101060 2023-10-26T06:58:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.462+0000 7ff8417fa700 1 -- 172.21.15.89:0/252189061 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff82c00cd40 con 0x7ff844101060 2023-10-26T06:58:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.463+0000 7ff8417fa700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8300a15f0 0x7ff8300a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.464+0000 7ff8417fa700 1 -- 172.21.15.89:0/252189061 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8543+0+0 (secure 0 0 0) 0x7ff82c014070 con 0x7ff844101060 2023-10-26T06:58:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.464+0000 7ff8437fe700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8300a15f0 0x7ff8300a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.464+0000 7ff8437fe700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8300a15f0 0x7ff8300a3ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff8340098c0 tx=0x7ff834019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:05.472 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.466+0000 7ff8417fa700 1 -- 172.21.15.89:0/252189061 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff82c096980 con 0x7ff844101060 2023-10-26T06:58:05.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:05 smithi089 ceph-mon[95031]: pgmap v655: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-26T06:58:05.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:05 smithi089 ceph-mon[95031]: osdmap e97: 12 total, 12 up, 12 in 2023-10-26T06:58:05.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4293138226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:58:05.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:05 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/27254983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:58:05.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-mon[131661]: pgmap v655: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-26T06:58:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-mon[131661]: osdmap e97: 12 total, 12 up, 12 in 2023-10-26T06:58:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4293138226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:58:05.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/27254983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:58:05.716 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.715+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff844101de0 con 0x7ff844101060 2023-10-26T06:58:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-mon[131403]: pgmap v655: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.3 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-10-26T06:58:05.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-mon[131403]: osdmap e97: 12 total, 12 up, 12 in 2023-10-26T06:58:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4293138226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-26T06:58:05.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/27254983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-26T06:58:05.719 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.718+0000 7ff8417fa700 1 -- 172.21.15.89:0/252189061 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff82c099fa0 con 0x7ff844101060 2023-10-26T06:58:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8300a15f0 msgr2=0x7ff8300a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.723 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8300a15f0 0x7ff8300a3ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff8340098c0 tx=0x7ff834019040 comp rx=0 tx=0).stop 2023-10-26T06:58:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff844101060 msgr2=0x7ff84419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff844101060 0x7ff84419cd90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00e9a0 tx=0x7ff82c00ecb0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 shutdown_connections 2023-10-26T06:58:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7ff8300a15f0 0x7ff8300a3ab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8441066b0 0x7ff844196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441006a0 0x7ff84419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.725 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 --2- 172.21.15.89:0/252189061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff844101060 0x7ff84419cd90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 >> 172.21.15.89:0/252189061 conn(0x7ff8440fc170 msgr2=0x7ff8440fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 shutdown_connections 2023-10-26T06:58:05.726 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.721+0000 7ff849dff700 1 -- 172.21.15.89:0/252189061 wait complete. 2023-10-26T06:58:05.727 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T06:58:05.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T06:58:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 -- 172.21.15.89:0/2227909231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc100720 msgr2=0x7fc0bc100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/2227909231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc100720 0x7fc0bc100ba0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00098f0 tx=0x7fc0b0009c00 comp rx=0 tx=0).stop 2023-10-26T06:58:05.948 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 -- 172.21.15.89:0/2227909231 shutdown_connections 2023-10-26T06:58:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/2227909231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc106730 0x7fc0bc106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/2227909231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0bc1010e0 0x7fc0bc10d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.949 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/2227909231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc100720 0x7fc0bc100ba0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 -- 172.21.15.89:0/2227909231 >> 172.21.15.89:0/2227909231 conn(0x7fc0bc0fc1b0 msgr2=0x7fc0bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 -- 172.21.15.89:0/2227909231 shutdown_connections 2023-10-26T06:58:05.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.946+0000 7fc0c21ff700 1 -- 172.21.15.89:0/2227909231 wait complete. 2023-10-26T06:58:05.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.947+0000 7fc0c21ff700 1 Processor -- start 2023-10-26T06:58:05.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.947+0000 7fc0c21ff700 1 -- start start 2023-10-26T06:58:05.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0c21ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc100720 0x7fc0bc072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0c21ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0bc1010e0 0x7fc0bc073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0c21ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc106730 0x7fc0bc073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0c21ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1a4f50 con 0x7fc0bc100720 2023-10-26T06:58:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0c21ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1a50c0 con 0x7fc0bc1010e0 2023-10-26T06:58:05.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0c21ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1a5280 con 0x7fc0bc106730 2023-10-26T06:58:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0bb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc100720 0x7fc0bc072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0bb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc100720 0x7fc0bc072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55798/0 (socket says 172.21.15.89:55798) 2023-10-26T06:58:05.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0bb7fe700 1 -- 172.21.15.89:0/94978367 learned_addr learned my addr 172.21.15.89:0/94978367 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:05.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0baffd700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0bc1010e0 0x7fc0bc073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0bbfff700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc106730 0x7fc0bc073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0bb7fe700 1 -- 172.21.15.89:0/94978367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc106730 msgr2=0x7fc0bc073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0bb7fe700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc106730 0x7fc0bc073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.948+0000 7fc0bb7fe700 1 -- 172.21.15.89:0/94978367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0bc1010e0 msgr2=0x7fc0bc073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:05.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0bb7fe700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0bc1010e0 0x7fc0bc073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:05.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0bb7fe700 1 -- 172.21.15.89:0/94978367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0095f0 con 0x7fc0bc100720 2023-10-26T06:58:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0bb7fe700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc100720 0x7fc0bc072af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00ec10 tx=0x7fc0ac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0b8ff9700 1 -- 172.21.15.89:0/94978367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00cd40 con 0x7fc0bc100720 2023-10-26T06:58:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0b8ff9700 1 -- 172.21.15.89:0/94978367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0ac0044e0 con 0x7fc0bc100720 2023-10-26T06:58:05.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0b8ff9700 1 -- 172.21.15.89:0/94978367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac010690 con 0x7fc0bc100720 2023-10-26T06:58:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b0009520 con 0x7fc0bc100720 2023-10-26T06:58:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.949+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc1a5860 con 0x7fc0bc100720 2023-10-26T06:58:05.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.950+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc108d40 con 0x7fc0bc100720 2023-10-26T06:58:05.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.952+0000 7fc0b8ff9700 1 -- 172.21.15.89:0/94978367 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc0ac003710 con 0x7fc0bc100720 2023-10-26T06:58:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.954+0000 7fc0b8ff9700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0a40a15f0 0x7fc0a40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.954+0000 7fc0b8ff9700 1 -- 172.21.15.89:0/94978367 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc0ac014070 con 0x7fc0bc100720 2023-10-26T06:58:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.955+0000 7fc0baffd700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0a40a15f0 0x7fc0a40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:05.961 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.955+0000 7fc0baffd700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0a40a15f0 0x7fc0a40a3ab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00098c0 tx=0x7fc0b0019040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:05.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:05.958+0000 7fc0b8ff9700 1 -- 172.21.15.89:0/94978367 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc0ac096950 con 0x7fc0bc100720 2023-10-26T06:58:06.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:05 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-a[95007]: 2023-10-26T06:58:05.716+0000 7f0673506700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T06:58:06.145 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-i[171581]: ignoring --setuser ceph since I am not root 2023-10-26T06:58:06.146 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-i[171581]: ignoring --setgroup ceph since I am not root 2023-10-26T06:58:06.146 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-mds[171606]: -- 172.21.15.192:0/547977683 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa42c391e0 con 0x55fa42c0e800 2023-10-26T06:58:06.146 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 06:58:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-i[171581]: starting mds.i at 2023-10-26T06:58:06.211 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.210+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc0bc073fd0 con 0x7fc0bc100720 2023-10-26T06:58:06.217 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b[170212]: ignoring --setuser ceph since I am not root 2023-10-26T06:58:06.217 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b[170212]: ignoring --setgroup ceph since I am not root 2023-10-26T06:58:06.218 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-mds[170236]: -- 172.21.15.135:0/1673712858 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c774a11e0 con 0x556c77477000 2023-10-26T06:58:06.218 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 06:58:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b[170212]: starting mds.b at 2023-10-26T06:58:06.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:06 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/252189061' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-26T06:58:06.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:06 smithi192 ceph-mon[131661]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-26T06:58:06.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:06 smithi192 ceph-mon[131661]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T06:58:06.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:06 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/252189061' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-26T06:58:06.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:06 smithi192 ceph-mon[131661]: osdmap e98: 12 total, 12 up, 12 in 2023-10-26T06:58:06.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:06 smithi192 ceph-mon[131661]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-10-26T06:58:06.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:06 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/94978367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:58:06.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:06 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/252189061' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-26T06:58:06.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:06 smithi135 ceph-mon[131403]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-26T06:58:06.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:06 smithi135 ceph-mon[131403]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T06:58:06.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:06 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/252189061' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-26T06:58:06.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:06 smithi135 ceph-mon[131403]: osdmap e98: 12 total, 12 up, 12 in 2023-10-26T06:58:06.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:06 smithi135 ceph-mon[131403]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-10-26T06:58:06.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:06 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/94978367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:58:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.723+0000 7fc0b8ff9700 1 -- 172.21.15.89:0/94978367 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc0ac099f70 con 0x7fc0bc100720 2023-10-26T06:58:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0a40a15f0 msgr2=0x7fc0a40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0a40a15f0 0x7fc0a40a3ab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00098c0 tx=0x7fc0b0019040 comp rx=0 tx=0).stop 2023-10-26T06:58:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc100720 msgr2=0x7fc0bc072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc100720 0x7fc0bc072af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00ec10 tx=0x7fc0ac00ef20 comp rx=0 tx=0).stop 2023-10-26T06:58:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 shutdown_connections 2023-10-26T06:58:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fc0a40a15f0 0x7fc0a40a3ab0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0bc100720 0x7fc0bc072af0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc106730 0x7fc0bc073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 --2- 172.21.15.89:0/94978367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0bc1010e0 0x7fc0bc073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.726+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 >> 172.21.15.89:0/94978367 conn(0x7fc0bc0fc1b0 msgr2=0x7fc0bc0fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.727+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 shutdown_connections 2023-10-26T06:58:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.727+0000 7fc0c21ff700 1 -- 172.21.15.89:0/94978367 wait complete. 2023-10-26T06:58:06.747 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:06 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/252189061' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-26T06:58:06.748 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:06 smithi089 ceph-mon[95031]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-26T06:58:06.748 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:06 smithi089 ceph-mon[95031]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T06:58:06.748 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:06 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/252189061' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-26T06:58:06.749 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:06 smithi089 ceph-mon[95031]: osdmap e98: 12 total, 12 up, 12 in 2023-10-26T06:58:06.749 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:06 smithi089 ceph-mon[95031]: fsmap cephfs:0/1 10 up:standby, 1 failed 2023-10-26T06:58:06.749 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:06 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/94978367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-26T06:58:06.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T06:58:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.954+0000 7f6e8bfff700 1 -- 172.21.15.89:0/3381253264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e841010e0 msgr2=0x7f6e8410d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:06.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.954+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/3381253264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e841010e0 0x7f6e8410d5f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00b0a0 tx=0x7f6e7c00b3b0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.954+0000 7f6e8bfff700 1 -- 172.21.15.89:0/3381253264 shutdown_connections 2023-10-26T06:58:06.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.954+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/3381253264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e841010e0 0x7f6e8410d5f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.954+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/3381253264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e84106730 0x7f6e84106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.954+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/3381253264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e84100720 0x7f6e84100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.954+0000 7f6e8bfff700 1 -- 172.21.15.89:0/3381253264 >> 172.21.15.89:0/3381253264 conn(0x7f6e840fc1b0 msgr2=0x7f6e840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:06.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.955+0000 7f6e8bfff700 1 -- 172.21.15.89:0/3381253264 shutdown_connections 2023-10-26T06:58:06.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.955+0000 7f6e8bfff700 1 -- 172.21.15.89:0/3381253264 wait complete. 2023-10-26T06:58:06.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 Processor -- start 2023-10-26T06:58:06.960 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 -- start start 2023-10-26T06:58:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e84100720 0x7f6e84198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:06.962 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e841010e0 0x7f6e84199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84106730 0x7f6e8419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419cb40 con 0x7f6e84106730 2023-10-26T06:58:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419ccb0 con 0x7f6e841010e0 2023-10-26T06:58:06.963 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8bfff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419ce70 con 0x7f6e84100720 2023-10-26T06:58:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8959a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e841010e0 0x7f6e84199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8959a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e841010e0 0x7f6e84199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.89:37706/0 (socket says 172.21.15.89:37706) 2023-10-26T06:58:06.964 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.956+0000 7f6e8959a700 1 -- 172.21.15.89:0/4033065751 learned_addr learned my addr 172.21.15.89:0/4033065751 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8a59c700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84106730 0x7f6e8419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e89d9b700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e84100720 0x7f6e84198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8959a700 1 -- 172.21.15.89:0/4033065751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e84100720 msgr2=0x7f6e84198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:06.965 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8959a700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e84100720 0x7f6e84198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8959a700 1 -- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84106730 msgr2=0x7f6e8419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8959a700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84106730 0x7f6e8419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:06.966 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8959a700 1 -- 172.21.15.89:0/4033065751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009d00 con 0x7f6e841010e0 2023-10-26T06:58:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8a59c700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84106730 0x7f6e8419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:58:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e89d9b700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e84100720 0x7f6e84198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T06:58:06.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.957+0000 7f6e8959a700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e841010e0 0x7f6e84199250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9a0 tx=0x7f6e7400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:06.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.958+0000 7f6e7affd700 1 -- 172.21.15.89:0/4033065751 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009790 con 0x7f6e841010e0 2023-10-26T06:58:06.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.958+0000 7f6e7affd700 1 -- 172.21.15.89:0/4033065751 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e74010430 con 0x7f6e841010e0 2023-10-26T06:58:06.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.958+0000 7f6e7affd700 1 -- 172.21.15.89:0/4033065751 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74018680 con 0x7f6e841010e0 2023-10-26T06:58:06.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.958+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8419d150 con 0x7f6e841010e0 2023-10-26T06:58:06.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.958+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e841a5ab0 con 0x7f6e841010e0 2023-10-26T06:58:06.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.959+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e84101e60 con 0x7f6e841010e0 2023-10-26T06:58:06.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.961+0000 7f6e7affd700 1 -- 172.21.15.89:0/4033065751 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6e740105a0 con 0x7f6e841010e0 2023-10-26T06:58:06.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.962+0000 7f6e7affd700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6e700a15f0 0x7f6e700a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.962+0000 7f6e7affd700 1 -- 172.21.15.89:0/4033065751 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6e74014070 con 0x7f6e841010e0 2023-10-26T06:58:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.963+0000 7f6e89d9b700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6e700a15f0 0x7f6e700a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:06.971 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.963+0000 7f6e89d9b700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6e700a15f0 0x7f6e700a3ab0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009fd0 tx=0x7f6e80009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:06.972 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:06.965+0000 7f6e7affd700 1 -- 172.21.15.89:0/4033065751 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e740967d0 con 0x7f6e841010e0 2023-10-26T06:58:07.169 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.168+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e84110690 con 0x7f6e841010e0 2023-10-26T06:58:07.645 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: pgmap v658: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 14 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-10-26T06:58:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-26T06:58:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-26T06:58:07.646 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: Cluster is now healthy 2023-10-26T06:58:07.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/94978367' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-26T06:58:07.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.192:6838/1972366847,v1:172.21.15.192:6839/1972366847] up:boot 2023-10-26T06:58:07.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: mds.? [v2:172.21.15.135:6838/2966282334,v1:172.21.15.135:6839/2966282334] up:boot 2023-10-26T06:58:07.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: fsmap 12 up:standby 2023-10-26T06:58:07.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-26T06:58:07.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-26T06:58:07.648 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/4033065751' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:07.649 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:07 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:07.717 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: pgmap v658: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 14 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-10-26T06:58:07.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-26T06:58:07.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-26T06:58:07.718 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: Cluster is now healthy 2023-10-26T06:58:07.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/94978367' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-26T06:58:07.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.192:6838/1972366847,v1:172.21.15.192:6839/1972366847] up:boot 2023-10-26T06:58:07.719 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: mds.? [v2:172.21.15.135:6838/2966282334,v1:172.21.15.135:6839/2966282334] up:boot 2023-10-26T06:58:07.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: fsmap 12 up:standby 2023-10-26T06:58:07.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-26T06:58:07.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-26T06:58:07.720 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/4033065751' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:07.721 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:07 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:07.731 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.730+0000 7f6e7affd700 1 -- 172.21.15.89:0/4033065751 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6e740260d0 con 0x7f6e841010e0 2023-10-26T06:58:07.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6e700a15f0 msgr2=0x7f6e700a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:07.735 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6e700a15f0 0x7f6e700a3ab0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009fd0 tx=0x7f6e80009260 comp rx=0 tx=0).stop 2023-10-26T06:58:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e841010e0 msgr2=0x7f6e84199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:07.736 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e841010e0 0x7f6e84199250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9a0 tx=0x7f6e7400ecb0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 shutdown_connections 2023-10-26T06:58:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7f6e700a15f0 0x7f6e700a3ab0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84106730 0x7f6e8419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.737 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e84100720 0x7f6e84198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 --2- 172.21.15.89:0/4033065751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e841010e0 0x7f6e84199250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.733+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 >> 172.21.15.89:0/4033065751 conn(0x7f6e840fc1b0 msgr2=0x7f6e841059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:07.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.734+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 shutdown_connections 2023-10-26T06:58:07.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.734+0000 7f6e8bfff700 1 -- 172.21.15.89:0/4033065751 wait complete. 2023-10-26T06:58:07.739 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2023-10-26T06:58:07.752 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: pgmap v658: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 14 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-10-26T06:58:07.753 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-26T06:58:07.753 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-26T06:58:07.754 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: Cluster is now healthy 2023-10-26T06:58:07.754 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/94978367' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-26T06:58:07.754 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.192:6838/1972366847,v1:172.21.15.192:6839/1972366847] up:boot 2023-10-26T06:58:07.754 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: mds.? [v2:172.21.15.135:6838/2966282334,v1:172.21.15.135:6839/2966282334] up:boot 2023-10-26T06:58:07.755 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: fsmap 12 up:standby 2023-10-26T06:58:07.756 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-26T06:58:07.756 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: from='mgr.14217 172.21.15.89:0/951595855' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-26T06:58:07.757 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/4033065751' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:07.757 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:07 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:07.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T06:58:07.967 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.965+0000 7fcfca3ff700 1 -- 172.21.15.89:0/2655789029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 msgr2=0x7fcfc410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:07.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.965+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/2655789029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc410f3d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009f90 tx=0x7fcfbc00b330 comp rx=0 tx=0).stop 2023-10-26T06:58:07.968 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.966+0000 7fcfca3ff700 1 -- 172.21.15.89:0/2655789029 shutdown_connections 2023-10-26T06:58:07.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.966+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/2655789029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc410f3d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.966+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/2655789029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfc4072c20 0x7fcfc4073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.969 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.966+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/2655789029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc4073540 0x7fcfc410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.966+0000 7fcfca3ff700 1 -- 172.21.15.89:0/2655789029 >> 172.21.15.89:0/2655789029 conn(0x7fcfc40fc1b0 msgr2=0x7fcfc40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:07.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.966+0000 7fcfca3ff700 1 -- 172.21.15.89:0/2655789029 shutdown_connections 2023-10-26T06:58:07.970 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.966+0000 7fcfca3ff700 1 -- 172.21.15.89:0/2655789029 wait complete. 2023-10-26T06:58:07.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 Processor -- start 2023-10-26T06:58:07.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 -- start start 2023-10-26T06:58:07.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc4072c20 0x7fcfc4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:07.973 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfc4073540 0x7fcfc4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:07.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:07.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc419de50 con 0x7fcfc410cf60 2023-10-26T06:58:07.974 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc419dfc0 con 0x7fcfc4073540 2023-10-26T06:58:07.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfca3ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc419e180 con 0x7fcfc4072c20 2023-10-26T06:58:07.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfc899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:07.975 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc4072c20 0x7fcfc4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:07.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc4072c20 0x7fcfc4198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.89:33740/0 (socket says 172.21.15.89:33740) 2023-10-26T06:58:07.976 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfc37fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfc4073540 0x7fcfc4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:07.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfc3fff700 1 -- 172.21.15.89:0/3019249290 learned_addr learned my addr 172.21.15.89:0/3019249290 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-10-26T06:58:07.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.967+0000 7fcfc899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55830/0 (socket says 172.21.15.89:55830) 2023-10-26T06:58:07.977 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc899c700 1 -- 172.21.15.89:0/3019249290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc4072c20 msgr2=0x7fcfc4198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:07.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc899c700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc4072c20 0x7fcfc4198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc899c700 1 -- 172.21.15.89:0/3019249290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfc4073540 msgr2=0x7fcfc4199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:07.978 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc899c700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfc4073540 0x7fcfc4199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:07.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc899c700 1 -- 172.21.15.89:0/3019249290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80095f0 con 0x7fcfc410cf60 2023-10-26T06:58:07.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc899c700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc419d810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b6c0 tx=0x7fcfbc0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:07.979 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc17fa700 1 -- 172.21.15.89:0/3019249290 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00e040 con 0x7fcfc410cf60 2023-10-26T06:58:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc17fa700 1 -- 172.21.15.89:0/3019249290 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfbc01b450 con 0x7fcfc410cf60 2023-10-26T06:58:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc009c40 con 0x7fcfc410cf60 2023-10-26T06:58:07.980 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfc17fa700 1 -- 172.21.15.89:0/3019249290 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0052c0 con 0x7fcfc410cf60 2023-10-26T06:58:07.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.968+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc41a5ac0 con 0x7fcfc410cf60 2023-10-26T06:58:07.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.969+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc4108170 con 0x7fcfc410cf60 2023-10-26T06:58:07.981 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.971+0000 7fcfc17fa700 1 -- 172.21.15.89:0/3019249290 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcfbc01b5c0 con 0x7fcfc410cf60 2023-10-26T06:58:07.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.973+0000 7fcfc17fa700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcfac0a15f0 0x7fcfac0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T06:58:07.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.973+0000 7fcfc17fa700 1 -- 172.21.15.89:0/3019249290 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8031+0+0 (secure 0 0 0) 0x7fcfbc0ce190 con 0x7fcfc410cf60 2023-10-26T06:58:07.983 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.974+0000 7fcfc3fff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcfac0a15f0 0x7fcfac0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T06:58:07.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.974+0000 7fcfc3fff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcfac0a15f0 0x7fcfac0a3ab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4007c90 tx=0x7fcfb4007480 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T06:58:07.984 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:07.977+0000 7fcfc17fa700 1 -- 172.21.15.89:0/3019249290 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcfbc0976e0 con 0x7fcfc410cf60 2023-10-26T06:58:08.183 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.181+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcfc40742d0 con 0x7fcfc410cf60 2023-10-26T06:58:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.732+0000 7fcfc17fa700 1 -- 172.21.15.89:0/3019249290 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcfbc09ad00 con 0x7fcfc410cf60 2023-10-26T06:58:08.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.737+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcfac0a15f0 msgr2=0x7fcfac0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:08.738 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.737+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcfac0a15f0 0x7fcfac0a3ab0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4007c90 tx=0x7fcfb4007480 comp rx=0 tx=0).stop 2023-10-26T06:58:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.737+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 msgr2=0x7fcfc419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T06:58:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.737+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc419d810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b6c0 tx=0x7fcfbc0079d0 comp rx=0 tx=0).stop 2023-10-26T06:58:08.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.740+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 shutdown_connections 2023-10-26T06:58:08.741 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.740+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:6800/1106080795,v1:172.21.15.89:6801/1106080795] conn(0x7fcfac0a15f0 0x7fcfac0a3ab0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:08.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.740+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfc410cf60 0x7fcfc419d810 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:08.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.740+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfc4072c20 0x7fcfc4198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:08.742 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.740+0000 7fcfca3ff700 1 --2- 172.21.15.89:0/3019249290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfc4073540 0x7fcfc4199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T06:58:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.740+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 >> 172.21.15.89:0/3019249290 conn(0x7fcfc40fc1b0 msgr2=0x7fcfc40fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T06:58:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.742+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 shutdown_connections 2023-10-26T06:58:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2023-10-26T06:58:08.742+0000 7fcfca3ff700 1 -- 172.21.15.89:0/3019249290 wait complete. 2023-10-26T06:58:08.748 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2023-10-26T06:58:08.782 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-26T06:58:08.910 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-26T06:58:08.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-26T06:58:09.003 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:08 smithi192 ceph-mon[131661]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-26T06:58:09.004 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:08 smithi192 ceph-mon[131661]: osdmap e99: 12 total, 12 up, 12 in 2023-10-26T06:58:09.004 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:08 smithi192 ceph-mon[131661]: pgmap v660: 65 pgs: 65 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2023-10-26T06:58:09.005 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:08 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3019249290' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:09.015 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-26T06:58:09.016 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-26T06:58:09.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:08 smithi089 ceph-mon[95031]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-26T06:58:09.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:08 smithi089 ceph-mon[95031]: osdmap e99: 12 total, 12 up, 12 in 2023-10-26T06:58:09.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:08 smithi089 ceph-mon[95031]: pgmap v660: 65 pgs: 65 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2023-10-26T06:58:09.041 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:08 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3019249290' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:09.099 DEBUG:teuthology.orchestra.run.smithi135:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-26T06:58:09.175 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-26T06:58:09.215 INFO:tasks.cephadm:Stopping all daemons... 2023-10-26T06:58:09.216 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-26T06:58:09.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a 2023-10-26T06:58:09.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:08 smithi135 ceph-mon[131403]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-26T06:58:09.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:08 smithi135 ceph-mon[131403]: osdmap e99: 12 total, 12 up, 12 in 2023-10-26T06:58:09.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:08 smithi135 ceph-mon[131403]: pgmap v660: 65 pgs: 65 active+clean; 13 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2023-10-26T06:58:09.219 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:08 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3019249290' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-26T06:58:09.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:09 smithi089 systemd[1]: Stopping Ceph mon.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:10.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:09 smithi089 ceph-mon[95031]: from='client.? 172.21.15.89:0/3019249290' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-26T06:58:10.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:09 smithi089 ceph-mon[95031]: osdmap e100: 12 total, 12 up, 12 in 2023-10-26T06:58:10.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:09 smithi192 ceph-mon[131661]: from='client.? 172.21.15.89:0/3019249290' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-26T06:58:10.147 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:09 smithi192 ceph-mon[131661]: osdmap e100: 12 total, 12 up, 12 in 2023-10-26T06:58:10.217 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:09 smithi135 ceph-mon[131403]: from='client.? 172.21.15.89:0/3019249290' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-26T06:58:10.218 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:09 smithi135 ceph-mon[131403]: osdmap e100: 12 total, 12 up, 12 in 2023-10-26T06:58:10.539 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-a[95007]: 2023-10-26T06:58:10.153+0000 7f0678d11700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:58:10.540 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-a[95007]: 2023-10-26T06:58:10.153+0000 7f0678d11700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-26T06:58:11.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Oct 26 06:58:10 smithi089 bash[178697]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-a 2023-10-26T06:58:11.446 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.a.service' 2023-10-26T06:58:11.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:58:11.627 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-26T06:58:11.627 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-26T06:58:11.628 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.b 2023-10-26T06:58:11.969 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:11 smithi135 systemd[1]: Stopping Ceph mon.b for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:12.572 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:12 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-b[131379]: 2023-10-26T06:58:12.241+0000 7f7a7b7ba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:58:12.572 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:12 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-b[131379]: 2023-10-26T06:58:12.241+0000 7f7a7b7ba700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-26T06:58:12.972 INFO:journalctl@ceph.mon.b.smithi135.stdout:Oct 26 06:58:12 smithi135 bash[174858]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-b 2023-10-26T06:58:13.203 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.b.service' 2023-10-26T06:58:13.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:58:13.327 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-26T06:58:13.327 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-26T06:58:13.328 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.c 2023-10-26T06:58:13.852 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:13 smithi192 systemd[1]: Stopping Ceph mon.c for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:14.145 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-c[131634]: 2023-10-26T06:58:13.850+0000 7f1b4f692700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:58:14.146 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-c[131634]: 2023-10-26T06:58:13.850+0000 7f1b4f692700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-26T06:58:14.514 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.c.service' 2023-10-26T06:58:14.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:14 smithi192 bash[176420]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mon-c 2023-10-26T06:58:14.542 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:14 smithi192 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mon.c.service: Succeeded. 2023-10-26T06:58:14.543 INFO:journalctl@ceph.mon.c.smithi192.stdout:Oct 26 06:58:14 smithi192 systemd[1]: Stopped Ceph mon.c for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T06:58:14.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:58:14.569 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-26T06:58:14.569 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-26T06:58:14.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.x 2023-10-26T06:58:15.039 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:58:14 smithi089 systemd[1]: Stopping Ceph mgr.x for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:15.391 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:58:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x[95279]: 2023-10-26T06:58:15.128+0000 7f581d5ee700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:58:15.789 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Oct 26 06:58:15 smithi089 bash[178839]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-x 2023-10-26T06:58:15.923 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.x.service' 2023-10-26T06:58:16.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:58:16.014 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-26T06:58:16.014 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-26T06:58:16.015 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.y 2023-10-26T06:58:16.435 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:58:16 smithi135 systemd[1]: Stopping Ceph mgr.y for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:16.717 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:58:16 smithi135 bash[174988]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-y 2023-10-26T06:58:16.717 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Oct 26 06:58:16 smithi135 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-26T06:58:16.818 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.y.service' 2023-10-26T06:58:16.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:58:16.856 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-26T06:58:16.856 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-26T06:58:16.856 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.z 2023-10-26T06:58:17.145 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:58:16 smithi192 systemd[1]: Stopping Ceph mgr.z for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:17.553 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:58:17 smithi192 bash[176558]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mgr-z 2023-10-26T06:58:17.736 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Oct 26 06:58:17 smithi192 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-26T06:58:17.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mgr.z.service' 2023-10-26T06:58:17.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:58:17.777 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-26T06:58:17.778 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-26T06:58:17.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.0 2023-10-26T06:58:18.278 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:58:17 smithi089 systemd[1]: Stopping Ceph osd.0 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:18.539 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:58:18 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-0[117229]: 2023-10-26T06:58:18.277+0000 7fa16126c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:58:18.540 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:58:18 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-0[117229]: 2023-10-26T06:58:18.277+0000 7fa16126c700 -1 osd.0 100 *** Got signal Terminated *** 2023-10-26T06:58:18.540 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:58:18 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-0[117229]: 2023-10-26T06:58:18.277+0000 7fa16126c700 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T06:58:48.540 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:58:48 smithi089 bash[178980]: time="2023-10-26T06:58:48Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-0 in 30 seconds, resorting to SIGKILL" 2023-10-26T06:58:49.170 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:58:48 smithi089 bash[178980]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-0 2023-10-26T06:58:49.539 INFO:journalctl@ceph.osd.0.smithi089.stdout:Oct 26 06:58:49 smithi089 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.0.service: Main process exited, code=exited, status=137/n/a 2023-10-26T06:58:50.803 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.0.service' 2023-10-26T06:58:50.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:58:50.892 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-26T06:58:50.893 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-26T06:58:50.893 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.1 2023-10-26T06:58:51.217 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:58:50 smithi135 systemd[1]: Stopping Ceph osd.1 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:58:51.717 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:58:51 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1[137576]: 2023-10-26T06:58:51.239+0000 7f3283a48700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:58:51.718 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:58:51 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1[137576]: 2023-10-26T06:58:51.239+0000 7f3283a48700 -1 osd.1 100 *** Got signal Terminated *** 2023-10-26T06:58:51.718 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:58:51 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1[137576]: 2023-10-26T06:58:51.239+0000 7f3283a48700 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T06:59:21.667 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:59:21 smithi135 bash[175141]: time="2023-10-26T06:59:21Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1 in 30 seconds, resorting to SIGKILL" 2023-10-26T06:59:21.668 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:59:21 smithi135 bash[175141]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-1 2023-10-26T06:59:21.967 INFO:journalctl@ceph.osd.1.smithi135.stdout:Oct 26 06:59:21 smithi135 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.1.service: Main process exited, code=exited, status=137/n/a 2023-10-26T06:59:22.963 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.1.service' 2023-10-26T06:59:23.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:59:23.004 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-26T06:59:23.005 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-26T06:59:23.005 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.2 2023-10-26T06:59:23.396 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:59:23 smithi192 systemd[1]: Stopping Ceph osd.2 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:59:23.895 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:59:23 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2[138885]: 2023-10-26T06:59:23.522+0000 7f04ff8cf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:59:23.896 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:59:23 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2[138885]: 2023-10-26T06:59:23.522+0000 7f04ff8cf700 -1 osd.2 100 *** Got signal Terminated *** 2023-10-26T06:59:23.896 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:59:23 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2[138885]: 2023-10-26T06:59:23.522+0000 7f04ff8cf700 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T06:59:49.895 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:49 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:49.422+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:49.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:49 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:49.640+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:50.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:50 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:50.228+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:50.895 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:50 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:50.469+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:50.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:50 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:50.668+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:51.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:51 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:51.217+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:51.785 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:51 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:51.484+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:51.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:51 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:51.640+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:52.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:51 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:51.784+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:52.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:52 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:52.214+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:52.757 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:52 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:52.437+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:52.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:52 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:52.591+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:53.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:52 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:52.755+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:53.173 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:52 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:52.883+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T06:59:53.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:53.171+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:53.718 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:53 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:53.451+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:53.719 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:59:53 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:59:53.537+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T06:59:53.719 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:59:53 smithi192 bash[176714]: time="2023-10-26T06:59:53Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2 in 30 seconds, resorting to SIGKILL" 2023-10-26T06:59:53.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:53 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:53.590+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:54.017 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:59:53 smithi192 bash[176714]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-2 2023-10-26T06:59:54.017 INFO:journalctl@ceph.osd.2.smithi192.stdout:Oct 26 06:59:54 smithi192 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.2.service: Main process exited, code=exited, status=137/n/a 2023-10-26T06:59:54.018 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:53 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:53.720+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:54.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:53.930+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T06:59:54.290 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:54.066+0000 7f7a0869f700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:23.031127+0000 front 2023-10-26T06:58:23.031144+0000 (oldest deadline 2023-10-26T06:59:54.034145+0000) 2023-10-26T06:59:54.290 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:54.140+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:54.896 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:54 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:54.454+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:54.897 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:59:54 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:59:54.565+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T06:59:54.897 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:54 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:54.680+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:54.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:54 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:54.634+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:55.189 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:55.022+0000 7f7a0869f700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:23.031127+0000 front 2023-10-26T06:58:23.031144+0000 (oldest deadline 2023-10-26T06:59:54.034145+0000) 2023-10-26T06:59:55.190 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:54.883+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T06:59:55.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.2.service' 2023-10-26T06:59:55.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T06:59:55.421 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-26T06:59:55.421 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-26T06:59:55.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.3 2023-10-26T06:59:55.484 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:55.187+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:55.633 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:59:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:59:55.548+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T06:59:55.633 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:55.485+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:55.789 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:55 smithi089 systemd[1]: Stopping Ceph osd.3 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T06:59:55.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:55.631+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:55.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:55 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:55.588+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:56.193 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:55.855+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T06:59:56.194 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:55.929+0000 7f7a0b8a8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T06:59:56.194 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:55.929+0000 7f7a0b8a8700 -1 osd.3 100 *** Got signal Terminated *** 2023-10-26T06:59:56.195 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:55.929+0000 7f7a0b8a8700 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T06:59:56.195 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:55.997+0000 7f7a0869f700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:23.031127+0000 front 2023-10-26T06:58:23.031144+0000 (oldest deadline 2023-10-26T06:59:54.034145+0000) 2023-10-26T06:59:56.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:56.192+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:56.895 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:59:56 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:59:56.578+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T06:59:56.896 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:56 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:56.464+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:56.897 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:56 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:56.680+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:56.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:56 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:56.584+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:57.234 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:57.047+0000 7f7a0869f700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:23.031127+0000 front 2023-10-26T06:58:23.031144+0000 (oldest deadline 2023-10-26T06:59:54.034145+0000) 2023-10-26T06:59:57.235 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:56.807+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T06:59:57.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:57.233+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:57.895 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:59:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:59:57.561+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T06:59:57.896 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:57.440+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:57.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:57.664+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:57.969 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:57 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:57.583+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:58.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:57 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:57.759+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T06:59:58.539 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:58 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:58.043+0000 7f7a0869f700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:23.031127+0000 front 2023-10-26T06:58:23.031144+0000 (oldest deadline 2023-10-26T06:59:54.034145+0000) 2023-10-26T06:59:58.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:58 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:58.271+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:58.895 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:59:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:59:58.516+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T06:59:58.896 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:58.460+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:58.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:58.705+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:58.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:58 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:58.588+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T06:59:59.250 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 06:59:59 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T06:59:59.041+0000 7f7a0869f700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:23.031127+0000 front 2023-10-26T06:58:23.031144+0000 (oldest deadline 2023-10-26T06:59:54.034145+0000) 2023-10-26T06:59:59.250 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:58 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:58.799+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T06:59:59.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 06:59:59 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T06:59:59.248+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T06:59:59.895 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 06:59:59 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T06:59:59.458+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T06:59:59.896 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 06:59:59 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T06:59:59.526+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T06:59:59.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 06:59:59 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T06:59:59.677+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T06:59:59.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 06:59:59 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T06:59:59.577+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:00.044 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 06:59:59 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T06:59:59.792+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:00.539 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 07:00:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3[125787]: 2023-10-26T07:00:00.043+0000 7f7a0869f700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:23.031127+0000 front 2023-10-26T06:58:23.031144+0000 (oldest deadline 2023-10-26T06:59:54.034145+0000) 2023-10-26T07:00:00.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:00.218+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:00.896 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:00 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:00.495+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:00.896 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:00 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:00.474+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:00.897 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:00 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:00.706+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:00.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:00 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:00.583+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:01.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:00 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:00.752+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:01.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:01 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:01.250+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:01.710 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:01 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:01.446+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:01.711 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:01 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:01.456+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:01.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:01 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:01.549+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:02.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:01 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:01.714+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:02.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:01 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:01.709+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:02.522 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 07:00:02 smithi089 bash[179282]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-3 2023-10-26T07:00:02.522 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:02 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:02.246+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:02.748 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:02 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:02.464+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:02.749 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:02 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:02.418+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:02.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:02 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:02.514+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:03.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:02 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:02.690+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:03.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:02 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:02.747+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:03.533 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:03 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:03.271+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:03.782 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:03 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:03.486+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:03.782 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:03 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:03.454+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:03.789 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:03 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:03.649+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:03.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:03 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:03.465+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:04.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:03 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:03.780+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:04.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.3.service' 2023-10-26T07:00:04.539 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 07:00:04 smithi089 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.3.service: Succeeded. 2023-10-26T07:00:04.540 INFO:journalctl@ceph.osd.3.smithi089.stdout:Oct 26 07:00:04 smithi089 systemd[1]: Stopped Ceph osd.3 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T07:00:04.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:04 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:04.283+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:04.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:00:04.600 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-26T07:00:04.601 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-26T07:00:04.601 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.4 2023-10-26T07:00:04.760 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:04 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:04.455+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:04.761 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:04 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:04.442+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:04.937 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 07:00:04 smithi135 systemd[1]: Stopping Ceph osd.4 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:00:04.938 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:04 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:04.513+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:05.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:04 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:04.677+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:05.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:04 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:04.759+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:05.217 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 07:00:04 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-4[146265]: 2023-10-26T07:00:04.935+0000 7fe202c8a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:00:05.217 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 07:00:04 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-4[146265]: 2023-10-26T07:00:04.935+0000 7fe202c8a700 -1 osd.4 100 *** Got signal Terminated *** 2023-10-26T07:00:05.218 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 07:00:04 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-4[146265]: 2023-10-26T07:00:04.935+0000 7fe202c8a700 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:00:05.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:05 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:05.260+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:05.760 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:05.433+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:05.760 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:05.475+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:05.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:05.517+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:06.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:05 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:05.692+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:06.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:05.758+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:06.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:06 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:06.273+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:06.778 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:06 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:06.423+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:06.779 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:06 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:06.524+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:06.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:06 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:06.559+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:07.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:06 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:06.687+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:07.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:06 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:06.777+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:07.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:07 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:07.260+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:07.800 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:07.398+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:07.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:07.552+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:07.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:07 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:07.580+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:08.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:07 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:07.725+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:08.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:07.798+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:08.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:08 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:08.225+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:08.646 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:08.375+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:08.647 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:08.513+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:08.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:08 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:08.627+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:09.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:08 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:08.751+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:09.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:08.799+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:09.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:09 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:09.265+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:09.645 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:09 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:09.368+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:09.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:09 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:09.524+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:09.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:09 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:09.582+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:10.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:09 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:09.723+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:10.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:09 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:09.777+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:10.765 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:10.308+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:10.804 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:10 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:10.401+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:10.805 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:10 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:10.547+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:10.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:10 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:10.561+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:11.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:10 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:10.764+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:11.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:10 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:10.803+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:11.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:11 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:11.545+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:11.646 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:11 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:11.363+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:11.775 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:11 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:11.306+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:11.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:11 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:11.582+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:12.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:11 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:11.773+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:12.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:11 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:11.783+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:12.645 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:12.319+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:12.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:12.529+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:12.789 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:12 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:12.336+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:12.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:12 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:12.628+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:13.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:12.773+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:13.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:12 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:12.788+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:13.645 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:13.335+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:13.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:13.491+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:13.789 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:13 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:13.316+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:13.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:13 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:13.677+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:14.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:13 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:13.781+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:14.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:13 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:13.793+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:14.645 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:14 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:14.379+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:14.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:14 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:14.474+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:14.789 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:14 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:14.302+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:14.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:14 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:14.644+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:15.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:14 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:14.771+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:15.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:14 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:14.793+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:15.645 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:15 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:15.369+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:15.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:15 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:15.448+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:15.789 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:15.288+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:15.923 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:15 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:15.657+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:16.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:15 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:15.741+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:16.217 INFO:journalctl@ceph.osd.4.smithi135.stdout:Oct 26 07:00:16 smithi135 bash[175417]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-4 2023-10-26T07:00:16.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:15 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:15.809+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:16.758 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:16.406+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:16.758 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:16.452+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:16.789 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:16.289+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:16.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:16 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:16.680+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:17.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:16 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:16.755+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:17.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:16.830+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:17.498 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.4.service' 2023-10-26T07:00:17.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:00:17.539 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-26T07:00:17.540 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-26T07:00:17.540 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.5 2023-10-26T07:00:17.743 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:17.454+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:17.744 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:17.741+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:17.745 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:17.401+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:17.745 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:17 smithi192 systemd[1]: Stopping Ceph osd.5 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:00:17.789 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:17 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:17.292+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:17.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:17 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:17.641+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:18.145 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:17.897+0000 7f4ae724e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:00:18.146 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:17.897+0000 7f4ae724e700 -1 osd.5 100 *** Got signal Terminated *** 2023-10-26T07:00:18.146 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:17 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:17.897+0000 7f4ae724e700 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:00:18.288 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:17 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:17.861+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:18.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:18 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:18.287+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:18.724 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:18 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:18.441+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:18.724 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:18 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:18.408+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:18.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:18 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:18.663+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:19.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:18 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:18.721+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:19.268 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:18 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:18.865+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:19.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:19 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:19.266+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:19.769 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:19 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:19.435+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:19.770 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:19 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:19.452+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:19.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:19 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:19.645+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:20.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:19 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:19.766+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:20.273 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:19 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:19.838+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:20.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:20 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:20.272+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:20.734 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:20 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:20.470+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:20.734 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:20 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:20.475+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:20.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:20 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:20.675+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:21.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:20 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:20.731+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:21.230 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:20 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:20.872+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:21.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:21 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:21.228+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:21.896 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:21 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:21.477+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:21.897 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:21 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:21.466+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:21.897 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:21 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:21.687+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:21.898 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:21 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:21.687+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:22.217 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:21 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:21.718+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:22.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:21 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:21.718+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:22.224 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:21 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:21.874+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:22.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:22 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:22.223+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:22.729 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:22 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5[146869]: 2023-10-26T07:00:22.475+0000 7f4ae4045700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.449497+0000 front 2023-10-26T06:58:20.449520+0000 (oldest deadline 2023-10-26T06:59:48.452911+0000) 2023-10-26T07:00:22.729 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:22 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:22.447+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:23.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:22 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:22.726+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:23.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:22 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:22.726+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:23.209 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:22 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:22.845+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:23.209 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:22 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:22.845+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:23.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:22 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:22.885+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:23.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:22 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:22.766+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:23.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:22 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:22.766+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:23.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:23 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:23.207+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:23.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:23 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:23.207+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:23.691 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:23 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:23.427+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:24.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:23 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:23.690+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:24.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:23 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:23.690+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:24.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:23 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:23.895+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:24.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:23 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:23.780+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:24.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:23 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:23.780+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:24.224 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:23 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:23.831+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:24.225 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:23 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:23.831+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:24.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:24.223+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:24.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:24.223+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:24.693 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:24 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:24.419+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:25.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:24.787+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:25.040 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:24.787+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:25.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:24 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:24.690+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:25.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:24 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:24.690+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:25.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:24 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:24.906+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:25.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:24 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:24.789+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:25.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:24 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:24.789+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:25.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:25 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:25.206+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:25.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:25 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:25.206+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:25.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:25 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:25.694+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:25.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:25 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:25.694+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:25.896 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:25 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:25.446+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:26.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:25 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:25.757+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:26.040 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:25 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:25.757+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:26.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:25 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:25.861+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:26.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:25 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:25.750+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:26.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:25 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:25.750+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:26.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:26 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:26.201+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:26.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:26 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:26.201+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:26.722 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:26 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:26.447+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:27.040 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:26 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:26.758+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:27.340 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:26 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:26.758+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:27.341 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:26 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:26.823+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:27.342 INFO:journalctl@ceph.osd.5.smithi192.stdout:Oct 26 07:00:26 smithi192 bash[176993]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-5 2023-10-26T07:00:27.342 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:26 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:26.770+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:27.342 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:26 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:26.770+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:27.343 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:26 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:26.732+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:27.343 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:26 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:26.732+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:27.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:27 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:27.225+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:27.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:27 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:27.225+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:27.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:27 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:27.481+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:28.028 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:27 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:27.731+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:28.028 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:27 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:27.731+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:28.034 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.5.service' 2023-10-26T07:00:28.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:27 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:27.774+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:28.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:27 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:27.774+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:28.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:00:28.077 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-26T07:00:28.078 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-26T07:00:28.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.6 2023-10-26T07:00:28.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:27 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:27.835+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:28.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:27 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:27.753+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:28.219 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:27 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:27.753+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:28.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:28 smithi089 systemd[1]: Stopping Ceph osd.6 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:00:28.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:28.241+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:28.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:28.241+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:28.731 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:28 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:28.475+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:29.040 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:28.607+0000 7feab7afc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:00:29.040 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:28.607+0000 7feab7afc700 -1 osd.6 100 *** Got signal Terminated *** 2023-10-26T07:00:29.040 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:28.607+0000 7feab7afc700 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:00:29.041 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:28.755+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:29.041 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:28 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:28.755+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:29.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:28 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:28.730+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:29.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:28 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:28.730+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:29.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:28 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:28.823+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:29.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:28 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:28.753+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:29.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:28 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:28.753+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:29.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:29 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:29.214+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:29.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:29 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:29.214+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:29.709 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:29 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:29.433+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:30.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:29 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:29.708+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:30.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:29 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:29.708+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:30.182 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:29 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:29.803+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:30.182 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:29 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:29.803+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:30.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:29 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:29.800+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:30.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:29 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:29.725+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:30.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:29 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:29.725+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:30.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:30 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:30.180+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:30.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:30 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:30.180+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:30.735 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:30 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:30.422+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:31.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:30 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:30.734+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:31.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:30 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:30.734+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:31.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:30 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:30.845+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:31.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:30 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:30.735+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:31.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:30 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:30.735+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:31.221 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:30 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:30.816+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:31.221 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:30 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:30.816+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:31.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:31 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:31.220+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:31.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:31 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:31.220+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:31.732 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:31 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:31.408+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:32.040 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:31 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:31.772+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:32.041 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:31 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:31.772+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:32.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:31 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:31.730+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:32.147 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:31 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:31.731+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:32.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:31 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:31.878+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:32.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:31 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:31.740+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:32.219 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:31 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:31.740+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:32.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:32 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:32.175+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:32.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:32 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:32.175+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:32.753 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:32 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:32.441+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:32.967 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:32 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:32.926+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:32.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:32 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:32.703+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:32.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:32 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:32.703+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:33.039 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:32 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:32.752+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:33.040 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:32 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:32.752+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:33.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:32 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:32.752+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:33.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:32 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:32.752+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:33.539 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:33 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:33.177+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:20.957768+0000 front 2023-10-26T06:58:20.957810+0000 (oldest deadline 2023-10-26T06:59:50.063061+0000) 2023-10-26T07:00:33.540 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:33 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6[133696]: 2023-10-26T07:00:33.177+0000 7feab08eb700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.864626+0000 front 2023-10-26T06:58:55.864683+0000 (oldest deadline 2023-10-26T07:00:23.168762+0000) 2023-10-26T07:00:33.731 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:33 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:33.410+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:33.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:33 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:33.667+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:33.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:33 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:33.667+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:34.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:33 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:33.729+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:34.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:33 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:33.729+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:34.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:33 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:33.796+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:34.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:33 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:33.796+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:34.467 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:33 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:33.968+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:34.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:34 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:34.377+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:34.789 INFO:journalctl@ceph.osd.6.smithi089.stdout:Oct 26 07:00:34 smithi089 bash[179562]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-6 2023-10-26T07:00:34.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:34 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:34.715+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:34.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:34 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:34.715+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:35.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:34 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:34.740+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:35.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:34 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:34.740+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:35.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:34 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:34.966+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:35.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:34 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:34.835+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:35.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:34 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:34.835+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:35.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:35 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:35.369+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:35.924 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:35 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:35.668+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:35.925 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:35 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:35.668+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:36.120 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:35 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:35.866+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:36.120 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:35 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:35.866+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:36.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:35 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:35.779+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:36.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:35 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:35.779+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:36.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:35 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:35.923+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:36.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.6.service' 2023-10-26T07:00:36.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:00:36.511 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-26T07:00:36.511 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-26T07:00:36.512 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.7 2023-10-26T07:00:36.766 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:36 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:36.397+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:36.874 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:36 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:36.663+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:36.875 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:36 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:36.663+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:36.876 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:36 smithi135 systemd[1]: Stopping Ceph osd.7 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:00:36.876 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:36 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:36.873+0000 7f6443861700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:00:37.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:36 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:36.765+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:37.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:36 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:36.765+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:37.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:36 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:36.873+0000 7f6443861700 -1 osd.7 100 *** Got signal Terminated *** 2023-10-26T07:00:37.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:36 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:36.873+0000 7f6443861700 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:00:37.218 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:36 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:36.925+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:37.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:36 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:36.861+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:37.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:36 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:36.861+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:37.726 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:37 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:37.398+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:37.913 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:37 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:37.643+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:37.913 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:37 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:37.643+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:38.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:37 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:37.725+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:38.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:37 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:37.725+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:38.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:37 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:37.912+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:38.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:37 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:37.859+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:38.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:37 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:37.859+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:38.750 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:38 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:38.425+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:38.967 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:38 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:38.911+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:38.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:38 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:38.679+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:38.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:38 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:38.679+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:39.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:38 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:38.749+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:39.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:38 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:38.749+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:39.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:38 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:38.895+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:39.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:38 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:38.895+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:39.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:39 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:39.380+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:40.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:39 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:39.716+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:40.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:39 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:39.716+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:40.217 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:39 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:39.901+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:40.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:39 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:39.728+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:40.218 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:39 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:39.728+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:40.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:39 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:39.893+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:40.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:39 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:39.893+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:40.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:40 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:40.346+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:40.967 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:40 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:40.915+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:40.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:40 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:40.699+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:40.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:40 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:40.699+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:41.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:40 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:40.761+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:41.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:40 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:40.761+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:41.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:40 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:40.936+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:41.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:40 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:40.936+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:41.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:41 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:41.345+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:41.967 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:41 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7[155039]: 2023-10-26T07:00:41.868+0000 7f643e654700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:55.079555+0000 front 2023-10-26T06:58:55.079572+0000 (oldest deadline 2023-10-26T07:00:22.184458+0000) 2023-10-26T07:00:41.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:41 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:41.673+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:41.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:41 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:41.673+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:42.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:41 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:41.714+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:42.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:41 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:41.714+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:42.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:41 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:41.909+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:42.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:41 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:41.909+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:42.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:42 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:42.390+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:42.967 INFO:journalctl@ceph.osd.7.smithi135.stdout:Oct 26 07:00:42 smithi135 bash[175690]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-7 2023-10-26T07:00:42.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:42 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:42.669+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:42.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:42 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:42.669+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:43.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:42 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:42.689+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:43.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:42 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:42.689+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:43.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:42 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:42.898+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:43.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:42 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:42.898+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:43.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:43 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:43.357+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:43.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:43 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:43.639+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:43.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:43 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:43.639+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:44.019 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.7.service' 2023-10-26T07:00:44.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:00:44.060 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-26T07:00:44.060 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-26T07:00:44.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.8 2023-10-26T07:00:44.091 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:43 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:43.719+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:44.092 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:43 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:43.719+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:44.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:43 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:43.924+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:44.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:43 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:43.924+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:44.373 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:44 smithi192 systemd[1]: Stopping Ceph osd.8 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:00:44.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:44 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:44.371+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:44.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:44 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:44.411+0000 7ff7e5196700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:00:44.646 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:44 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:44.411+0000 7ff7e5196700 -1 osd.8 100 *** Got signal Terminated *** 2023-10-26T07:00:44.647 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:44 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:44.411+0000 7ff7e5196700 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:00:44.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:44 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:44.637+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:44.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:44 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:44.637+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:45.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:44 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:44.699+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:45.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:44 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:44.699+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:45.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:44 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:44.877+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:45.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:44 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:44.877+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:45.701 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:45 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:45.410+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:45.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:45 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:45.608+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:45.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:45 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:45.608+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:46.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:45 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:45.700+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:46.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:45 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:45.700+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:46.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:45 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:45.916+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:46.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:45 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:45.916+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:46.711 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:46 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:46.414+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:46.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:46 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:46.563+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:46.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:46 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:46.563+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:47.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:46 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:46.710+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:47.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:46 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:46.710+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:47.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:46 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:46.905+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:47.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:46 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:46.905+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:47.712 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:47 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:47.409+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:47.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:47 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:47.599+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:47.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:47 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:47.599+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:48.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:47 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:47.711+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:48.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:47 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:47.711+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:48.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:47 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:47.856+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:48.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:47 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:47.856+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:48.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:48 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:48.380+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:48.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:48 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:48.616+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:48.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:48 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:48.616+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:49.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:48 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:48.686+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:49.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:48 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:48.686+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:49.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:48 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:48.820+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:49.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:48 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:48.820+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:49.645 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:49 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8[155665]: 2023-10-26T07:00:49.374+0000 7ff7df788700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.813138+0000 front 2023-10-26T06:58:19.813061+0000 (oldest deadline 2023-10-26T06:59:52.516560+0000) 2023-10-26T07:00:49.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:49 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:49.626+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:49.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:49 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:49.626+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:50.010 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:49 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:49.680+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:50.011 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:49 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:49.680+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:50.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:49 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:49.845+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:50.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:49 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:49.845+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:50.300 INFO:journalctl@ceph.osd.8.smithi192.stdout:Oct 26 07:00:50 smithi192 bash[177261]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-8 2023-10-26T07:00:50.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:50 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:50.636+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:50.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:50 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:50.636+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:50.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:50 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:50.602+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:50.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:50 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:50.602+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:51.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:50 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:50.833+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:51.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:50 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:50.833+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:51.661 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.8.service' 2023-10-26T07:00:51.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:00:51.743 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-26T07:00:51.743 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-26T07:00:51.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.9 2023-10-26T07:00:51.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:51 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:51.581+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:51.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:51 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:51.581+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:52.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:51 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:51.677+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:52.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:51 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:51.677+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:52.271 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:51 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:51.797+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:52.272 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:51 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:51.797+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:52.272 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:51 smithi089 systemd[1]: Stopping Ceph osd.9 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:00:52.539 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:52 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:52.269+0000 7f1d5db8f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:00:52.540 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:52 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:52.269+0000 7f1d5db8f700 -1 osd.9 100 *** Got signal Terminated *** 2023-10-26T07:00:52.540 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:52 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:52.269+0000 7f1d5db8f700 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:00:52.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:52 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:52.572+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:52.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:52 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:52.572+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:53.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:52 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:52.677+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:53.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:52 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:52.677+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:53.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:52 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:52.841+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:53.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:52 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:52.841+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:53.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:53 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:53.629+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:53.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:53 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:53.629+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:53.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:53 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:53.548+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:53.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:53 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:53.548+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:54.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:53.839+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:54.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:53 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:53.839+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:54.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:54 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:54.619+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:54.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:54 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:54.619+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:54.970 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:54 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:54.504+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:54.970 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:54 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:54.504+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:55.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:54.806+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:55.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:54.806+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:55.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:54 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:54.806+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.980344+0000 front 2023-10-26T06:59:27.980360+0000 (oldest deadline 2023-10-26T07:00:54.784051+0000) 2023-10-26T07:00:55.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:55 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:55.523+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:55.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:55 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:55.523+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:55.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:55 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:55.523+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:00:56.040 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:55.757+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:56.041 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:55.757+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:56.041 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:55 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:55.757+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.980344+0000 front 2023-10-26T06:59:27.980360+0000 (oldest deadline 2023-10-26T07:00:54.784051+0000) 2023-10-26T07:00:56.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:55.664+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:56.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:55.664+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:56.147 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:55.664+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:00:56.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:56 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:56.545+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:56.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:56 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:56.545+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:56.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:56 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:56.545+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:00:57.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:56 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:56.650+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:57.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:56 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:56.650+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:57.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:56 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:56.650+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:00:57.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:56.794+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.869053+0000 front 2023-10-26T06:58:22.868990+0000 (oldest deadline 2023-10-26T06:59:52.073344+0000) 2023-10-26T07:00:57.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:56.794+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.974141+0000 front 2023-10-26T06:58:54.974168+0000 (oldest deadline 2023-10-26T07:00:22.079545+0000) 2023-10-26T07:00:57.290 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:56 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9[142386]: 2023-10-26T07:00:56.794+0000 7f1d5697e700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.980344+0000 front 2023-10-26T06:59:27.980360+0000 (oldest deadline 2023-10-26T07:00:54.784051+0000) 2023-10-26T07:00:57.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:57.623+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:57.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:57.623+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:57.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:57 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:57.623+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:00:57.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:57 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:57.501+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:57.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:57 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:57.501+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:57.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:57 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:57.501+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:00:58.539 INFO:journalctl@ceph.osd.9.smithi089.stdout:Oct 26 07:00:58 smithi089 bash[179835]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-9 2023-10-26T07:00:58.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:58.619+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:58.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:58.619+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:58.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:58.619+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:00:58.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:58 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:58.479+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:58.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:58 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:58.479+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:58.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:58 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:58.479+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:00:59.717 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:59 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:59.454+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:00:59.718 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:59 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:59.454+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:00:59.718 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:00:59 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:00:59.454+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:00:59.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:59 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:59.578+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:00:59.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:59 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:59.578+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:00:59.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:00:59 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:00:59.578+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:00.098 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.9.service' 2023-10-26T07:01:00.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:00.191 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-26T07:01:00.191 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-26T07:01:00.191 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.10 2023-10-26T07:01:00.546 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:00 smithi135 systemd[1]: Stopping Ceph osd.10 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:00.547 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:00 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:00.406+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:01:00.547 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:00 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:00.406+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:01:00.547 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:00 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:00.406+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:01:00.547 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:00 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:00.544+0000 7f4c232a4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:00.548 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:00 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:00.544+0000 7f4c232a4700 -1 osd.10 100 *** Got signal Terminated *** 2023-10-26T07:01:00.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:00 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:00.561+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:00.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:00 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:00.561+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:00.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:00 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:00.561+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:00.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:00 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:00.544+0000 7f4c232a4700 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:01:01.696 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:01 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:01.439+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:01:01.696 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:01 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:01.439+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:01:01.696 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:01 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:01.439+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:01:01.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:01 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:01.543+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:01.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:01 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:01.543+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:01.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:01 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:01.543+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:02.717 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:02 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:02.453+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:01:02.718 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:02 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:02.453+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:01:02.718 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:02 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:02.453+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:01:02.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:02 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:02.579+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:02.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:02 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:02.579+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:02.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:02 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:02.579+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:03.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:03 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:03.597+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:03.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:03 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:03.597+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:03.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:03 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:03.597+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:03.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:03 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:03.492+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:01:03.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:03 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:03.492+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:01:03.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:03 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:03.492+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:01:04.895 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:04 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:04.641+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:04.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:04 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:04.641+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:04.896 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:04 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:04.641+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:04.967 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:04 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:04.526+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:01:04.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:04 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:04.526+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:01:04.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:04 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:04.526+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:01:05.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:05.485+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:19.577028+0000 front 2023-10-26T06:58:19.577017+0000 (oldest deadline 2023-10-26T06:59:48.680876+0000) 2023-10-26T07:01:05.968 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:05.486+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:54.581673+0000 front 2023-10-26T06:58:54.581691+0000 (oldest deadline 2023-10-26T07:00:21.385499+0000) 2023-10-26T07:01:05.969 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:05.486+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:27.286240+0000 front 2023-10-26T06:59:27.286321+0000 (oldest deadline 2023-10-26T07:00:54.788949+0000) 2023-10-26T07:01:05.969 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:05 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10[163196]: 2023-10-26T07:01:05.486+0000 7f4c1e097700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2023-10-26T06:59:59.489939+0000 front 2023-10-26T06:59:59.489960+0000 (oldest deadline 2023-10-26T07:01:05.390012+0000) 2023-10-26T07:01:06.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:05.658+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:06.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:05.658+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:06.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:05 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:05.658+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:06.524 INFO:journalctl@ceph.osd.10.smithi135.stdout:Oct 26 07:01:06 smithi135 bash[175958]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-10 2023-10-26T07:01:07.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:06 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:06.673+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:07.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:06 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:06.673+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:07.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:06 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:06.673+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:07.783 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.10.service' 2023-10-26T07:01:07.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:07.824 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-26T07:01:07.825 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-26T07:01:07.825 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.11 2023-10-26T07:01:08.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:07.706+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:08.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:07.706+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:08.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:07 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:07.706+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:08.147 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:07 smithi192 systemd[1]: Stopping Ceph osd.11 for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:08.645 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:08.181+0000 7fddc993c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:08.646 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:08.181+0000 7fddc993c700 -1 osd.11 100 *** Got signal Terminated *** 2023-10-26T07:01:08.646 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:08.181+0000 7fddc993c700 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-26T07:01:09.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:08.667+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:09.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:08.667+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:09.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:08 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:08.667+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:10.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:09 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:09.670+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:10.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:09 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:09.670+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:10.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:09 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:09.670+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:11.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:10 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:10.705+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:11.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:10 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:10.705+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:11.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:10 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:10.705+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:12.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:11 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:11.719+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:12.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:11 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:11.719+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:12.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:11 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:11.719+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:13.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:12.727+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2023-10-26T06:58:22.565630+0000 front 2023-10-26T06:58:22.565795+0000 (oldest deadline 2023-10-26T06:59:51.168615+0000) 2023-10-26T07:01:13.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:12.727+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2023-10-26T06:58:51.169134+0000 front 2023-10-26T06:58:51.169123+0000 (oldest deadline 2023-10-26T07:00:21.472499+0000) 2023-10-26T07:01:13.146 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:12 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11[164354]: 2023-10-26T07:01:12.727+0000 7fddc272b700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2023-10-26T06:59:26.773348+0000 front 2023-10-26T06:59:26.773368+0000 (oldest deadline 2023-10-26T07:00:54.776495+0000) 2023-10-26T07:01:14.396 INFO:journalctl@ceph.osd.11.smithi192.stdout:Oct 26 07:01:14 smithi192 bash[177539]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-osd-11 2023-10-26T07:01:15.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@osd.11.service' 2023-10-26T07:01:15.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:15.568 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-26T07:01:15.569 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-26T07:01:15.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.a 2023-10-26T07:01:16.039 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 07:01:15 smithi089 systemd[1]: Stopping Ceph mds.a for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:16.539 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 07:01:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-a[150923]: 2023-10-26T07:01:16.108+0000 7f3842ce7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:16.540 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 07:01:16 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-a[150923]: 2023-10-26T07:01:16.108+0000 7f3842ce7700 -1 mds.a *** got signal Terminated *** 2023-10-26T07:01:20.539 INFO:journalctl@ceph.mds.a.smithi089.stdout:Oct 26 07:01:20 smithi089 bash[180116]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-a 2023-10-26T07:01:20.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.a.service' 2023-10-26T07:01:20.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:20.851 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-26T07:01:20.852 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-26T07:01:20.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.d 2023-10-26T07:01:21.039 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 07:01:20 smithi089 systemd[1]: Stopping Ceph mds.d for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:21.789 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 07:01:21 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-d[149923]: 2023-10-26T07:01:21.366+0000 7f45bc033700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:21.789 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 07:01:21 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-d[149923]: 2023-10-26T07:01:21.366+0000 7f45bc033700 -1 mds.d *** got signal Terminated *** 2023-10-26T07:01:23.789 INFO:journalctl@ceph.mds.d.smithi089.stdout:Oct 26 07:01:23 smithi089 bash[180273]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-d 2023-10-26T07:01:24.041 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.d.service' 2023-10-26T07:01:24.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:24.128 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-26T07:01:24.128 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-26T07:01:24.129 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.g 2023-10-26T07:01:24.289 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 07:01:24 smithi089 systemd[1]: Stopping Ceph mds.g for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:25.039 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 07:01:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-g[150536]: 2023-10-26T07:01:24.631+0000 7f48e55fd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:25.040 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 07:01:24 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-g[150536]: 2023-10-26T07:01:24.631+0000 7f48e55fd700 -1 mds.g *** got signal Terminated *** 2023-10-26T07:01:26.285 INFO:journalctl@ceph.mds.g.smithi089.stdout:Oct 26 07:01:25 smithi089 bash[180420]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-g 2023-10-26T07:01:26.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.g.service' 2023-10-26T07:01:26.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:26.396 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-26T07:01:26.397 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-26T07:01:26.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.j 2023-10-26T07:01:26.789 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 07:01:26 smithi089 systemd[1]: Stopping Ceph mds.j for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:27.289 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 07:01:26 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-j[149539]: 2023-10-26T07:01:26.889+0000 7f87fd540700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:27.290 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 07:01:26 smithi089 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-j[149539]: 2023-10-26T07:01:26.889+0000 7f87fd540700 -1 mds.j *** got signal Terminated *** 2023-10-26T07:01:28.727 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 07:01:28 smithi089 bash[180565]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-j 2023-10-26T07:01:28.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.j.service' 2023-10-26T07:01:29.024 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 07:01:28 smithi089 systemd[1]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.j.service: Succeeded. 2023-10-26T07:01:29.025 INFO:journalctl@ceph.mds.j.smithi089.stdout:Oct 26 07:01:28 smithi089 systemd[1]: Stopped Ceph mds.j for e22b2d1a-73c9-11ee-8db9-212e2dc638e7. 2023-10-26T07:01:29.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:29.046 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-26T07:01:29.047 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-26T07:01:29.047 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.b 2023-10-26T07:01:29.409 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 07:01:29 smithi135 systemd[1]: Stopping Ceph mds.b for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:29.717 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 07:01:29 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b[170212]: 2023-10-26T07:01:29.406+0000 7f3f33235700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:29.718 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 07:01:29 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b[170212]: 2023-10-26T07:01:29.406+0000 7f3f33235700 -1 mds.b *** got signal Terminated *** 2023-10-26T07:01:33.898 INFO:journalctl@ceph.mds.b.smithi135.stdout:Oct 26 07:01:33 smithi135 bash[176234]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-b 2023-10-26T07:01:33.904 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.b.service' 2023-10-26T07:01:33.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:33.940 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-26T07:01:33.940 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-26T07:01:33.940 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.e 2023-10-26T07:01:34.217 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 07:01:34 smithi135 systemd[1]: Stopping Ceph mds.e for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:34.717 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 07:01:34 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-e[169529]: 2023-10-26T07:01:34.293+0000 7f09a75aa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:34.718 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 07:01:34 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-e[169529]: 2023-10-26T07:01:34.293+0000 7f09a75aa700 -1 mds.e *** got signal Terminated *** 2023-10-26T07:01:38.117 INFO:journalctl@ceph.mds.e.smithi135.stdout:Oct 26 07:01:37 smithi135 bash[176382]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-e 2023-10-26T07:01:38.123 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.e.service' 2023-10-26T07:01:38.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:38.162 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-26T07:01:38.162 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-26T07:01:38.163 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.h 2023-10-26T07:01:38.467 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 07:01:38 smithi135 systemd[1]: Stopping Ceph mds.h for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:38.967 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 07:01:38 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-h[169869]: 2023-10-26T07:01:38.518+0000 7f4f62662700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:38.967 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 07:01:38 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-h[169869]: 2023-10-26T07:01:38.518+0000 7f4f62662700 -1 mds.h *** got signal Terminated *** 2023-10-26T07:01:40.717 INFO:journalctl@ceph.mds.h.smithi135.stdout:Oct 26 07:01:40 smithi135 bash[176534]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-h 2023-10-26T07:01:40.833 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.h.service' 2023-10-26T07:01:40.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:40.867 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-26T07:01:40.867 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-26T07:01:40.868 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.k 2023-10-26T07:01:41.217 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 07:01:40 smithi135 systemd[1]: Stopping Ceph mds.k for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:41.717 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 07:01:41 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-k[169183]: 2023-10-26T07:01:41.237+0000 7f883a90d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:41.718 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 07:01:41 smithi135 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-k[169183]: 2023-10-26T07:01:41.237+0000 7f883a90d700 -1 mds.k *** got signal Terminated *** 2023-10-26T07:01:45.691 INFO:journalctl@ceph.mds.k.smithi135.stdout:Oct 26 07:01:45 smithi135 bash[176685]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-k 2023-10-26T07:01:45.783 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.k.service' 2023-10-26T07:01:45.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:45.819 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-26T07:01:45.819 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-26T07:01:45.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.c 2023-10-26T07:01:46.146 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 07:01:45 smithi192 systemd[1]: Stopping Ceph mds.c for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:46.645 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 07:01:46 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-c[171243]: 2023-10-26T07:01:46.201+0000 7f45b60d7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:46.646 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 07:01:46 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-c[171243]: 2023-10-26T07:01:46.201+0000 7f45b60d7700 -1 mds.c *** got signal Terminated *** 2023-10-26T07:01:50.349 INFO:journalctl@ceph.mds.c.smithi192.stdout:Oct 26 07:01:50 smithi192 bash[177810]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-c 2023-10-26T07:01:50.539 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.c.service' 2023-10-26T07:01:50.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:50.575 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-26T07:01:50.576 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-26T07:01:50.577 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.f 2023-10-26T07:01:50.895 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 07:01:50 smithi192 systemd[1]: Stopping Ceph mds.f for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:51.395 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 07:01:50 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-f[170903]: 2023-10-26T07:01:50.950+0000 7f55dd22e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:51.396 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 07:01:50 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-f[170903]: 2023-10-26T07:01:50.950+0000 7f55dd22e700 -1 mds.f *** got signal Terminated *** 2023-10-26T07:01:55.395 INFO:journalctl@ceph.mds.f.smithi192.stdout:Oct 26 07:01:55 smithi192 bash[177953]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-f 2023-10-26T07:01:55.550 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.f.service' 2023-10-26T07:01:55.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:55.586 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-26T07:01:55.587 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-26T07:01:55.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.i 2023-10-26T07:01:55.897 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 07:01:55 smithi192 systemd[1]: Stopping Ceph mds.i for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:56.395 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 07:01:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-i[171581]: 2023-10-26T07:01:55.964+0000 7f3d8a8cc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:56.396 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 07:01:55 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-i[171581]: 2023-10-26T07:01:55.964+0000 7f3d8a8cc700 -1 mds.i *** got signal Terminated *** 2023-10-26T07:01:58.409 INFO:journalctl@ceph.mds.i.smithi192.stdout:Oct 26 07:01:58 smithi192 bash[178096]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-i 2023-10-26T07:01:58.414 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.i.service' 2023-10-26T07:01:58.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:01:58.449 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-26T07:01:58.449 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-26T07:01:58.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.l 2023-10-26T07:01:58.834 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 07:01:58 smithi192 systemd[1]: Stopping Ceph mds.l for e22b2d1a-73c9-11ee-8db9-212e2dc638e7... 2023-10-26T07:01:59.145 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 07:01:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-l[170562]: 2023-10-26T07:01:58.831+0000 7fdff4cb8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-26T07:01:59.146 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 07:01:58 smithi192 ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-l[170562]: 2023-10-26T07:01:58.831+0000 7fdff4cb8700 -1 mds.l *** got signal Terminated *** 2023-10-26T07:02:03.287 INFO:journalctl@ceph.mds.l.smithi192.stdout:Oct 26 07:02:03 smithi192 bash[178239]: ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7-mds-l 2023-10-26T07:02:03.429 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e22b2d1a-73c9-11ee-8db9-212e2dc638e7@mds.l.service' 2023-10-26T07:02:03.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-26T07:02:03.464 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-26T07:02:03.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 --force --keep-logs 2023-10-26T07:02:03.864 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T07:02:07.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 --force --keep-logs 2023-10-26T07:02:07.405 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T07:02:08.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 --force --keep-logs 2023-10-26T07:02:09.276 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T07:02:10.850 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-26T07:02:10.930 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-26T07:02:10.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-26T07:02:10.997 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-26T07:02:10.999 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi089/crash 2023-10-26T07:02:11.000 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash -- . 2023-10-26T07:02:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-26T07:02:11.073 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2023-10-26T07:02:11.076 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi135/crash 2023-10-26T07:02:11.078 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash -- . 2023-10-26T07:02:11.122 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-26T07:02:11.123 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2023-10-26T07:02:11.124 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi192/crash 2023-10-26T07:02:11.125 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash -- . 2023-10-26T07:02:11.162 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-26T07:02:11.162 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2023-10-26T07:02:11.163 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-26T07:02:11.164 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-26T07:02:11.245 INFO:teuthology.orchestra.run.smithi089.stderr:grep: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph.log: No such file or directory 2023-10-26T07:02:11.250 INFO:tasks.cephadm:Compressing logs... 2023-10-26T07:02:11.250 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-26T07:02:11.255 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-26T07:02:11.258 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-26T07:02:11.285 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-26T07:02:11.285 INFO:teuthology.orchestra.run.smithi192.stderr:: No such file or directory 2023-10-26T07:02:11.285 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.5.log 2023-10-26T07:02:11.286 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.2.log 2023-10-26T07:02:11.288 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.11.log 2023-10-26T07:02:11.288 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.z.log 2023-10-26T07:02:11.288 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.8.log 2023-10-26T07:02:11.289 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.2.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.11.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.z.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-26T07:02:11.289 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-26T07:02:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:: No such file or directory 2023-10-26T07:02:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.10.log 2023-10-26T07:02:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.h.log 2023-10-26T07:02:11.290 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.e.log 2023-10-26T07:02:11.291 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.4.log 2023-10-26T07:02:11.291 INFO:teuthology.orchestra.run.smithi135.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-26T07:02:11.293 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.8.log: 92.7% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-26T07:02:11.293 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.f.log 2023-10-26T07:02:11.293 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.c.log 2023-10-26T07:02:11.293 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.i.log 2023-10-26T07:02:11.295 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.1.log 2023-10-26T07:02:11.295 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.k.log 2023-10-26T07:02:11.295 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.4.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log 2023-10-26T07:02:11.295 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.k.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log: 94.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.h.log.gz 2023-10-26T07:02:11.296 INFO:teuthology.orchestra.run.smithi135.stderr: 94.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.e.log.gz 2023-10-26T07:02:11.296 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.7.log 2023-10-26T07:02:11.296 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.b.log 2023-10-26T07:02:11.297 INFO:teuthology.orchestra.run.smithi135.stderr: 94.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.k.log.gz 2023-10-26T07:02:11.297 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.c.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.i.log: 94.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.f.log.gz 2023-10-26T07:02:11.298 INFO:teuthology.orchestra.run.smithi192.stderr: 94.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.c.log.gz 2023-10-26T07:02:11.298 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.7.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.b.log 2023-10-26T07:02:11.298 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.c.log 2023-10-26T07:02:11.299 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log 2023-10-26T07:02:11.300 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.c.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.l.log 2023-10-26T07:02:11.301 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.b.log: 91.3% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log.gz 2023-10-26T07:02:11.305 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.y.log 2023-10-26T07:02:11.306 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.l.log: 94.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.l.log.gz 2023-10-26T07:02:11.311 INFO:teuthology.orchestra.run.smithi192.stderr: 91.3% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log.gz 2023-10-26T07:02:11.313 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.y.log: 92.7% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-26T07:02:11.317 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-26T07:02:11.318 INFO:teuthology.orchestra.run.smithi089.stderr:: No such file or directory 2023-10-26T07:02:11.318 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.0.log 2023-10-26T07:02:11.319 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.g.log 2023-10-26T07:02:11.321 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.a.log 2023-10-26T07:02:11.322 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.g.log: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-26T07:02:11.322 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.6.log 2023-10-26T07:02:11.323 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.3.log 2023-10-26T07:02:11.324 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.d.log 2023-10-26T07:02:11.326 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.3.log: 94.9% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.g.log.gz 2023-10-26T07:02:11.326 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.x.log 2023-10-26T07:02:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.9.log 2023-10-26T07:02:11.328 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.x.log: 94.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.a.log.gz 2023-10-26T07:02:11.332 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log 2023-10-26T07:02:11.332 INFO:teuthology.orchestra.run.smithi089.stderr: 94.7% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.d.log.gz 2023-10-26T07:02:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.j.log 2023-10-26T07:02:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log: /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.a.log 2023-10-26T07:02:11.342 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.a.log: 94.9% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.j.log.gz 2023-10-26T07:02:11.347 INFO:teuthology.orchestra.run.smithi089.stderr: 91.2% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-volume.log.gz 2023-10-26T07:02:11.620 INFO:teuthology.orchestra.run.smithi192.stderr: 91.9% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.c.log.gz 2023-10-26T07:02:11.647 INFO:teuthology.orchestra.run.smithi089.stderr: 89.9% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-26T07:02:11.770 INFO:teuthology.orchestra.run.smithi135.stderr: 91.9% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.b.log.gz 2023-10-26T07:02:12.128 INFO:teuthology.orchestra.run.smithi192.stderr: 95.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.i.log.gz 2023-10-26T07:02:12.275 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mon.a.log.gz 2023-10-26T07:02:16.917 INFO:teuthology.orchestra.run.smithi089.stderr: 93.2% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.9.log.gz 2023-10-26T07:02:17.931 INFO:teuthology.orchestra.run.smithi135.stderr: 94.0% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-mds.b.log.gz 2023-10-26T07:02:18.138 INFO:teuthology.orchestra.run.smithi135.stderr: 93.5% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.4.log.gz 2023-10-26T07:02:18.240 INFO:teuthology.orchestra.run.smithi192.stderr: 93.2% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.8.log.gz 2023-10-26T07:02:18.329 INFO:teuthology.orchestra.run.smithi192.stderr: 93.2% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.2.log.gz 2023-10-26T07:02:18.519 INFO:teuthology.orchestra.run.smithi135.stderr: 93.2% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.10.log.gz 2023-10-26T07:02:20.197 INFO:teuthology.orchestra.run.smithi192.stderr: 93.1% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.5.log.gz 2023-10-26T07:02:20.372 INFO:teuthology.orchestra.run.smithi089.stderr: 93.1% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.6.log.gz 2023-10-26T07:02:20.582 INFO:teuthology.orchestra.run.smithi135.stderr: 93.1% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.1.log.gz 2023-10-26T07:02:21.442 INFO:teuthology.orchestra.run.smithi192.stderr: 93.1% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.11.log.gz 2023-10-26T07:02:21.444 INFO:teuthology.orchestra.run.smithi192.stderr: 2023-10-26T07:02:21.444 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m10.170s 2023-10-26T07:02:21.444 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m32.978s 2023-10-26T07:02:21.445 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m1.242s 2023-10-26T07:02:21.479 INFO:teuthology.orchestra.run.smithi089.stderr: 93.0% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.0.log.gz 2023-10-26T07:02:22.737 INFO:teuthology.orchestra.run.smithi089.stderr: 92.8% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.3.log.gz 2023-10-26T07:02:22.741 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-10-26T07:02:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m11.450s 2023-10-26T07:02:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m34.839s 2023-10-26T07:02:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m2.740s 2023-10-26T07:02:24.090 INFO:teuthology.orchestra.run.smithi135.stderr: 93.2% -- replaced with /var/log/ceph/e22b2d1a-73c9-11ee-8db9-212e2dc638e7/ceph-osd.7.log.gz 2023-10-26T07:02:24.092 INFO:teuthology.orchestra.run.smithi135.stderr: 2023-10-26T07:02:24.092 INFO:teuthology.orchestra.run.smithi135.stderr:real 0m12.818s 2023-10-26T07:02:24.093 INFO:teuthology.orchestra.run.smithi135.stderr:user 0m41.801s 2023-10-26T07:02:24.093 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m1.471s 2023-10-26T07:02:24.094 INFO:tasks.cephadm:Archiving logs... 2023-10-26T07:02:24.094 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi089/log 2023-10-26T07:02:24.096 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-26T07:02:29.338 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi135/log 2023-10-26T07:02:29.340 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-26T07:02:35.154 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi192/log 2023-10-26T07:02:35.156 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-26T07:02:40.154 INFO:tasks.cephadm:Removing cluster... 2023-10-26T07:02:40.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 --force 2023-10-26T07:02:40.561 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T07:02:41.440 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 --force 2023-10-26T07:02:41.746 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T07:02:42.278 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e22b2d1a-73c9-11ee-8db9-212e2dc638e7 --force 2023-10-26T07:02:42.590 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: e22b2d1a-73c9-11ee-8db9-212e2dc638e7 2023-10-26T07:02:43.205 INFO:tasks.cephadm:Removing cephadm ... 2023-10-26T07:02:43.205 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-26T07:02:43.254 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-26T07:02:43.277 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-26T07:02:43.295 INFO:tasks.cephadm:Teardown complete 2023-10-26T07:02:43.296 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-26T07:02:43.307 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-26T07:02:43.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-26T07:02:43.313 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-26T07:02:43.320 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-26T07:02:43.462 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-26T07:02:43.473 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-26T07:02:43.474 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T07:02:43.479 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T07:02:43.481 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T07:02:43.497 INFO:teuthology.orchestra.run.smithi135.stderr:bash: ntpq: command not found 2023-10-26T07:02:43.497 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2023-10-26T07:02:43.501 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T07:02:43.501 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-10-26T07:02:43.502 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 +712us[ +699us] +/- 83ms 2023-10-26T07:02:43.502 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 7 377 34 -522us[ -535us] +/- 57ms 2023-10-26T07:02:43.502 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +122us[ +109us] +/- 78ms 2023-10-26T07:02:43.502 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T07:02:43.504 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T07:02:43.504 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2023-10-26T07:02:43.505 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 38 +711us[ +704us] +/- 83ms 2023-10-26T07:02:43.505 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv02.front.sepia.ceph.com 3 7 377 37 -470us[ -477us] +/- 57ms 2023-10-26T07:02:43.505 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +116us[ +116us] +/- 78ms 2023-10-26T07:02:43.505 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T07:02:43.516 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-10-26T07:02:43.524 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T07:02:43.524 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-10-26T07:02:43.525 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +701us[ +702us] +/- 83ms 2023-10-26T07:02:43.525 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -541us[ -540us] +/- 56ms 2023-10-26T07:02:43.525 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +120us[ +120us] +/- 78ms 2023-10-26T07:02:43.526 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T07:02:43.527 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-26T07:02:43.538 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-26T07:02:43.539 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-26T07:02:43.560 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-26T07:02:43.737 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-26T07:02:43.838 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-26T07:02:43.930 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T07:02:44.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T07:02:44.007 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T07:02:44.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T07:02:44.042 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T07:02:44.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T07:02:44.075 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-26T07:02:44.087 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-26T07:02:44.120 INFO:teuthology.task.internal:Duration was 2649.950932 seconds 2023-10-26T07:02:44.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-26T07:02:44.164 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-26T07:02:44.164 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-26T07:02:44.170 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-26T07:02:44.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-26T07:02:44.218 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T07:02:44.219 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T07:02:44.282 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T07:02:44.658 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-26T07:02:44.659 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2023-10-26T07:02:44.659 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-26T07:02:44.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2023-10-26T07:02:44.719 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-26T07:02:44.744 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2023-10-26T07:02:44.745 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-26T07:02:44.768 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-26T07:02:44.769 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-26T07:02:44.774 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-26T07:02:44.787 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-26T07:02:44.871 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-26T07:02:44.872 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-26T07:02:44.876 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-26T07:02:44.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-26T07:02:45.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-26T07:02:45.049 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-26T07:02:45.050 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-26T07:02:45.119 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-26T07:02:45.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-26T07:02:45.175 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-26T07:02:45.187 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-26T07:02:45.192 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-26T07:02:45.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-26T07:02:45.219 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2023-10-26T07:02:45.241 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2023-10-26T07:02:45.255 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2023-10-26T07:02:45.292 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-26T07:02:45.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T07:02:45.334 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-26T07:02:45.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T07:02:45.351 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-26T07:02:45.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T07:02:45.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-26T07:02:45.380 INFO:teuthology.task.internal:Transferring archived files... 2023-10-26T07:02:45.382 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi089 2023-10-26T07:02:45.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-26T07:02:45.468 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi135 2023-10-26T07:02:45.468 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-26T07:02:45.510 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471/remote/smithi192 2023-10-26T07:02:45.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-26T07:02:45.549 INFO:teuthology.task.internal:Removing archive directory... 2023-10-26T07:02:45.550 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-26T07:02:45.555 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-26T07:02:45.557 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-26T07:02:45.600 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-26T07:02:45.614 INFO:teuthology.task.internal:Not uploading archives. 2023-10-26T07:02:45.614 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-26T07:02:45.644 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-26T07:02:45.645 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-26T07:02:45.650 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-26T07:02:45.652 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-26T07:02:45.666 INFO:teuthology.orchestra.run.smithi135.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 26 07:02 /home/ubuntu/cephtest 2023-10-26T07:02:45.668 INFO:teuthology.orchestra.run.smithi192.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 26 07:02 /home/ubuntu/cephtest 2023-10-26T07:02:45.688 INFO:teuthology.orchestra.run.smithi089.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 26 07:02 /home/ubuntu/cephtest 2023-10-26T07:02:45.691 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-26T07:02:45.705 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-26T07:02:45.811 INFO:teuthology.nuke:Checking targets against current locks 2023-10-26T07:02:45.837 DEBUG:teuthology.nuke:shortname: smithi089 2023-10-26T07:02:45.838 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T07:02:45.862 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 06:10:14.354363', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-26T07:02:45.866 DEBUG:teuthology.nuke:shortname: smithi135 2023-10-26T07:02:45.867 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T07:02:45.893 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 06:10:14.355533', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-26T07:02:45.895 DEBUG:teuthology.nuke:shortname: smithi192 2023-10-26T07:02:45.895 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T07:02:45.918 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438471', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 06:10:14.352997', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-26T07:02:45.928 INFO:teuthology.orchestra.console.smithi089:Power off 2023-10-26T07:02:45.929 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-26T07:02:45.952 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2023-10-26T07:02:45.953 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:45.974 INFO:teuthology.orchestra.console.smithi135:Power off 2023-10-26T07:02:45.974 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-26T07:02:45.988 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-26T07:02:46.001 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off 2023-10-26T07:02:46.002 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:46.017 INFO:teuthology.orchestra.console.smithi192:Power off 2023-10-26T07:02:46.018 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-26T07:02:46.027 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2023-10-26T07:02:46.042 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2023-10-26T07:02:46.042 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:46.065 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-10-26T07:02:49.989 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:50.029 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:50.067 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:50.100 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-26T07:02:50.142 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2023-10-26T07:02:50.179 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-10-26T07:02:54.102 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:54.144 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:54.181 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:54.213 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-10-26T07:02:54.255 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2023-10-26T07:02:54.292 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-10-26T07:02:54.315 INFO:teuthology.orchestra.console.smithi089:Power off completed 2023-10-26T07:02:54.393 INFO:teuthology.orchestra.console.smithi192:Power off completed 2023-10-26T07:02:58.258 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T07:02:58.368 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off 2023-10-26T07:02:58.469 INFO:teuthology.orchestra.console.smithi135:Power off completed 2023-10-26T07:02:58.576 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} duration: 2649.9509320259094 failure_reason: 'Command failed (workunit test suites/ffsb.sh) on smithi089 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=384993681abceb8cb8affea10538d607052db4b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=bc459db9ebcf452688ad94ee3f52b7d9 status: fail success: false 2023-10-26T07:02:58.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-26T07:02:58.671 INFO:teuthology.run:FAIL